Donkey

In this demonstration, we are going to take a CSV file called employee.csv and transform each record in the file into a SQL statement that we will write to an output file called employee.sql. This is really basic and not very realistic, but it demonstrates how to configure the basic moving parts of the Donkey.

The demonstration test consists of the following parts:

• Input data: employee.csv
• Expected output: employee-insert.sql
• The test itself: CsvToSqlDemoTest.java
• A specification class: CsvToSqlSpecification.java
• A wiring class: CsvToSqlWirer.java
• A properties file: csv2sql.properties

Let's go through each part in turn. Basically, we want to turn this...

employee.csv

...into this...

employee-insert.sql

The CsvToSqlDemoTest.java contains code that does this. This test launches the Donkey, and checks that the SQL file output matches the expected results. Pretty simple really.

CsvToSqlDemoTest.java

You will perhaps notice that the Donkey is launched via a Trebuchet. A Trebuchet takes a Specification.

Specification

A Specification contains 2 things.

• A config object containing all the configuration for the job. Things like input directories, database credentials, and output file names.
• A wirer class. Wirers contain class configuration information for dependency injection. Normally, they specify a Slurper (to read data), a Transformer (used by the Gargler to transform and map data), and a Spitter (a destination to write the transformation result). There is more about Wirers below.

Here is the Specification class for this demonstration test...

CsvToSqlSpecification.java

Wirer

CsvToSqlWirer.java

This file contains dependency injection configuration. The Donkey uses a field based injection framework called the Spider. It has been created by the developers of Boost and the folks at Leapstream (disclaimer: Me is one).

These wiring instructions specify a Job, a Slurper, a Transform (used by a Gargler) and a Spitter. There are two types of wiring instructions here, Dna and Wire. Using Dna is a way of wiring up implementations that contain delegates that implement the same interface as the calling class. This is sometimes referred to as "chaining". Information on using Wire can be found on the Java Boost web site.

Wiring is the trickiest bit in the whole setup as it is impossible to tell what to do without a clear idea of what each Slurper, Transform, Spit implementation does. We acknowledge this and are trying to improve and document every implementation.

The configuration required by each of these implementations is contained in the csv2sql.properties.

csv2sql.properties

The input.filename is required by the CsvSlurper. The rest of the parameters are required by the FileSpitter. You will note that the output.template contains token variables (delimited by "@" symbols) that match the column names in the CSV file. These tokens are replaced by record values from the CSV file when the output file is written.

This concludes the walk-through of the CSV to SQL Demonstration Test. Check out the code and run the CsvToSqlDemoTest.java for yourself. Rock.