Prepare data

See a screencast covering this section’s steps

Once you have a sense of the dataset, you’re ready to start cleaning and creating new features using a Prepare recipe.

Important

Recipes in Dataiku contain the transformation steps, or processing logic, that act upon datasets. They can be visual or code (Python, R, SQL, etc).

1. From the job_postings dataset, click the Actions () tab in the right panel.

2. From the menu of visual recipes, select Prepare.

3. Click Create Recipe, accepting the default output name and storage location for your instance.

Add a step from the processor library

You can add steps to the empty script of the Prepare recipe in many different ways. Let’s start with the processor library, the complete collection of available data transformation steps.

1. Click + Add a New Step to open the processor library.

2. Search for and select the Split column processor.

3. In the step, provide location as the column and , as the delimiter.

4. Check the box to Truncate, and set the maximum number of columns to keep to 3.

5. Observe the preview output columns highlighted in blue.

Add a step from the column header

You can also add frequent and suggested steps directly from the column header.

1. Still in the Script tab of the Prepare recipe, click on the location_0 column header to open a dropdown menu of actions.

2. Select Rename.

3. Enter country as the new name, and click OK.

4. Repeat this process for the columns location_1 and location_2, entering the new names state and city.

5. When finished, you’ll have one step with three renamed columns.

Add a step to multiple columns

It’s also possible to add certain steps to multiple columns at once.

1. Switch from the Table view to the Columns view.

2. Check the box next to four natural language columns: company_profile, description, requirements, and benefits.

3. Click Actions to open the dropdown.

4. From the menu, choose Simplify text.

5. In the script, observe four new steps, each one normalizing a natural language column.

Add a Formula step

Dataiku also has its own spreadsheet-like formula language. This formula language includes common mathematical and string operations, comparison and logical operators, and conditional statements that you can use to create new columns.

1. Switch back to the Table view.

2. Click + Add a New Step.

3. Search for and select a Formula step.

4. In the Formula for field, name the output column len_company_profile.

5. In the empty expression field, start typing length(company_profile), noting the availability of auto-completion.

6. Click Apply.

7. Observe the new column preview highlighted in blue.

Duplicate a step

The Prepare recipe has many other conveniences, such as grouping, coloring, commenting, duplicating, and copy-pasting steps.

1. Click the horizontal three dots () at the right of the last step.

2. Click Duplicate step.

3. Change the output column name to len_description.

4. Change the expression to length(description).

Run the Prepare recipe

You could definitely keep preparing this dataset, but let’s stop here. This recipe is a repeatable record of transformation steps that you can apply to the entire input dataset whenever required.

1. Click the Run button (or type the keyboard shortcut @ + r + u + n).

2. When the job finishes, click Explore dataset job_postings_prepared.

3. After inspecting the output dataset, navigate back to the Flow (g + f).