Concept | Recipes in Dataiku#

Recipes in Dataiku contain the transformation steps, or processing logic, that act upon datasets.

Recipes in the Flow#

In the Flow, blue squares represent datasets. The yellow, orange, pink, and red circles, on the other hand, which connect datasets to one another, represent recipes.

Keeping the processing logic separate from datasets has a number of benefits:

One is that data storage technologies rapidly change. When this happens, you can change the underlying storage infrastructure of a dataset (for example, switching cloud providers) without impacting the processing logic found in the recipes of a Flow.
Another is a clear sense of data lineage in a project. By looking at the Flow, you can see all actions that have been applied to the data recorded in recipes – from the raw imported data to the final output dataset.

Recipe types#

A circle in the Flow represents a recipe, but its color represents the category of recipe. Dataiku recipes can be divided into visual, code, and plugin recipes.

Recipe type	Description
Visual recipes (Yellow circles)	Visual recipes accomplish the most common data transformation operations through a pre-defined graphical user interface. Instead of coding data transformations yourself, you can apply a visual recipe to guide tasks such as cleaning, grouping, or filtering data.
Code recipes (Orange circles)	At any time, instead of using a pre-defined visual recipe, you are free to define your own processing logic in a code recipe, using a language such as Python, R, or SQL.
LLM recipes (Pink circles)	LLM recipes leverage large language models to facilitate tasks such as prompt engineering, classification, summarization, building a custom Questions & Answers application using RAG - Retrieval Augmented Generation.
Plugin recipes (Red circles)	Plugin recipes are a way for coders to extend the native capabilities of Dataiku through plugins. Caution Be aware that plugin recipes only appear after manually installing the relevant plugins. If code recipes give you complete freedom to perform any data processing task, and visual recipes can be used and understood by everyone in your team, a plugin recipe combines these benefits by wrapping a visual interface on top of a code recipe.

Recipe type

Description

Visual recipes

(Yellow circles)

Visual recipes accomplish the most common data transformation operations through a pre-defined graphical user interface.

Instead of coding data transformations yourself, you can apply a visual recipe to guide tasks such as cleaning, grouping, or filtering data.

Code recipes

(Orange circles)

At any time, instead of using a pre-defined visual recipe, you are free to define your own processing logic in a code recipe, using a language such as Python, R, or SQL.

LLM recipes

(Pink circles)

LLM recipes leverage large language models to facilitate tasks such as prompt engineering, classification, summarization, building a custom Questions & Answers application using RAG - Retrieval Augmented Generation.

Plugin recipes

(Red circles)

Plugin recipes are a way for coders to extend the native capabilities of Dataiku through plugins.

Caution

Be aware that plugin recipes only appear after manually installing the relevant plugins.

If code recipes give you complete freedom to perform any data processing task, and visual recipes can be used and understood by everyone in your team, a plugin recipe combines these benefits by wrapping a visual interface on top of a code recipe.

What’s next?#

In this article, you learned about recipes and how they can be used to keep processing logic separate from datasets used in a Dataiku project. Continue getting to know the basics of Dataiku by learning about the Prepare recipe.

Tip

You can find this content (and more) by registering for the Dataiku Academy course, Visual Recipes. When ready, challenge yourself to earn a certification!