Concept | Variables in Dataiku#

Watch the video

In this article, we’ll introduce the concept of variables in Dataiku, including:

  • The benefits of variables in general.

  • Different types of variables in Dataiku.

  • Different components where they can be used (visual as well as code).

  • Most importantly, the syntax for defining and calling them.

Why variables?#

Programmers will already be familiar with the benefits of variables. They are a way to avoid hard-coding values that may change, and then can be reused in many places.

Spreadsheet users will understand the same principle. A formula points to a particular cell, whatever its actual value may be, rather than hard-coding the cell’s present value.

The motivation for using variables in Dataiku is the same.

Slide depicting the concept of variables for programmers and spreadsheet users.

Variable types#

Our focus here is primarily project variables, but note that there are a few different kinds of variables in Dataiku:

Variable type

Description

Instance-level

Variables accessible to administrators. These might be used to store an API key needed in several different projects on the instance.

Project-level

Variables for use anywhere in a project. We’ll focus on the global variety, but there are also project variables that remain local to the instance. In other words, they are not exported when bundling the project. (Think of API credentials that you don’t want to share).

Scenario-level

Variables that do not persist after a scenario ends.

Slide depicting types of variables in Dataiku.

Variable usage#

Variables can be used in many different components throughout Dataiku:

Component type

Examples

Visual

  • Visual recipes

  • Scenarios

  • Dataiku applications

Code

  • Notebooks and recipes

  • Custom metrics, checks, data quality rules, and scenarios

  • Webapps and plugins

Slide depicting components in Dataiku where variables can be used.

Variable syntax#

You define project variables from the … > Variables page from the top navigation bar.

Variables should be defined as a JSON object. That means every project variable should be defined as a key-value pair, separated by commas, and wrapped within curly braces.

Here are two examples of project variables — one defined as a string and the other an integer.

Slide depicting variable syntax in Dataiku.

Variables in visual components#

Once a variable is defined, you can call its value where it is needed. Let’s start with the example of a formula in a Prepare recipe.

When defining a formula, just type $ to pull up a list of accessible variables.

Dataiku screenshot of an example showing how to find available variables.

After making a selection, the editor includes the necessary curly braces around the variable name. That’s it! You now have replaced a hard-coded value with a variable.

Dataiku screenshot of an example showing how to use an integer variable in a Prepare recipe processor.

Variable evaluation#

It’s helpful though to have a better understanding of how Dataiku actually evaluates variables.

The dollar sign, and the following variable name within the curly braces, gets replaced with the value stored in the variable.

Let’s demonstrate by calling a variable stored as a string. When Dataiku evaluates ${my_state}, it replaces it with the value of the variable, in this case New York.

Dataiku screenshot of an example showing how a string variable in a Prepare recipe processor needs to be wrapped in quotes.

But now we have an unquoted string value in our formula. We need to add an extra set of quotation marks around the dollar sign and curly braces to re-quote the string value.

Dataiku screenshot of an example showing how to use a string variable in a Prepare recipe processor.

Now our formula is fixed and flags the intended rows!

Coding with variables#

In addition to features like visual recipes, you can also define and call variables from any place where you can write code within Dataiku. The syntax differs according to the programming language.

The syntax to use variables with SQL code is similar to that found in visual recipes. For example, the following code in the WHERE clause of an SQL query would call a variable your_variable_name.

WHERE "column" = '${your_variable_name}'

See also

To learn more about using SQL with Dataiku, see the Academy course Dataiku & SQL.

What’s next?#

Now that you know the basics, you can start using variables to make your work more efficient in Dataiku!

Gain practice using variables in visual components or with code: