Setting a Code Environment

This tutorial describes how to set Python or R code environments for plugins, projects, and recipes (or other objects within a project).

Dataiku DSS code environments address the problem of managing dependencies and versions when writing code in R and Python. They are similar to the Python virtualenv.

Note

Code environments provide a number of benefits, including:

• Isolation. Two teams can work independently on different projects using different versions of Python (or R) and a set of libraries whose versions differ.

• Reproducibility of results. When you create a project bundle or API service package and push it to production, Dataiku DSS includes the specification for the project’s code environment, and then rebuilds the code environment according to that specification when you import the bundle into the Dataiku Automation node or the package into the Dataiku API node. In this way, environments are versioned on your production server and you can rollback your code to a previous version together with its code environment.

Creating New Code Environments

Your Dataiku DSS administrator, or any user in a group with the proper permissions, can create different code environments , which are then available for you to use.

Using Code Environments

A separate code environment can be specified for many Dataiku objects in the same project.

Setting a Project-Level Code Environment

By default, projects inherit the code environment according to the global settings of the instance (Administration > Settings > Misc.). Unless otherwise specified, this is the DSS builtin environment.

• From the top navigation bar, go to … > Settings > Code env selection.

For the default Python or R code env:

• Change the mode to Select an environment.

• A dropdown appears that allows you to select a different environment from those already created.

If there is an environment you expect to see that is missing, contact your administrator. They may need to create a new code environment or give you permission to use an existing one.

Tip

If you plan to use visual machine learning, the project-level code environment must include the scipy, scikit-learn, jinja2 and xgboost packages.

Setting a Code Environment in a Recipe

By default, Python and R recipes use the project’s code environment. For each recipe, you can set a different code environment to be used when processing code within that recipe.

• On the Advanced tab of a recipe, find the Python or R environment panel.

• Adjust the “Selection behavior” to select an environment as needed.

Setting a Code Environment in a Notebook

By default, Python and R notebooks use the project’s code environment. For each notebook, you can set a different code environment to be used when processing code within that notebook.

You can set the code environment at notebook creation time, or by changing the kernel (Kernel > Change kernel from the notebook menus).

Setting a Code Environment in a Webapp or R Markdown Report

By default, Python Bokeh webapps, R Shiny webapps, and R Markdown reports use the project’s code environment. For each of these objects, you can set a different code environment to be used when processing code within the object.

While in Edit mode, on the Settings tab, you can set the code environment using the Code env dropdown.

Setting a Code Environment in a Plugin

The plugin developer defines the code environment specification as part of the plugin. After installing a plugin that contains a code environment definition, you are prompted to create a code environment for the plugin.

Setting a Code Environment in Visual Machine Learning

Dataiku DSS Visual Machine Learning allows you to create custom models using Python, in addition to the built-in models. The Python code environment to be used for training those custom models can be set in the Runtime environment panel of the Design tab.

Setting a Code Environment in Other Dataiku Objects

There are many other places in Dataiku DSS where you can use custom code! For example, you can insert Python code in custom scenario steps and triggers, dataset metrics and checks, as well as custom models.

For DSS objects that are not focused on code, but accept custom code, the dropdown list is typically placed near the custom code.