Deploying to Production

Once you have designed a flow and automated updates to the flow, you can deploy it to a production environment.

Before jumping into the hands-on portion of the tutorial, watch the following video, which walks through an outline of the steps.

Note

Development and Production environments

A development (or sandbox) environment is an environment where you test new analyses in your project. Failures in this environment are an expected part of its experimental nature.

A production environment is where serious operational jobs are run. This environment should be available whenever necessary and may serve external consumers for their day-to-day decisions, whether those consumers are humans or software. Failure is not an option in production, and the ability to roll-back to a previous version is critical.

Dataiku provides two dedicated nodes to handle development and production:

  • Dataiku Design Node is used for the development of data projects.

    • It provides capabilities for the creation of data pipelines amd models, plus the definition of how they are meant to be reconstructed. Projects developed in the Design Node are packaged and handed off to the Automation Node.

  • Dataiku Automation Node is used to import packaged projects defined in the Design Node and run them in the production environment.

    • When you make updates to the project in the Design node, you can create an updated version of the project package, import the new package into the Automation node, and control which version of the project runs in production.

Development work from the Design node flows to the Automation node, and while it is technically possible to make changes to a project in the Automation node, those changes don’t flow back to the Design node, so it’s best practice to do all development in the Design node.

Let’s Get Started!

In this tutorial, you will learn how the Design and Automation Nodes work together:

  • Packaging flows for deployment

  • Versioning flows

  • Deploying packages in a production environment

We will work with the fictional retailer Haiku T-Shirt’s data.

Prerequisites

This tutorial assumes that:

  • You have completed the Automation course.

  • You have access to both a Dataiku Design node and a Dataiku Automation node.

Create Your Project

From the homepage of the Dataiku Design node, click +New Project > DSS Tutorials > Automation > Deployment (Tutorial).

For the purposes of this tutorial, the flow and automation scenarios are complete, and we simply need to package the flow and deploy it to the Automation node.