Version a deployed project#

As you monitor the health of deployments, you’ll need to deploy updated versions of projects over time — especially since this one is already failing!

Where to make changes to a project#

When it is necessary to make a change to a deployed project, it’s critical to make all such changes in the development environment (the Design node), and then push a new bundle to the production environment (the Automation node).

It may be tempting just to make a quick change to the project on the Automation node, but you should avoid this temptation, as the project in the production environment would no longer be synced with its counterpart in the development environment.

Consider a situation where you want to revert back to an earlier version of the project. If you’ve made changes in the Automation node, these changes will be lost. Accordingly, actual development should always happen in the Design node, and new versions of bundles should be pushed from there.

Fix the failing scenario#

Taking this advice, let’s return to the Design node project. We know the source of the problem is a data quality rule on the tx_prepared dataset. Let’s fix it so the scenario can succeed.

  1. In the Design node project, open the tx_prepared dataset.

  2. Navigate to the Data Quality tab.

  3. Click Edit Rules.

  4. Select the Record count rule.

  5. Decrease the min to 5000 and turn Off the soft min.

  6. Click Run Test to confirm it returns OK.

Dataiku screenshot of a data quality rule.

Create a second bundle#

With the scenario’s problem fixed, let’s update the existing deployment with a new bundle.

  1. From the More Options (…) menu of the top navigation bar, click Bundles.

  2. Click + New Bundle.

  3. Name it v2.

  4. In the release notes, add Fixed data quality rule.

  5. Click Create.

Dataiku screenshot of the second version of the project bundle.

Note

Note how when creating the second bundle, the configuration of the previous one is inherited. In this case, the uploaded dataset and the managed folder are already included.

Deploy the new bundle#

The process for deploying the new bundle is the same as for the first one.

  1. Click on the newly-created v2 bundle, and click Publish on Deployer.

  2. Confirm that you indeed want to Publish on Deployer.

  3. Click to Open in Deployer to view the bundle details on the Deployer.

  4. Once on the Deployer, click Deploy on the v2 bundle.

    Dataiku gives the option to create a new deployment or update the existing one.

  5. Since this is a new version of an existing deployment, verify Update is selected, and click OK.

  6. Click OK again to confirm the deployment you want to edit.

Dataiku screenshot for updating a deployed bundle.

We’re not done yet!

  1. Navigate to the Status tab of the deployment, and note how Dataiku warns that the active bundle on the Automation node does not match the configured bundle.

  2. Click the green Update button to deploy the new bundle. Then Confirm.

  3. Navigate to the Deployments tab of the Project Deployer to see the new bundle as the currently deployed version of this project.

Dataiku screenshot of the Deployer showing a second version of the deployment.

Activate a scenario from the Deployer#

Previously we activated the scenario directly from the Automation node project. Now let’s control it from the Project Deployer.

  1. Open the deployment, and navigate to the Settings tab.

  2. Go to the Scenarios panel.

  3. Uncheck the box for Disable automatic triggers.

  4. Click Activate All to enable the auto-triggers for any scenario.

  5. Click Save and Update and then Confirm.

Dataiku screenshot of the scenarios panel of a deployment.

Tip

Once you’ve done this, verify the v2 scenario successfully! You can check this on the Automation project, the Project Deployer, or on the Unified Monitoring page (keeping in mind the synchronization interval).

Revert to a previous bundle#

It’s also important to be able to revert to an earlier version, should a newer bundle not work as expected. Let’s demonstrate that now.

  1. From the Deployments tab of the Deployer, find the project in the left hand panel.

  2. Click Deploy next to the v1 bundle.

  3. With Update selected, click OK, and confirm this is correct.

  4. Now on the Settings tab with v1 as the source bundle, click the green Update button, and Confirm the change.

Dataiku screenshot of the dialog to revert a bundle.

Important

If you return to the Status tab of this deployment, or open the project homepage on the Automation node, you’ll see that v1 is once again the active bundle running in production.

Before signing off, be sure to disable automatic triggers for this deployment either from the Project Deployer or the Automation project!

See also

See the reference documentation to learn more about reverting bundles.

What’s next?#

Congratulations! To recap, in this tutorial, you:

  • Created a project bundle on the Design node.

  • Published a bundle to the Automation node via the Deployer.

  • Activated (and disabled) a scenario to run on the Automation node.

  • Saw where to monitor the health of deployments.

  • Switched bundle versions within a deployment.

Now that you have seen the batch processing framework for production, your next step may be to examine the real-time API scoring method of production presented in the API Deployment course.

See also

For more information on batch processing, please refer to the reference documentation on Production deployments and bundles.