Version a deployed project#

As you monitor the health of deployments, you’ll need to deploy updated versions of projects over time — especially since this one is already failing!

Where to make changes to a project#

When it’s necessary to make a change to a deployed project, it’s critical to make all such changes in the development environment (the Design node). Only then should you push a new bundle to the production environment (the Automation node).

It may be tempting just to make a quick change to the project on the Automation node. However, you should avoid this temptation because the project in the production environment would no longer be synced with its counterpart in the development environment.

Consider a situation where you want to revert back to an earlier version of the project. If you’ve made changes in the Automation node, you will lose these changes. Accordingly, actual development should always happen in the Design node. You should push new versions of bundles from there.

Fix the failing scenario#

Taking this advice, return to the Design node project. You know the source of the problem is a data quality rule on the tx_prepared dataset. Fix it so the scenario can succeed.

  1. In the Design node project, open the tx_prepared dataset. If it’s empty, build it.

  2. Navigate to the Data Quality tab.

  3. Click Edit Rules.

  4. Select the Record count rule.

  5. Decrease the min to 5000, and turn Off the soft min.

  6. Click Run Test to confirm it returns OK.

Dataiku screenshot of a data quality rule.

Create a second bundle#

With the scenario’s problem fixed, update the existing deployment with a new bundle.

  1. From the More Options (Horizontal dots icon.) menu of the top navigation bar, select Bundles.

  2. Click + New Bundle.

  3. Name it v2.

  4. In the release notes, add Fixed data quality rule.

  5. Click Create.

Dataiku screenshot of the second version of the project bundle.

Note

The second bundle inherited the configuration of the first one. In this case, the second bundle already includes the uploaded dataset and the managed folder.

Deploy the new bundle#

The process for deploying the new bundle is the same as for the first one.

  1. Click on the newly created v2 bundle, and click Publish on Deployer.

  2. Confirm that you indeed want to Publish on Deployer.

  3. Click to Open in Deployer to view the bundle details on the Deployer.

  4. Once on the Deployer, click Deploy on the v2 bundle.

    You have the option to create a new deployment or update the existing one.

  5. Since this is a new version of an existing deployment, verify Update is selected, and click OK.

  6. Click OK again to confirm the deployment you want to edit.

Dataiku screenshot for updating a deployed bundle.

You’re not done yet!

  1. Navigate to the Status tab of the deployment, and note how Dataiku warns that the active bundle on the Automation node doesn’t match the configured bundle.

  2. Click the Update button to deploy the new bundle. Then Confirm.

  3. Navigate to the Deployments tab of the Project Deployer to see the new bundle as the currently deployed version of this project.

Activate a scenario from the Deployer#

Previously you activated the scenario directly from the Automation node project. Now control it from the Project Deployer.

  1. Open the deployment, and navigate to the Settings tab.

  2. Go to the Scenarios panel.

  3. Uncheck the box for Disable automatic triggers.

  4. Click Activate All to enable the auto-triggers for any scenario.

  5. Click Save and Update and then Confirm.

Dataiku screenshot of the scenarios panel of a deployment.

Tip

Once you’ve done this, verify the scenario in the v2 bundle ran successfully! You can check this on the Automation project or the Project Deployer.

Keeping in mind the synchronization interval, you can also view this on the Unified Monitoring page. Fixing the data quality rule means the data status should be OK. In turn, the successfully running scenario means the execution status should be OK. Therefore, given that the model and governance statuses are empty, the global status should also be OK.

Revert to a previous bundle#

It’s also important to be able to revert to an earlier version, should a newer bundle not work as expected.

  1. From the Deployments tab of the Project Deployer, find the project in the left hand panel.

  2. Click Deploy next to the v1 bundle.

  3. With Update selected, click OK, and confirm this is correct.

  4. Now on the Settings tab with v1 as the source bundle, click the Update button, and Confirm the change.

Dataiku screenshot of the dialog to revert a bundle.

Important

Return to the Status tab of this deployment, or open the project homepage on the Automation node. You’ll see that v1 is once again the active bundle running in production.

Before signing off, be sure to Disable automatic triggers for this deployment either from the Project Deployer or the Automation project!

See also

See the reference documentation to learn more about reverting bundles.

Next steps#

Congratulations! To recap, in this tutorial, you:

  • Created a bundle in a project on the Design node.

  • Published a bundle to the Automation node via the Deployer.

  • Activated (and deactivated) a scenario to run on the Automation node.

  • Saw where to monitor the health of deployments.

  • Switched bundle versions within a deployment.

Now you have seen the batch processing framework for production. Your next step may be to examine the real-time API scoring method of production presented in the API Deployment course.

See also

For more information on batch processing, please refer to the reference documentation on Production deployments and bundles.