Version a deployed project#
As you monitor the health of deployments, you’ll need to deploy updated versions of projects over time — especially since this one is already failing!
Where to make changes to a project#
When it is necessary to make a change to a deployed project, it’s critical to make all such changes in the development environment (the Design node), and then push a new bundle to the production environment (the Automation node).
It may be tempting just to make a quick change to the project on the Automation node, but you should avoid this temptation, as the project in the production environment would no longer be synced with its counterpart in the development environment.
Consider a situation where you want to revert back to an earlier version of the project. If you’ve made changes in the Automation node, these changes will be lost. Accordingly, actual development should always happen in the Design node, and new versions of bundles should be pushed from there.
Fix the failing scenario#
Taking this advice, let’s return to the Design node project. There are actually three changes that should be made.
Edit the data quality rule#
We first need to fix the failing data quality rule. Instead of an error, let’s reduce it to a warning.
On the Design project, navigate to the Data Quality tab of the tx_prepared dataset.
Click the pencil to edit the “Record count is above 50000” rule.
Check the box to Auto-compute metric.
Turn off the Min setting, and turn ON a Soft min of
50000
to produce a warning instead of an error.Click Run Test to confirm the warning.
Click Save.
![Dataiku screenshot of a data quality rule.](../../../_images/batch-edit-rule.png)
Add a Compute metrics step to the scenario#
We can also add a step in the scenario to explicitly compute metrics.
Navigate to the Steps tab of the Data Refresh scenario.
Click Add Step
Select Compute metrics from the list of steps.
Drag the metrics step to the first position.
Click Add Dataset to Compute > tx_prepared > Add.
Click Save.
As a matter of good practice, click Run to make sure it returns the expected warning result.
![Dataiku screenshot of a compute metrics step in a scenario.](../../../_images/batch-scenario-metrics-step.png)
Enable the scenario trigger#
Finally, let’s turn on the time-based trigger, but not the auto-trigger for the scenario itself. This way, once we activate the scenario’s auto-triggers in the production environment, it will begin running.
Navigate to the Settings tab of the Data Refresh scenario.
Turn On the Time-based trigger.
Verify the Auto-triggers remain Off.
Click Save.
![Dataiku screenshot of scenario trigger settings.](../../../_images/batch-scenario-triggers.png)
Create a second bundle#
Now let’s demonstrate the process for updating an existing deployment with a new bundle.
From the Bundles page on the Design node project, click + New Bundle.
Name it
v2
.In the release notes, add
fixed scenario
.Click Create.
![Dataiku screenshot of the second version of the project bundle.](../../../_images/v2-bundle.png)
Note
Note how when creating the second bundle, the configuration of the previous one is inherited. In this case, the uploaded dataset and the managed folder are already included.
Deploy the new bundle#
The process for deploying the new bundle is the same as for the first one.
Click on the newly-created v2 bundle, and click Publish on Deployer.
Confirm that you indeed want to Publish on Deployer.
Click to Open in Deployer to view the bundle details on the Deployer.
Once on the Deployer, click Deploy on the v2 bundle.
Dataiku gives the option to create a new deployment or update the existing one.
Since this is a new version of an existing deployment, verify Update is selected, and click OK.
Click OK again to confirm the deployment you want to edit.
![Dataiku screenshot for updating a deployed bundle.](../../../_images/update-deployment.png)
We’re not done yet!
Navigate to the Status tab of the deployment, and note how Dataiku warns that the active bundle on the Automation node does not match the configured bundle.
Click the green Update button to deploy the new bundle. Then Confirm.
Navigate to the Deployments tab of the Project Deployer to see the new bundle as the currently deployed version of this project.
![Dataiku screenshot of the Deployer showing a second version of the deployment.](../../../_images/new-bundle-deployed.png)
Activate a scenario from the Deployer#
Previously we activated the scenario directly from the Automation node project. Now let’s control it from the Project Deployer.
Navigate to the Settings tab of the deployment.
Uncheck the box for Disable automatic triggers.
Click Activate All to enable the auto-triggers for any scenario.
Click Save and Update and then Confirm.
![Dataiku screenshot of the scenarios panel of a deployment.](../../../_images/batch-deployer-activation.png)
Tip
Once you’ve done this, verify the v2 scenario runs and produces a warning. You can check this on the Automation project, the Project Deployer, or on the Unified Monitoring page (depending on the synchronization interval).
Revert to a previous bundle#
It’s also important to be able to revert to an earlier version, should a newer bundle not work as expected. Let’s demonstrate that now.
From the Deployments tab of the Deployer, find the project in the left hand panel.
Click Deploy next to the v1 bundle.
With Update selected, click OK, and confirm this is correct.
Now on the Settings tab with v1 as the source bundle, click the green Update button, and Confirm the change.
![Dataiku screenshot of the dialog to revert a bundle.](../../../_images/batch-revert-bundle.png)
Important
If you return to the Status tab of this deployment, or open the project homepage on the Automation node, you’ll see that v1 is once again the active bundle running in production.
Before signing off, be sure to disable automatic triggers for this deployment either from the Project Deployer or the Automation project!
See also
See the reference documentation to learn more about reverting bundles.
What’s next?#
Congratulations! To recap, in this tutorial, you:
Created a project bundle on the Design node.
Published a bundle to the Automation node via the Deployer.
Activated (and disabled) a scenario to run on the Automation node.
Saw where to monitor the health of deployments.
Switched bundle versions within a deployment.
Now that you have seen the batch processing framework for production, your next step may be to examine the real-time API scoring method of production presented in the API Deployment course.
See also
For more information on batch processing, please refer to the reference documentation on Production deployments and bundles.