Tutorial | Update an API deployment automatically (MLOps part 8)

The tutorial on bundle updates demonstrated how to automatically update a project deployment with a new bundle for use cases requiring a more sophisticated MLOps setup. This was for a batch scoring framework. For real-time scoring, we can implement a very similar approach, noting the same risks.

Instead of scenario steps to create a bundle and update a project deployment, we’ll use steps to create an API service version and update an API deployment.

Objectives

In this tutorial, you will:

  • Design a scenario that creates a new version of an API service and updates an existing API deployment when a certain condition is met.

Starting here?

This section requires having created the scenario from Part 1 and an existing API service deployment (as in Part 5), and so it’s required to complete those sections in order to reproduce the steps here.

Start with a Retrain model scenario

Let’s start by duplicating the scenario that retrains the model if the data drift metric fails. To recall, this scenario retrains the model when our chosen metric in the model evaluation store exceeds the specified threshold.

  • Navigate to the Scenarios page from the top navigation bar of the Design node.

  • Check the box to the left of the Retrain Model scenario to open the Actions tab, and click Duplicate.

  • Name it Retrain Model & Deploy API, and click Duplicate.

Add a Create API service version step

In the current scenario, the step that retrains the model runs only if a previous step (in our case the MES check) fails. However, the box is ticked to reset the failure state, and so this scenario can continue with other steps.

Let’s proceed with creating a new version of the API service in cases where a new model is retrained.

  • In the Retrain Model & Deploy API scenario, add a Create API service version step from the Deployer section.

  • Name it Create auto_deploy_api service.

  • Select fraud_detection as the API service.

  • Name the New version id as auto_deploy_api.

  • Check the box to Make version id. unique. This will suffix the version id with a unique identifier every time a new version is created.

  • Name the Target variable versionid.

  • Check the box to Publish on Deployer. The Target service field should automatically select the previously-deployed fraud_detection API service.

Dataiku screenshot of the scenario step to create an API service version.

Add an Update API deployment step

As we have seen in the process for API deployment, once we have an API service, we need to deploy it. There’s a scenario step for this too!

  • In the Retrain Model & Deploy API scenario, add an Update API deployment step from the Deployer section.

  • Name it Update auto_deploy_api.

  • Provide the Deployment id, which takes the form of <project-key>-on-<infrastructure>. Click on the field or start typing to see available options.

  • Set the New version id field to ${versionid}. Be sure to use the variable syntax here since this references the target variable in the previous step.

  • Click Save.

Dataiku screenshot of a scenario step to update an API deployment.

Run the scenario & observe the outcome

Let’s imagine that some specified unit of time has passed, triggering the scenario to run.

  • From the project on the Design node, click to manually Run the Retrain Model & Deploy API scenario.

  • Observe its progress in the Last Runs tab of the scenario.

With no new data in this situation, we already know the check on data drift in the model evaluation store will fail, and so we can anticipate the outcome.

First, there should be a new auto_deploy_api version within the API service on the Design node.

  • Navigate to the Versions tab of the fraud_detection API service to see it.

Dataiku screenshot of the versions tab of an API service.

Second, in the Deployments tab of the API Deployer, notice that the active version of the API service on the development infrastructure is the one that was just deployed by the scenario.

Dataiku screenshot of the auto deployed API service.

You can also view the version history of this API service on the API Deployer.

  • From the Versions to deploy panel on the left side of the API Deployer, click auto_deploy_api in the fraud_detection API service.

See the version history of the API service.

There, you can see that all versions of the API service remain in the history. If needed, you can easily roll back to a previous version by simply clicking Deploy on the version you want.

Tip

Run the scenario again to see how the bundle ID increments to auto_deploy_api1, and so on.

Note

It should be stated again that this scenario is only for educational purposes rather than for live operation on a production use case. The same ideas for how to make this setup more robust can be applied here.

Next steps

Congratulations! You’ve created a scenario that can automatically update an API deployment.

Now that you are comfortable deploying and updating API services, let’s explore how to manage multiple versions of an API endpoint for A/B testing.

Note

For more information, please refer to the reference documentation on MLOps or API deployments.