Manage multiple versions of an API service#

Now that you are comfortable deploying and updating API services, let’s explore how to manage multiple versions of an API endpoint for A/B testing.

Deploy multiple versions of the endpoint for A/B testing#

When you’ve deployed multiple versions of your prediction endpoint, you may decide to run multiple generations of the endpoint at once. This allows the multiple versions of your prediction model to be used at random for scoring requests to the API service.

On the API Deployer, return to the Deployments tab.
In the main panel, click to open the active deployment of the fraud_detection API service.
Navigate to the Settings tab of the deployment.
Within the General panel, change the Active version mode from Single generation to Multiple generations.

In the Entries field, we need to define a mapping to enable multiple versions of the endpoint. The mapping is in JSON format and specifies one or more generations and their corresponding probabilities. The probabilities indicate the likelihood that calls to the API node will be served to each of the generations. Therefore, the sum of the probabilities of all generations must equal one.

Let’s direct 70% of calls to the v2 version and the remaining 30% to the new v3 version.

Copy-paste the JSON array below into the Entries field:

[
   {
      "generation": "v2",
      "proba": 0.7
   },
   {
      "generation": "v3",
      "proba": 0.3
   }
]

Click the Save and Update button to update the API service with the new settings.
Select the default Light Update option.

Important

Setting up multiple generations must be done manually and cannot be done through the automated deployment of API services.

Test multiple generations of the endpoint#

Let’s test it to confirm that queries will be sent to different generations of the API endpoint.

Navigate to the Status tab of the deployment.
Go to the Run and test subtab of the predict_fraud endpoint.
Click Run All to run the test queries.
Click the Details of each response to see which version of the API endpoint generated the response.

Rollback versions#

The Project Deployer enables you to switch or rollback a deployment to previous project bundles. The API Deployer allows the same for API services. However, the ability to activate multiple generations of an API lets you take this a step further.

Here for example, the v3 version of the API service is actually deploying v2 and v3 with a certain probability. Still, we can always revert to an earlier version, just like on the Project Deployer.

From the API Deployer, select Services from the top navigation bar.
Select the fraud_detection service ID.
Next to the v1 version, click Deploy.
Click OK to confirm the update.
Click OK again.
Click the green Update button, and select the default Light update.

Monitor an API service#

Whether you have single or multiple generations of an API service, you’ll need to be able to monitor all API services in production. Monitoring an API service has many facets to it, including model evaluations, model comparisons, and drift analysis. We’ll address this topic in the Production Monitoring course.

For now, we can take a quick peak at the currently-deployed API service in the Unified Monitoring page.

From the API Deployer, click Deployer to go to the Deployer home.
Select Monitoring.
Navigate to the API Endpoints tab.

Important

Depending on how quickly you’ve made it through these steps, you may not yet see a monitored API endpoint. The default synchronization interval is five minutes. Learn more in the reference documentation on Unified Monitoring.

Tip

Why is the model status empty? As explained in the reference documentation on Understanding Model Status, we deployed this model without any checks in the model evaluation store. Contrast this outcome with what will occur in Tutorial | Model monitoring with a model evaluation store.

What’s next?#

Congratulations! Let’s recap all that you’ve accomplished:

We created an API endpoint from a prediction model in the Flow.
We packaged this API endpoint in an API service.
We published the API service on the API Deployer.
We deployed the API service to an API node, where it could answer live requests.
We added a query enrichment to the API endpoint and redeployed the API service.
We added a second endpoint (a dataset lookup) to an API service.
We deployed multiple versions of an API service simultaneously.

Although we’ve deployed an API service into production, it is not yet properly monitored. To do so, begin exploring resources in the Production Monitoring course.