Tutorial | Real-time API basics#

Get started#

The framework demonstrated in the batch deployment tutorial works well for many use cases, such as creating periodic reports. For many other use cases though, we need to score records in real-time!

For this tutorial, you’ll start by working with a prediction model already deployed in the Flow of a project on the Design node. You’ll use the model to create an endpoint that will be included in an API service for deployment to an API node.

Objectives#

In this tutorial, you will:

  • Package a prediction model as an API endpoint in an API service.

  • Deploy an API service from the Deployer to deployment infrastructure (an API node).

  • Add an enrichment to a prediction endpoint in an API service.

  • Create a dataset lookup endpoint in an API service.

  • Design a scenario that creates a new version of an API service and updates an existing API deployment when a certain condition is met.

  • Deploy multiple generations of a prediction endpoint to use for A/B testing.

Prerequisites#

  • You may also want to review this tutorial’s associated concept article.

Create the project#

We’ll start from a project that includes a basic classification model and a zone for scoring new, incoming data.

  1. From the Dataiku Design homepage, click +New Project > DSS tutorials > MLOps Practitioner > Real-time APIs.

  2. From the project homepage, click Go to Flow (or g + f).

Note

You can also download the starter project from this website and import it as a zip file.

You’ll next want to build the Flow.

  1. Click Flow Actions at the bottom right of the Flow.

  2. Click Build all.

  3. Keep the default settings and click Build.

Use case summary#

You’ll work with a simple credit card fraud use case. Using data about transactions, merchants, and cardholders, we have a Flow including a model that predicts which transactions should be authorized and which are potentially fraudulent.

  • A score of 1 for the target variable, authorized_flag, represents an authorized transaction.

  • A score of 0, on the other hand, is a transaction that failed authorization.

Putting this model into production can enable two different styles of use cases commonly found in machine learning workflows:

Scoring framework

Example

Batch

A bank employee creates a monthly fraud report.

Real-time

A bank’s internal systems authorize each transaction as it happens.

Tip

This use case is just an example to practice monitoring and deploying MLOps projects into production. Rather than thinking about the data here, consider how you’d apply the same techniques and Dataiku features to solve problems that matter to you!

Create an API endpoint and test queries#

Let’s start by creating an API service with one of the most common types of endpoint: a prediction endpoint.

Create an API service with a prediction endpoint#

To get started, we need a prediction model deployed to the Flow.

  1. From the project Flow, select the prediction model Predict authorized_flag (binary).

  2. In the Actions tab on the right, click Create API.

  3. In the New scoring API from model dialog, name the API service ID fraud_detection.

    Important

    The API service ID must be unique. If an API service with the same name already exists in your project, consider adding a unique identifier (such as your name) to the API service ID.

  4. Name the Endpoint ID predict_fraud. This ID should be unique within the API service.

  5. Click Append to open the API designer.

Create an API service with a prediction endpoint.

Test the prediction endpoint#

The fraud_detection API service now includes the model as an endpoint named predict_fraud.

Next, we’ll run some test queries on records found in the test_scored dataset to confirm that the endpoint properly generates a prediction for whether a credit card transaction was fraudulent or not.

  1. If not already there, navigate to the API Designer from the More Options menu in the top navigation bar. Click fraud_detection to open the API service.

  2. Go to the Test queries panel on the left.

  3. Click +Add Queries.

  4. Choose to create 5 test queries from the test_scored dataset.

  5. Click Add.

Dataiku screenshot of the page for adding test queries to verify that the endpoint works.

Now let’s run them!

  1. Click Run Test Queries at the top right.

  2. Click the Details of the first query to see more details, such as the probabilities of the two outcomes according to the model.

Dataiku screenshot of output from test queries.

Note

The Logs tab (next to Details) provides information that can be useful for debugging and troubleshooting, in the event that something fails.

Deploy a real-time API service#

You’ve successfully designed and tested an API service with a prediction endpoint. The next step is to actually deploy this API service from the Design node into a production environment!

Publish the API service on the API Deployer#

Recall that our Flow has a prediction model to classify credit card transactions as fraudulent or not. We have packaged this model as a prediction endpoint in an API service, but this service only exists on the Design node (our development environment). It can’t answer real queries yet.

Important

The remaining steps require that you are connected to the API Deployer. Dataiku Cloud users can follow instructions for adding the API node extension. Administrators of self-managed Dataiku instances should follow the reference documentation.

The next step is to publish the API service from the Design node to the Deployer.

  1. If not already open, from the More options menu in the top navigation bar on the Design node, click on API Designer. Select the fraud_detection API service.

  2. Click the Publish on Deployer button.

  3. Click OK, keeping the default version id name (v1).

Dataiku screenshot of the dialog for deploying an API service.

Deploy the API service to the deployment infrastructure#

We now have the API service including the prediction endpoint on the Deployer, but to query the endpoint, we still need to deploy the API service to an infrastructure.

  1. If still on the Design node, from the Applications menu in the right-hand corner of the top navigation bar, click Local (or Remote) Deployer.

  2. On the Deployer homepage, select Deploying API Services.

  3. On the Deployments tab of the API Deployer, find the fraud_detection API service on the left, and click Deploy.

  4. If not already selected in the dialog, choose an infrastructure for the new deployment.

  5. Note the default Deployment ID (which takes the form <api-service-name>-on-<infrastructure-name>).

  6. Click Deploy.

  7. On the Status tab of the new deployment, click Deploy once again.

Dataiku screenshot of API Deployer node displaying services available for deployment.

You now have a prediction endpoint available to serve real-time API calls. Note the dedicated URL for this API endpoint.

Dataiku screenshot of an API endpoint available to serve real-time API calls.

Query the API endpoint#

You can now submit real queries to this service by calling the endpoint URL. The Sample code tab provides snippets for calling the API in various languages, such as Shell (cURL), Python, R, or Java.

  1. Within the Status tab of the prediction endpoint, navigate to the Sample code subtab.

  2. Copy-paste the Shell code into a terminal window.

  3. Execute the live query, and see the prediction for this record returned in real-time.

Dataiku screenshot of the sample code tab to test a prediction endpoint.

You can also run the test queries previously defined in the API Designer of your project.

  1. Still within the Status tab of the prediction endpoint, navigate to the Run and test subtab.

  2. Click Run All.

Now the same queries tested in the API Designer on the Design node have been run on the API node.

Optional: Copy the deployment to another infrastructure#

Important

This section requires having a second deployment infrastructure. If you don’t have this available, feel free to read, but skip completing this section.

When choosing an infrastructure in the previous step, you may have seen pre-configured lifecycle stages. An instance admin can modify these stages as desired, so your options may be different depending on the complexity of your organization’s deployment strategy.

Let’s imagine we have another stage of deployment infrastructure, and all tests on the first deployment infrastructure were successful. We now are ready to copy the existing deployment to a new pool of API nodes.

  1. Still within the Status tab of the API service, click Actions > Copy this deployment at the top right.

  2. Select a new infrastructure, keep the other default values, and click Copy.

The service is now ready to be deployed.

  1. Click Deploy again on the page that opens up for the deployment.

  2. Once the deployment has been updated successfully, click Deployments in the top navigation bar to see that the fraud_detection API service was deployed to the development infrastructure and then to the production infrastructure.

View deployed API service on dev and prod infrastructure.

Add an enrichment to a prediction endpoint#

You have successfully published an API service from the Design node to the API Deployer and then to a deployment infrastructure (an API node). This allowed you to receive a response from a query to a live prediction endpoint.

In many cases though, you may be missing information required for the model to score the request. Accordingly, let’s return to the Design node, and add an enrichment to the prediction endpoint within the existing API service. Then we’ll re-deploy a new version of the API service.

Add a query enrichment#

Note

For greater context on the concepts at work here, see our resources on API Query Enrichment or the reference documentation on enriching prediction queries.

The prediction model in the Flow was trained on six features from the training dataset. You can confirm this by opening the active version of the model, and navigating to the Features panel in the Model Information section.

Ideally, an incoming transaction to the API endpoint would have values for all six features. However, suppose at the time a transaction occurs, a merchant point of sale system sends values for only a subset of these features:

  • signature_provided

  • merchant_subsector_description

  • purchase_amount

  • merchant_state

We first need to retrieve the missing values for the features card_fico_score and card_age from our internal database, and then use these values to enrich the API queries.

Using the cardholder_info dataset in the project, we’ll use the card_id value of each real-time transaction to look up the corresponding values for fico_score and age, and then pass the complete feature set to the prediction model.

Dataiku screenshot of a dataset and lookup key to use for enrichment.
  1. From the API Designer page, open the fraud_detection API service, navigate to the Enrichments panel.

  2. Click + Add Enrichment.

  3. Select cardholder_info as the dataset to use for enrichment.

  4. Leave the default Bundled deployment policy.

Tip

If you want to try the referenced deployment policy, you’ll need an SQL connection, and so follow our resources on remapping connections.

Now let’s provide the lookup key and retrieve the two missing columns, keeping in mind the names of the columns used to train the model.

  1. Next to Lookup keys definition, click + Add Key, and select the internal_card_mapping column.

  2. Provide card_id as the name in the query for the lookup key.

  3. In Columns to retrieve, specify the two missing features to retrieve from the dataset: fico_score and age.

  4. Remap these columns to the names card_fico_score and card_age.

Configure settings for data enrichment.

Let’s change one more setting before we test the enrichment.

  1. Navigate to the Advanced panel of the API endpoint.

  2. Check the box Return post-enrichment to return a more verbose response to each API query.

Test the query enrichment#

To test the enrichment, we’ll use a query that includes only four of the six features that were used to train the prediction model.

  1. Navigate to the Test queries panel of the API endpoint.

  2. Click +Add Queries, and then Add to add 1 new empty query.

  3. For the new test Query 6, paste the following JSON code sample in the query window.

    {
      "features": {
          "card_id": "C_ID_23626074d5",
          "purchase_amount": 3704.17,
          "signature_provided": 0,
          "merchant_subsector_description": "luxury goods",
          "merchant_state": "Wyoming"
        }
    }
    
  4. Click Run Test Queries.

  5. Click Details in the API response for Query 6, and observe the values for card__fico_score and card_age despite them not being present in the query.

Dataiku screenshot of a test data enrichment for a prediction query.

To summarize, the enrichment uses the card_id to retrieve the missing features (card_fico_score and card_age) so that the model has all features needed to determine a prediction.

Note

You can also test the enrichment by modifying the JSON code for any of the previous test queries. To do this, delete all the features except for the four used in the JSON code sample above. When you run the test queries, you’ll notice that the endpoint returns the same prediction as before for the modified test query, even with the missing features.

Redeploy the API service#

Now that we’ve added an enrichment to the prediction endpoint, we need to redeploy a new version of the API service, just as if we had a new project bundle.

  1. From the fraud_detection API service, click the green Publish on Deployer button.

  2. Accept the default version ID (v2), and click OK.

  3. Open the API service on the API Deployer, and click Deploy on v2.

  4. In the Deploy version dialog, click OK to update the version used in the service.

  5. Click OK again to confirm on which deployment you want to edit.

  6. Now on the API service page, click the green Update button.

  7. Select the default Light Update.

  8. Navigate back to the Deployments tab of the API Deployer to confirm v2 is the new version.

Dataiku screenshot of the second version of an API service.

Add a dataset lookup endpoint to an API service#

Congratulations! You added an enrichment to a prediction endpoint, and redeployed the API service to the production environment.

A prediction endpoint though is just one of many kinds of supported endpoints. Now let’s see another kind of endpoint: a dataset lookup. In fact, this endpoint is essentially the enrichment step of a prediction query — but without passing the query to a prediction model.

Add a dataset lookup endpoint#

Suppose we want to send a query including a specific merchant_id to an endpoint in an API service and receive all information relating to this specific merchant from the database.

We can add an endpoint to the API service just for this purpose. The new endpoint will perform a dataset lookup in the merchant_info dataset, returning the other values in the dataset for a unique merchant_id.

  1. From the API Designer page on the Design instance, open the fraud_detection API service.

  2. Near the top left, click on the first endpoint to open the dropdown menu.

  3. Click Create new endpoint.

  4. Name the endpoint merchant_lookup.

  5. Select Dataset(s) lookup as the Endpoint type.

  6. Click Append.

Dataiku screenshot of a dialog for appending a dataset lookup endpoint to an API service.

Now we can define the enrichment.

  1. In the Lookups panel, click + Add Enrichment.

  2. Select merchant_info as the dataset to use for enrichment.

  3. Leave the default Bundled deployment policy.

  4. Next to Lookup keys definition, click + Add Key, and select the merchant_id column. The Name in query for this feature will be the same, and so we don’t need to remap the name.

  5. Click the check box next to Select all to retrieve all columns.

Dataiku screenshot of the configuration of a dataset lookup endpoint.

Test the dataset lookup endpoint#

The dataset lookup endpoint is now included in the fraud_detection API service and ready for use. But first, let’s test the endpoint to confirm that it works properly.

  1. Navigate to the Test queries panel of the merchant_lookup endpoint.

  2. Click + Add Queries, and copy-paste the following:

    {
      "data": {"merchant_id" : "M_ID_000572bc62"}
    }
    
  3. Click Run Test Queries, and observe the retrieved values for all of the columns in the merchant_info dataset:

Dataiku screenshot of a test query on a dataset lookup endpoint.

Note

See the reference documentation for more information about supported endpoints or the dataset lookup endpoint in particular.

Optional: Update an API deployment automatically#

You have now added a dataset lookup endpoint to an API service. Although we could manually redeploy a new version of this API service (as done previously), let’s next demonstrate how to automatically update an API deployment with a scenario.

Start with a Retrain model scenario#

Let’s start by duplicating a scenario that retrains the model if the data drift metric fails. To recall, this scenario retrains the model when our chosen metric in the model evaluation store exceeds the specified threshold.

  1. Navigate to the Scenarios page from the top navigation bar of the Design node.

  2. Check the box to the left of the Retrain Model scenario to open the Actions tab, and click Duplicate.

  3. Name the new scenario Retrain Model & Deploy API.

  4. Click Duplicate.

Add a Create API service version step#

In the current scenario, the step that retrains the model runs only if a previous step (in our case the MES check) fails. However, the box is ticked to reset the failure state, and so this scenario can continue with other steps.

Let’s proceed with creating a new version of the API service in cases where a new model is retrained.

  1. In the Retrain Model & Deploy API scenario, navigate to the Steps tab.

  2. Click Add Step.

  3. Select Create API service version from the Deployer section.

  4. Name the step Create auto_deploy_api service.

  5. Select fraud_detection as the API service.

  6. Name the New version id as auto_deploy_api.

  7. Check the box to Make version id. unique. This will suffix the version id with a unique identifier every time a new version is created.

  8. Name the Target variable versionid.

  9. Check the box to Publish on Deployer. The Target service field should automatically select the previously-deployed fraud_detection API service.

Dataiku screenshot of the scenario step to create an API service version.

Add an Update API deployment step#

As we have seen in the process for API deployment, once we have an API service, we need to deploy it. There’s a scenario step for this too!

  1. In the Retrain Model & Deploy scenario, click Add Step.

  2. Select Update API deployment from the Deployer section.

  3. Name the step Update auto_deploy_api.

  4. Provide the Deployment id, which takes the form of <project-key>-on-<infrastructure>. Click on the field or start typing to see available options.

  5. Set the New version id field to ${versionid}. Be sure to use the variable syntax here since this references the target variable in the previous step.

  6. Click Save.

Dataiku screenshot of a scenario step to update an API deployment.

Run the scenario & observe the outcome#

Let’s imagine that some specified unit of time has passed, triggering the scenario to run.

  1. Click Run to manually trigger the Retrain Model & Deploy API scenario on the Design node project.

  2. Switch to the Last Runs tab to observe its progress, including the two new steps just added.

With no new data in this situation, we already know the check on data drift in the model evaluation store will fail, and so we can anticipate the outcome.

  1. From the More options menu in the top navigation bar, return to the API Designer.

  2. Open the the fraud_detection API service.

  3. Navigate to the Versions tab to see a new auto_deploy_api version within the API service on the Design node.

Dataiku screenshot of the versions tab of an API service.

Now let’s check the API Deployer.

  1. From the Applications menu of the Design node, select Local (or Remote) Deployer.

  2. Select API Services.

  3. Confirm auto_deploy_api is the active version deployed.

Dataiku screenshot of the auto deployed API service.

You can also view the version history of this API service on the API Deployer.

  1. From the Versions to deploy panel on the left side of the API Deployments page, click auto_deploy_api in the fraud_detection API service.

  2. Notice a new active version of the saved model.

  3. Also note how all versions of the API service remain in the history. If needed, you can easily roll back to a previous version by simply clicking Deploy on the version you want.

See the version history of the API service.

Tip

Run the scenario again to see how the bundle ID increments to auto_deploy_api1, and so on.

Note

It should be stated again that this scenario is only for educational purposes rather than for live operation on a production use case. The same ideas for how to make this setup more robust can be applied here.

Manage multiple versions of an API service#

Congratulations! You’ve created a scenario that can automatically update an API deployment.

Now that you are comfortable deploying and updating API services, let’s explore how to manage multiple versions of an API endpoint for A/B testing.

Deploy multiple versions of the endpoint for A/B testing#

When you’ve deployed multiple versions of your prediction endpoint, you may decide to run multiple generations of the endpoint at once. This allows the multiple versions of your prediction model to be used at random for scoring requests to the API service.

  1. On the API Deployer, return to the Deployments tab.

  2. In the main panel, click to open the active deployment of the fraud_detection API service.

  3. Navigate to the Settings tab of the deployment.

  4. Within the General panel, change the Active version mode from Single generation to Multiple generations.

Dataiku screenshot of the active version mode of an API deployment.

In the Entries field, we need to define a mapping to enable multiple versions of the endpoint. The mapping is in JSON format and specifies one or more generations and their corresponding probabilities. The probabilities indicate the likelihood that calls to the API node will be served to each of the generations. Therefore, the sum of the probabilities of all generations must equal one.

Let’s direct 70% of calls to the auto_deploy_api version and the remaining 30% to v1.

  1. Copy-paste the JSON below into the Entries field:

    [
       {
          "generation": "auto_deploy_api",
          "proba": 0.7
       },
       {
          "generation": "v1",
          "proba": 0.3
       }
    ]
    
  2. Click the Save and Update button to update the API service with the new settings.

  3. Select either update option.

Note

Setting up multiple generations must be done manually and cannot be done through the automated deployment of API services.

Test multiple generations of the endpoint#

Let’s test it to confirm that some queries will be sent to a different generation of the API endpoint.

  1. Navigate to the Status tab of the deployment.

  2. Switch to the Run and test subtab of the predict_fraud endpoint.

  3. Click Run All to run the test queries.

  4. Click the Details of each response to see which version of the API endpoint generated the response.

Dataiku screenshot of test queries showing which version of the API endpoint generated the response.

What’s next?#

Congratulations! You successfully deployed multiple versions of a prediction endpoint in an API service to implement A/B testing.

To continue learning, see the tutorial on monitoring the output of API endpoints to learn how to set up a monitoring system that centralizes the logs from the API node and monitors the responses of endpoints.

See also

The reference documentation provides more information on managing versions of your endpoint.