Concept: API Query Enrichments

Let’s discuss API query enrichment in Dataiku.

Tip

This content is also included in a free Dataiku Academy course on Real-Time APIs, which is part of the MLOps Practitioner learning path. Register for the course there if you’d like to track and validate your progress alongside concept videos, summaries, hands-on tutorials, and quizzes.

Query Enrichment

In some cases, an API request to an endpoint will not include all of the features required for processing the request. In such cases, a query enrichment, that is, a lookup on another table, may be able to retrieve the additional features needed for the endpoint to process the request and return a valid response.

Slide depicting the concept of query enrichment.

For example, in a credit card fraud detection use case, at the time of the transaction, the client making the API request could know the transaction amount, but might not have any information on the cardholder’s FICO score, a measure of creditworthiness in the US.

Since the features that are missing from the API request, such as the FICO score, are stored in the company’s internal database, we can use an enrichment to retrieve these features and pass them to the prediction endpoint. This way, the endpoint will have all the features needed for processing the request and return a valid response.

Slide depicting an API query enrichment for a credit card fraud example.

Configure an Enrichment

You can configure query enrichments in the Enrichment tab of the API designer. In our credit card fraud example, we’ll enrich each query with a few additional features about our cardholders.

Enrichment data can either be bundled or referenced.

  • A bundled data deployment policy means that the contents of the lookup table are copied and deployed as part of the API service on the API node. This mode is generally preferred when the lookup table is fairly small.

  • A referenced data deployment policy means that the data for the lookup table is not managed by the API node. When you export an API service from Dataiku, the service does not contain the data. It only contains a reference to the original dataset, such as a connection name and a table name.

Slide depicting enrichment deployment policy.

Once the enrichment is configured, you can try running a few test queries that have missing features, so that these missing features can be retrieved through enrichment. If you’ve selected the option to “return post-query enrichment” in the “Advanced” tab, you will see the results of the enrichment in the “Details” section of each query.

Configure settings for data enrichment.

What’s Next?

To get hands-on practice on using endpoints and enrichments, follow the Hands-On Tutorial: Create Endpoint and Test Queries.