Concept | API endpoints¶
Let’s discuss API endpoints and what kinds are available in Dataiku. Then we’ll see how to create a particular kind of endpoint and test it.
This content is also included in a free Dataiku Academy course on Real-Time APIs, which is part of the MLOps Practitioner learning path. Register for the course there if you’d like to track and validate your progress alongside concept videos, summaries, hands-on tutorials, and quizzes.
An API service exposes one or more endpoints. An endpoint is a single path on the API (that is, a URL to which HTTP requests are posted, and from which a response is expected).
Each endpoint fulfills a single function; therefore, an API service that provides several different functions necessarily contains several endpoints. For example, a banking API service that returns predictions for credit card fraud detection and also returns the probability of loan default would expose two different prediction endpoints.
Types of endpoints¶
You can have many different kinds of endpoints in an API service. Dataiku’s API Designer supports endpoints such as:
Prediction or Clustering endpoints to perform predictions or group clusters using models created with the visual ML tool. This endpoint can include optional data preparation steps that have been performed in the preparation script of the visual ML tool.
Custom prediction (Python or R) endpoint to perform predictions using a custom model developed in Python or R.
Python/R function endpoint to call specific functions developed in Python or R.
SQL query endpoint to perform parameterized SQL queries.
Dataset lookup endpoint to perform data lookups in one or more Dataiku datasets.
Create an endpoint¶
To demonstrate endpoint creation, we’ll create a prediction endpoint. Given a Dataiku project that contains a visual ML prediction model in the Flow, we’ll expose the model as a prediction endpoint.
To create API services and their endpoints, we use the API designer tool. Here, we can specify the kind of endpoint to create; in this example, a prediction endpoint.
Alternatively, a quick way to create a prediction endpoint is to select the deployed model directly from the Flow, and select the Create API option from the right-hand panel. This will open the API designer tool where the kind of endpoint will automatically be specified as a prediction endpoint.
Note that if the visual analysis used to create the prediction model, or a clustering model for that matter, includes data preparation steps (in the “Script” tab), these steps will be included in the endpoint that is part of the API service deployed to the API node. As a result, records sent as API calls will first be preprocessed according to the steps in the Prepare script of the visual analysis, before being passed to the model for scoring.
Test the endpoint¶
In the “Test queries” section of the API designer tool, you can write your own test query in JSON format, or you can specify to run a few random queries selected from a dataset.
When you run the test queries, the endpoint will output predictions for each test query, and you can view the details of the run to get the complete endpoint response.
For many types of endpoints, you’ll also want to perform a query enrichment on that endpoint. Learn more in the next lesson.