Concept | Scenarios#
Watch the video
Scenarios are key to automating tasks related to your Dataiku project. Let’s learn about the different types of scenarios and their various components.
Use cases#
Automation scenarios are a set of actions that are scheduled to run when certain conditions are satisfied. They are most useful when automating various kinds of tasks when a project is in production. For example:
If new data arrives on a regular basis, a scenario can rebuild the Flow once per day or each time it detects a dataset change.
If a metric for a machine learning model falls outside a specified threshold range, a scenario can be triggered to retrain the model.
Scenarios can also automate administrative tasks such as cleaning logs or starting and stopping a cluster.
Scenario types#
There are two types of scenarios in Dataiku.
Step-based, where scenario steps are configured using the visual interface.
Code-based, where the set of actions performed are fully defined by Python code.
Note
Learn more about custom Python script scenarios in our article on custom scenarios.
Scenario components#
Scenarios consist of three main components.
Steps that are actions configured by the user.
Triggers that define when to execute a scenario.
Reporters that send information or alerts about a scenario via a variety of channels.
Scenario steps#
Scenario steps let you control what the scenario will do. Common scenario steps include:
Building or clearing a dataset.
Training a model.
Running metrics and checks.
Sending messages.
Refreshing the cache of charts and dashboards.
Scenario steps run sequentially. However, you can control whether a step runs based on the outcome of a check. Therefore, you have some control over the flow logic of a scenario using metrics and checks.
Note
All available scenarios steps are defined in the reference documentation.
Scenario triggers#
Triggers allow users to define a condition or set of conditions that, if satisfied, start a scenario. Each trigger can be enabled or disabled.
Trigger |
Description |
---|---|
Time-based |
This will launch the scenario at regular intervals. Example: Repeat every 30 minutes. |
Dataset change |
This starts a scenario whenever a change is detected in the dataset. This type of trigger is used for filesystem-based datasets. |
SQL query change |
This runs a query at a specified interval and starts the scenario when the output of the query changes with respect to the last execution of the query. |
Custom (Python) |
This will execute a custom Python script that activates a trigger. |
Note
Different types of triggers may be available depending on your license.
Reporters#
Dataiku lets you add reporters to a scenario to inform users about scenario activities through email and other channels. Reporters can be sent when a scenario starts or ends on the condition that it succeeds or fails.
Reporters operate through several channels, including:
Mail
Slack
Microsoft Teams
Webhook
Twilio
Shell command
Send to dataset
What’s next?#
To learn more about scenarios and try hands-on tutorials, please register for the free Academy course on this subject found in the Advanced Designer learning path.
Note
You can also find more information about scenarios in the reference documentation.