Concept | Checks#

Important

In Dataiku versions 12.6 and above, checks for datasets have been replaced by data quality rules. Checks on folders, models, and model evaluation stores covered in this article are available in all versions of Dataiku.

Checks monitor the measurements, or metrics, on certain Flow objects — managed folders, saved models, or model evaluation stores.

Metrics are measurements on the object, such as the size of a folder or accuracy of a model. Checks then use the latest metric measurement to monitor the status of the item.

For instance, we could use checks to verify that:

  • The size of a folder does not exceed 3GB.

  • The model accuracy does not fall below 0.8.

Examples of metrics and checks used on managed folders, saved models, or model evaluation stores.

Checks will return one of four outputs:

Output

Meaning

OK

The rule outcome satisfied the set condition.

Error

The rule condition is not respected or the computation itself failed.

Warning

The rule fails a soft condition but not a hard one.

Empty

The rule cannot be computed.

Configuring checks#

You can configure checks in the Settings tab of saved models and model evaluation stores or in the Status tab of managed folders.

Example of a check on the AUC of a saved model.

Note

You can create custom checks with Python code. See the documentation here to learn more.