Concept | ML assertions#

Watch the video

When validating machine learning models, it is common to perform sanity checks to ensure that model predictions align with your expectations for known cases. An ML assertion can serve as a systematic sanity check that lets you see if your model is behaving intuitively.

Adding an assertion#

To create some assertions, you can navigate to the Design tab of a visual model analysis. In the Debugging panel, you can add any assertions that would help keep your model on track.

When creating an assertion, you can define the specific conditions (or rules) of your assertion and add a reliability threshold for that rule. Let’s look at an example assertion for more clarity.

Titanic modeling#

This example use case involves designing a model that predicts survival outcomes for passengers on the Titanic.

According to experts, we know that the majority of first class female passengers on the Titanic survived. Using this known information, we can check if the model follows our expectations using an assertion with the following conditions:

A screenshot of a model assertion that checks that first class females are assigned class 1 with a 90 percent threshold.

If the assertion is met when you train the model, then you can have more trust that your model will work well on more complex predictions. Alternatively, if the model fails the check, a warning will appear in the Diagnostics pop-up after training.

You can always review the status of your assertions after training in the Metrics and assertions panel of your model report. According to the context of your model design, you can use these results to improve your model.

Review#

To summarize, you can use ML assertions to build greater confidence in your model by including domain expertise in your validation process.