Solution | ISO 42001 Readiness

Overview

This Solution provides governance and procurement teams with a series of templates designed to support their alignment to ISO 42001’s approach to managing AI systems.

Introduction to ISO 42001

ISO 42001 is an international standard that specifies requirements for establishing, implementing, maintaining, and continually improving AI Management Systems. It is designed for entities providing or utilizing AI-based products or services, ensuring responsible development and use of AI systems.

Legal Disclaimer

Dataiku’s ISO 42001 Readiness Solution is designed to assist users in evaluating potential alignment with the framework based on publicly available information. However, this tool does not constitute legal advice, approval, or a definitive determination of compliance. The implementation of ISO 42001 is subject to interpretation, which may affect how alignment with its content is assessed.

Users should be aware that:

• This tool provides general informational support and should not be relied upon as a substitute for professional legal counsel.

• Compliance determinations depend on specific use cases, sectoral regulations, and evolving interpretations.

• Dataiku does not guarantee the accuracy, completeness, or applicability of the tool’s outputs for any specific AI system or regulatory requirement.

• Dataiku does not guarantee that using this tool is equivalent to any formal certification process related to ISO 42001.

By using this tool, users acknowledge that Dataiku is not liable for any decisions, actions, or legal consequences arising from reliance on its assessments. We strongly recommend consulting a qualified legal expert or regulatory professional before making any compliance-related decisions.

Installation

Please reach out to your Account Team who will support you with the Solution installation.

Technical Requirements

For this Solution to work on Dataiku’s Govern Node, the user must have access to Advanced Govern. The Solution can be implemented on either cloud or self-managed instances of Dataiku.

We recommend that you are running Dataiku v13.0.0 or newer as these versions will have conditional workflow views and table artifact references enabled allowing for the full functionality of the solution. Older versions of Dataiku and Dataiku Govern will require adapting the conditionality, table view, and Python hooks running in the solution.

Walkthrough

The Dataiku Govern ISO 42001 Solution uses a comprehensive end-to-end workflow, starting with a Pre-Assessment preceding AI system development and concludes with an on-going evaluation of an AI system’s value and success (Post-Assessment).

From the Pre-Assessment custom page, click on the “Create” button to start a new ISO 42001 Impact Assessment.

Pre-Assessment, ISO 42001 Impact Assessment

The ISO 42001 solution assumes that ideas for AI systems need to be evaluated before a project can begin. The Pre-Assessment workflow is composed of the following steps:

Pre-Assessment workflow

Step	Purpose
Overview	A broad overview of the pre-assessment, referencing any associated Business Initiative and Projects.
Project Scope	A high-level section to define a future project’s intended purpose, any possible consequences and misuse scenarios, and the scope and composition of the proposed AI system.
Project Details	A detailed section documenting key project criteria such as complexity, feasibility, cost, legal requirements, SME resourcing, human oversight design, and the creation of a resourcing plan.
Risk Identification	A detailed section fulfilling risk management requirements where risks and mitigations are identified and documented. While new risks and mitigations can be created at the time of pre-assessment, it is recommended administrators build and maintain an organization’s risk registry and mitigations registry beforehand. Following identification, for each risk, define how and by whom these risks will be controlled and mitigations implemented.
Project Creation	A sign-off confirming that possible risks associated with the prospective project are sufficiently assessed, mitigated, accepted, and/or transferred. Upon sign-off, the project is automatically created in design via a Python-based hook. This hook can be disabled if desired, with little implication on the ISO workflow.

The Pre-Assessment is now completed.

Project

Like other govern project templates, the ISO 42001 solution leverages metadata from a project created in the Design node. The Project workflow is composed of the following steps:

Project workflow

Step	Purpose
Overview	At a glance, a space to reference other artifacts within the AI System, including the Pre-Assessment from which this project originated, any business initiatives, the currently active bundle, and any managed datasets.
Project Framing	A detailed section to define a project’s use case, design choices, build roadmap, and any known system limitations.
Nomination Sign-off	A sign-off confirming that resources nominated during the Pre-Assessment are both available and competent in their assigned duties throughout the project pipeline. Nominations may be changed at this stage and changes will be automatically reflected in the Pre-Assessment.
Data	A detailed section for documenting datasets required and used, data-related policies, labelling, and preparation steps necessary for building the AI System.
Model	A section for both high-level details and overview details pertaining to model and algorithmic use within the AI System. Any associated model and model version workflows require completion. Upon their completion, the project artifact will reference any associated model. Model and model version workflows can be completed concurrently to the project workflow.
Data and Model Sign-off	A sign-off confirming that datasets and models are appropriate for the AI system and avoid any prohibited use cases documented in the Pre-Assessment.
System	A detailed section documenting hardware requirements, security and adversarial threats, and AI system interactivity for cases where several models and/or projects may occur downstream or upstream of one another. Additionally, a systems’ environmental impact may be assessed if mechanisms to measure gCo2 or KwH are available.
Human interactivity	A detailed section describing human oversight throughout the AI system architecture and ability for reviewers of generated output to evaluate the system’s continued quality following deployment.
End users	A detailed section documenting how end users interact with the AI system, the system’s benefits and harms, and mechanisms for raising issues and/or opting-out.
Post-Assessment	A two part section for bundle management and post-assessment improvement plan (“ISO 42001 Improvement Plan”, see separate section below).

Before completing the ISO 42001 Improvement Plan proceed to the bundle artifact and complete the workflow.

Upon completion of the bundle workflow, complete the Post-Assessment.

Following the completion of both bundle and post-assessment, the Project template is now completed. There are no additional steps to take, but projects should be kept up-to-date should new model versions, bundles, or incidents arise.

Model

ISO 42001 does not require a customized Model blueprint and instead uses the Dataiku Standard blueprint for a model. A model sits between a project and any associated model versions.

Model Version

Model Versions capture development information pertinent to training and testing machine learning and associated algorithms. It is recommended to govern active model versions, denoted by a green encircled check mark. The Model Version workflow is composed of the following steps:

Model version workflow

Step	Purpose
Overview	A broad overview of the model version, referencing any associated models.
Development	A detailed section for documenting the development of the AI System, including suitability, amalgamated datasets used for training/testing/validation including any associated data metrics for each dataset, and system requirements required for pre-deployment testing.
Testing	A section for documenting the methodology and tooling required to complete suitable tests ensuring the model version will perform according to design specifications and business expectations.
Review	A sign-off confirming that the model version is designed according to requirements and meeting functional expectations prior to productionalizing the AI System. For active model versions, this workflow is now complete.
Model Decommissioning	A section to assist with version control of model versions, including reasons and steps to decommission and/or replace a model version.

Bundle

The Bundle workflow captures testing and deployment information pertinent to the finalization of the AI System. The Bundle workflow is composed of the following steps:

Bundle workflow

Step	Purpose
Overview	A broad overview of the bundle referencing the associated project.
Monitoring Plan	A three part section for defining a Model Monitoring Plan (“ISO 42001 Model Monitoring Plan”, see separate section below), links to monitoring dashboards and/or the Deployer Node’s Unified Monitoring, and the date this Monitoring Plan was last reviewed.
QA Environment Testing	A section for defining and reporting QA checks prior to pushing a bundle into UAT.
QA Sign-off	A sign-off confirming that QA checks were successfully completed and that the bundle is ready for User Acceptance Testing.
User Testing	A section for defining and reporting UAT results prior to pushing a bundle into production.
Business Approval	A detailed section for documenting full business review and approval, including a recommended course of action to determine if a bundle is sufficiently achieving its business goals and intended purpose.
System Overview	A section for documenting deployment and system requirements as well as any additional architectural diagrams.
Review	A sign-off confirming that the bundle is ready for production. This sign-off gates deployments and as such should be treated as the final guardrail with respect to ensuring your organization is satisfied with the AI system’s compliance.

Monitoring Plan

A Monitoring Plan is a defined plan for ensuring an AI system remains performant over time while in production. During the development and pre-deployment process, it is recommended that metrics are determined and measured to observe possible drifts across four different metric categories:

Type of drift	Model performance…
Concept drift	Degrades from predictive metric expectations (e.g. accuracy, recall, etc).
Data drift	Degrades due to changes in data (e.g. data distributions).
Bias drift	Begins to unfairly benefit/harm subpopulations and protected classes (measured statistically, across a wide range of metrics, but with a specific emphasis on bias mitigation).
Continuous reinforcement	Changes due to further machine learning model re-weighting.

The Monitoring Plan supports defining metrics, their acceptable thresholds and associated benchmarks, and actionable plans, including responsible parties, to address instances of detected drift. There is no associated workflow, but instead one section per metric category and allows as many metrics to be added as desired. The Monitoring Plan does not enforce the creation and/or monitoring of these metrics, but instead helps organizations identify, document, and coordinate metrics before an AI system is fully productionalized and their monitoring and review cycle.

Post-Assessment, ISO 42001 Improvement Plan

The ISO 42001 solution concludes with an on-going evaluation of a project’s value through the Post-Assessment. There is no associated workflow, but instead one section for time-based improvements and one section for incident-based improvements.

Time-based improvements document updates, evaluation results, changes in ownership, and AI system regular impact evaluation procedures. Incident-based improvements document instances of feedback and incidents. Both sections should be continuously added to and updated over the entire course of the project’s life, until full decommissioning.