Requirement
"The organization shall determine what AI system technical documentation is needed for each relevant category of interested parties, such as users, partners, supervisory authorities, and provide the technical documentation to them in the appropriate form."[1]
Explanation
Different audiences — users, partners, regulators — need different technical documentation about an AI system: its purpose, limitations, design choices, failure plans and operating procedures. This control requires the organization to decide what each group needs and provide it, kept accurate and management-approved. Auditors sample the documentation for currency and approval.
Implementation Guidance
- The AI system technical documentation can include, but is not limited to the following elements:
- a general description of the AI system including its intended purpose;
- usage instructions;
- technical assumptions about its deployment and operation (run-time environment, related software and hardware capabilities, assumptions made on data, etc.);
- technical limitations (e.g. acceptable error rates, accuracy, reliability, robustness);
- monitoring capabilities and functions that allow users or operators to influence the system operation.
- Documentation elements related to all AI system life cycle stages (as defined in ISO/IEC 22989) can include, but are not limited to:
- design and system architecture specification;
- design choices made and quality measures taken during the system development process;
- information about the data used during system development;
- assumptions made and quality measures taken on data quality (e.g. assumed statistical distributions);
- management activities (e.g. risk management) taken during development or operation of the AI system;
- verification and validation records;
- changes made to the AI system when it is in operation;
- impact assessment documentation as described in B.5.
- The organization should document technical information related to the responsible operation of the AI system. This can include, but is not limited to:
- documenting a plan for managing failures. This can include for example, the need to describe a rollback plan for the AI system, turning off features of the AI system, an update process or a plan for notifying customers, users, etc. of changes to the AI system, updated information on system failures and how these can be mitigated;
- documenting processes for monitoring the health of the AI system (i.e. the AI system operates as intended and within its normal operating margins, also referred to as observability) and processes for addressing AI system failures;
- documenting standard operating procedures for the AI system, including which events should be monitored and how event logs are prioritized and reviewed. It can also include how to investigate failures and the prevention of failures;
- documenting the roles of personnel responsible for operation of the AI system as well as those responsible for accountability of the system use, especially in relation to handling the effects of AI system failures or managing updates to the AI system;
- documenting system updates like changes in the system operations, new or modified intended uses, or other changes in system functionality.
- The organization should have procedures in place to address operational changes including communication to users and internal evaluations on the type of change.
- Documentation should be up to date and accurate. Documentation should be approved by the relevant management within the organization.
- When provided as part of the user documentation, the controls provided in Table A.1 should be taken into account.
References
- ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system, A.6.2.7
- ISO/IEC 42001:2023 Annex B, B.6.2.7 (implementation guidance)
