Requirement
"The organization shall define and document requirements for data quality and ensure that data used to develop and operate the AI system meet those requirements."[1]
Explanation
Poor data quality produces unreliable AI outputs and hidden bias. The organization must define measurable data quality requirements and show that training, validation, test and production data actually meet them. Auditors look for written requirements, quality measurements against them, and evidence that bias has been assessed and addressed.
Implementation Guidance
- The quality of data used to develop and operate AI systems potentially has significant impacts on the validity of the system's outputs. ISO/IEC 25024 defines data quality as the degree to which the characteristics of data satisfy stated and implied needs when used under specified conditions. For AI systems that use supervised or semi-supervised machine learning, it is important that the quality of training, validation, test and production data are defined, measured and improved to the extent possible, and the organization should ensure that the data are suitable for its intended purpose. The organization should consider the impact of bias on system performance and system fairness and make such adjustments as necessary to the model and data used to improve performance and fairness so they are acceptable for the use case.
- Other information: Additional information regarding data quality is available in the ISO/IEC 5259 series on data quality for analytics and ML. Additional information regarding different forms of bias in data used in AI systems is available in ISO/IEC TR 24027.
References
- ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system, A.7.4
- ISO/IEC 42001:2023 Annex B, B.7.4 (implementation guidance)
