Requirement
"The organization shall ensure that its responsible approach to the development and use of AI systems considers their customer expectations and needs."[1]
Explanation
When the organization supplies AI products or services, it must understand what its customers expect and need — and where responsibility sits on each side of the relationship. Auditors look for evidence that customer expectations shape development and use, and that customers receive the information needed to use the system safely, such as its limits of use.
Implementation Guidance
- The organization should understand customer expectations and needs when it is supplying a product or service related to an AI system (i.e. when it is itself a supplier). These can come in the form of requirements for the product or service itself during a design or engineering phase, or in the form of contractual requirements or general usage agreements. One organization can have many different types of customer relationships, and these can all have different needs and expectations.
- The organization should particularly understand the complex nature of supplier and customer relationships and understand where responsibility lies with the provider of the AI system and where it lies with the customer, while still meeting needs and expectations.
- For example, the organization can identify risks related to the use of its AI products and services by the customer and can decide to treat the identified risks by giving appropriate information to its customer, so that the customer can then treat the corresponding risks.
- As an example of appropriate information, when an AI system is valid for a certain domain of use, the limits of the domain should be communicated to the customer. See B.6.2.7 and B.8.2.
References
- ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system, A.10.4
- ISO/IEC 42001:2023 Annex B, B.10.4 (implementation guidance)
