Achieving trustworthy AI with standards

Artificial intelligence (AI) systems promise to bring many benefits, however considerations in the area of trust, ethics and societal  concerns stemming from the application domain, business requirements, regulatory policy, consumers and end users as well as society-at-large must first be addressed. For instance, AI systems need to ensure reliability, fairness and transparency. In other words, they need to be trustworthy.

 

Machine learning training on humans and object recognition
AI uses machine learning to develop predictive analytics and databases

The IEC and ISO joint technical committee which develops international standards for AI (ISO/IEC JTC 1/SC 42) has published a new Technical Report (ISO/IEC TR 24028: 2020). The TR provides an overview of topics relevant to building trustworthiness of AI systems. One of its aims is to assist the standards community in identifying specific standardization gaps in AI.

“The importance of a trustworthy AI system to ensure wide scale adoption cannot be understated,” said Wael William Diab, chair of SC 42. “Through its ecosystem of international AI standards, that include a suite of projects on AI trustworthiness, SC 42 is enabling that which in turn will make the promise of AI enabled digital transformation a reality.”

An ecosystem of trust

SC 42 takes a comprehensive look at the ecosystem in which AI systems are developed and deployed. By looking at the context of use of the technology, such as application domain, business, societal and regulatory requirements, SC 42 develops horizontal standards that standards for applications, open source and proprietary communities can provide a foundation upon which to create AI solutions to market.

“Recognizing that artificial intelligence has the potential to be a ubiquitous technology for good and the data centric world we live in, SC 42 took the novel approach to focus on the full AI ecosystem addressing areas such as data quality, privacy, security, trustworthiness and ethics upfront,” said Diab. “The programme of work covers the entire lifecycle of the technology, including design through application deployment. The committee collaborates closely with other IEC, ISO and JTC 1 committees.”

Within the context of trustworthiness (addressing ethics and societal concerns and building for robustness and reliability), SC 42 has a suite of standards projects at various stages of development. These deliverables extend and build upon relevant IEC and ISO security, privacy, quality and risk management standards. 

Layers of trust

The Technical Report considers several existing approaches being used for building trustworthiness in technical systems and discusses their potential applicability to AI systems.

AI systems can be seen as operating across multiple layers within an ecosystem consisting of other hardware and software subsystems. While the definition varies across standards and publications, trust must be established and maintained for each layer.

The TR refers to the ISO/IEC SQuaRE series of Standards, which deals with software quality through models and measurement (ISO/IEC 2501x on models and ISO/IEC 2502x on measurement), that result in a list of characteristics for software quality and characteristics for data quality.

The SQuaRE quality standards have been developed for traditional software systems. For example, the data quality model in ISO/IEC 25012 does not sufficiently address the characteristics of the data-driven nature of AI systems. In addition, a data quality model for AI systems needs to take into consideration other characteristics not currently described in ISO/IEC 25012, such as bias in the data used to develop the AI system. As a result, SC 42 is looking to extend the SQuaRE definitions to more adequately cover AI systems and the data they depend on.

Risk management helps to ensure that “by design” a specific AI product or AI service is trustworthy throughout its lifecycle and lends itself to new technologies carrying risk inherently where the unknown is greater than the known, including dealing with human errors and malicious attacks.
ISO 31000 defines the process of risk management. SC 42 is working on the ISO/IEC 23894 project to define the application of ISO 31000 to AI.

Stakeholders, assets and values

Different stakeholders, assets and values come into play throughout the lifecyle of an AI system. The TR describes these different areas.

There are diverse stakeholders with distinct roles in the AI value chain. These include organizations or individuals providing data that is used to train an AI system, AI systems developers, producers and users and test and evaluation agencies offering independent testing and possibly a certification. There are also associations representing the viewpoints of individuals, and governance organizations that study the use of AI, such as national governments and international organizations.

Assets and values that play a role or can be affected by the development and use of a product or service include:

  • data used to train an AI system
  • a product or service that uses one or more AI systems
  • data used to test the AI-related behaviour of a product or service
  • data fed to a product or service operation, based on which AI-based decisions are made
  • computing resources and software used to train, test and operate AI systems
  • human resources with the skills to train, test and operate AI systems; develop software used in or for those tasks; generate, annotate or select data needed for AI training.

The TR notes that stakeholders may have different views on what appropriate characteristics of a trustworthy AI system should be. These views may be based on personal values, different moral worldviews or systems of values, such as fundamental rights from different regions of the world.

Responsibility, accountability and safety

Complex AI systems apply IT in a multi-stakeholder environment. It is vital to define responsibilities and the accountability between stakeholders, so that all involved have a clear understanding of what they are undertaking and how they will be held responsible if the AI system fails. The TR discusses ISO/IEC 38500, which guides high-level decision-makers in an organization in understanding and fulfilling their legal, regulatory and ethical obligations in the use of IT.

Safety is a key part of trust. The higher the risk, the greater the considerations must be for safety aspects throughout the lifecycle of an AI system, from its design to its disposal. The TR refers to ISO/IEC Guide 51, which provides requirements and recommendations for the inclusion of safety aspects in standards.

Vulnerabilities of AI systems

The TR surveys vulnerabilities of AI systems that can reduce their trustworthiness, such as attacks on machine learning (ML) systems for self-driving vehicles.

Hardware-focused threats in ML applications affect data confidentiality, integrity and computation. Other attacks may lead to denial of service (loss of availability) or cause leakage of information or invalid computation.

Other issues arise around data such as acquisition, preparation, modelling and bias.

AI systems need to be predictable so that people using them will know what they will or will not do at a given point.

Improving trustworthiness

The TR identifies possible measures that improve trustworthiness of an AI system by mitigating vulnerabilities across its lifecycle. These relate to:

  • transparency for the features, components and procedures of an AI system
  • controllability by providing reliable mechanisms for an operator to take over control from the AI system
  • aspects of data handling, such as strategies for reducing unfair bias and maintaining privacy by de-identifying personal data
  • robustness or the ability of a system to maintain its level of performance under any circumstances including external interference or harsh environmental conditions
  • testing, evaluation and use

The TR notes that AI systems are complex and their impact on stakeholders should be carefully considered case by case to decide whether their use is appropriate. A risk-based approach helps organizations identify relevant AI systems’ vulnerabilities and their possible impacts on organizations, their partners, intended users and society and will help mitigate risks. It is important that all stakeholders understand the nature of the potential risks and the mitigation measures implemented. SC 42 is working on ISO/IEC 23894, Information technology – Artificial intelligence – Risk Management that will define how to apply the risk-based approach to AI.

Looking ahead

“We need to make sure that AI systems are trustworthy, technically robust, controllable and verifiable over their entire lifecycle, wherever they are being deployed in the world. Many aspects including societal concerns, such as data quality, privacy, potentially unfair bias and safety must be addressed. This recently published Technical Report is the first of many works that will help achieve this”, said David Filip, Convenor of the SC 42 working group on trustworthiness.

SC 42 has a portfolio of active study topics and seven ongoing projects in this area such as bias, robustness, risk management, functional safety, ethics and societal concerns. It recently approved a new project ISO/IEC 5059, Software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Quality Model for AI-based systems.

“I’m thrilled about the timely publication of this report by SC 42,” said Orit Levin, project lead of ISO/IEC 24028. “Creating it was only possible with the numerous contributions and enthusiasm of the diverse SC 42 community representing, at times, contrasting experiences, needs, and points of view as related to AI. The methodology and findings of this work have already resulted in multiple SC 42 projects and, I believe, will contribute to more projects in the future.”

The international standards SC 42 is developing around the AI ecosystem, including trustworthiness, will enable innovation and broad adoption of AI for good.

Learn more about SC 42.