International standards committee for AI ecosystem expands into new areas

New AI standards projects have been approved for management systems data quality and knowledge engineering 

Standards for data quality

Artificial intelligence (AI) technologies are part of many products, systems and services we use on a daily basis. They help deliver healthcare, manage investments, track online fraud, streamline manufacturing processes and much more. AI brings many benefits but it also raises concerns, for instance regarding  data privacy, unintended bias and ethical and societal concerns of people who use or come into contact with such technologies, or whose personal data may be used by these systems.

IEC and ISO develop publications and international standards for information and communication technologies through their joint technical committee (ISO/IEC JTC 1), for all aspects of the artificial intelligence (AI) ecosystem (SC 42), including ethics and trustworthiness, to ensure a broad and rapid adoption of these technologies.

“We are starting to see the huge potential of emerging IT technologies like AI, Big Data and analytics bring by revolutionizing our lives from the playground to the workplace. SC 42 is enabling this digital transformation by addressing the ecosystem and collaborating with partners to allow for broad adoption. The excitement and enthusiasm continue to be evident where SC 42 has almost doubled it’s projects while completing and publishing a number of projects. Seven of the 11 new projects were approved and initiated in the last couple of months. These new projects complement the ecosystem coverage by:

  • expanding into new areas like data quality and knowledge engineering
  • enabling application / application standards development by building on well established lifecycle standards and providing guidelines to application standards
  • novel and comprehensive approach to address, from the start, increasingly important issues related to the context of use of the technology such as ethical and societal considerations. Despite the unfortunate events of COVID, the enthusiasm and energy of the committee remains high as reflected by the growth in the work programme, partnerships and membership. It’s a great time to join the effort,” said Wael Diab, who is Chair of SC 42.

Data quality assigned to SC 42/Working Group 2

The quality of an organization’s data is crucial to its successful implementation of big data and AI systems, from data collection to analysis and use in model training for machine learning. Low quality data will give unreliable results. Services using models trained with low quality data could directly threaten safety.

The new series addresses data quality as it pertains to AI, big data and analytics and the crucial need to look at data quality in these emerging systems.

SC 42 recognized the importance the data ecosystem plays, expanded the scope of WG 2 and is starting this new series, so that these technologies, applications and data are no longer considered in isolation.

The ISO/IEC 5259 series of standards projects relates to data quality for machine learning (ML) and analytics.

ISO/IEC 5259, Data quality for analytics and ML - Part 1: Overview, terminology, and examples, will cover the overall concepts and the scope of “Data quality for analytics and ML” series, as well as examples of data quality in analytics and ML to help users understand this series of standards.

Part 3, Data Quality Management Requirements and Guidelines, will include requirements and provide guidance for establishing, implementing, maintaining and continually improving the quality for data used in the areas of analytics and machine learning. It will define the requirements and guidance for a quality management process along with a reference process and methods that can be tailored to meet the requirements in this document. This document is generic and intended for use by all organizations, regardless of type, size or nature.

Part 4, Data quality process framework, provides general common organizational approaches, regardless of type, size or nature of the applying organization, to ensure data quality for training and evaluation in analytics and ML.

It will be applicable to training and evaluation data that comes from different sources, including data acquisition and composition, pre-processing, labelling, evaluation, and use. This standard will propose a process framework of data quality for analytics and machine learning and provide technical aspects over the quality process of training and evaluation data, including general principles, process frameworks and organizational approaches of data quality for machine learning. It will allow the implementation of a standardized common procedure of data processing for data quality.

Part 4 will include guidelines for:

  • supervised machine learning with regard to the labelling of data used for training machine learning systems, including common organizational approaches for training data labelling
  • unsupervised machine learning
  • semi-supervised machine learning
  • reinforcement machine learning

AI life cycle processes assigned to SC 42/WG 4

Some examples of AI systems include machine translation, speech recognition, image classification and information retrieval. When building an AI system, it is necessary to consider the AI-specific life cycle processes and the life cycle processes for a traditional software system.

Many applicable lifecycle standards, such as ISO/IEC/IEEE 15288, System life cycle processes, and ISO/IEC/IEEE 12207, Software lifecycle processes, which describe the life cycle of software and system, are widely deployable and by and large applicable. While it is recognized that AI systems share and benefit from such lifecycle standards, they also have unique characteristics, which need to be addressed from a lifecycle point of view. SC 42 is building on existing standards, while addressing uniqueness.

ISO/IEC 5338, Information technology - Artificial intelligence - AI system life cycle processes, will build on a number of successful life cycle standards for AI specific life cycle components and define a set of processes and associated terminology for describing the lifecycle of AI systems. Additionally, it will provide processes that support the definition, control and improvement of the AI system life cycle processes used within an organization or a project.

Guidelines for AI applications assigned to SC 42/WG 4

ISO/IEC 5339, Information Technology - Artificial Intelligence - Guidelines for AI Applications, will provide the basis for other technical committees, standards development organizations and proprietary implementations to understand how AI systems are built. It will enable them to leverage SC 42 horizontal standards while providing a point of view of the application perspective and concerns.

The guidelines will help users to identify the context, opportunities, and processes for developing and applying AI Applications. They will also provide a macro level view of the AI application stakeholders and their roles, relationship to the life cycle of the system, and common AI application characteristics.

This standard is aimed at AI application users, rather than researchers, for instance, standards development organizations, open source communities for AI and AI systems developers. It presents the AI ecosystem from the application point of view.

IEC and ISO have hundreds of technical committees which develop applications and are looking at how AI will work in their roadmaps.

SC 42 works with many internal and external liaison partners and is tasked with providing guidance to internal committees in IEC, ISO and JTC 1.

AI management systems assigned to WG 1 for foundational standards

ISO/IEC 42001, Information Technology - Artificial intelligence - Management System, will contain AI specific process requirements which would allow for assessment or conformance of auditability of the processes. It will also and provide guidance for establishing, implementing, maintaining and continually improving an artificial intelligence management system within the context of an organization.

The standard will help organizations develop or use AI responsibly in pursuing their objectives, and to meet applicable regulatory requirements, obligations related to interested parties and expectations from them.

Organizations in the domains of health, defence, transport, finance, employment and energy will be able to show that they have implemented and continually iterate on the improvement of processes unique to the development or use of an AI system, for example processes identifying and treating bias of learning data, or more generally fairness, inclusiveness, safety, security, privacy, accountability, explainability and transparency of the AI system.

This will help to increase consumers trust in the products and services that use AI provided by large industry or small and medium sized enterprises. Equally, governments and policy makers may be able to use the proposed standard for possible third party certification to provide a basis for regulation of a large number of industry sectors.

Knowledge engineering assigned to WG 5 for computational approaches and computational characteristics of AI systems

Knowledge engineering is a field of artificial intelligence (AI) that aims to emulate human expert judgment and behaviour in a particular area. It refers to the process of automatically or semi-automatically acquiring from huge-scale multi-source heterogeneous data, integrating knowledge into knowledge-based systems and providing intelligent knowledge services. This technology is used in expert systems deployed in healthcare, customer service, financial services, manufacturing and law.

ISO/IEC 5392, Information technology - Artificial intelligence - Reference Architecture of Knowledge Engineering, will describe knowledge engineering roles, activities, constructional layers, components and their relationships from user and functional views.