Watching the road ahead

Road transportation and machine vision

Machine vision has a great future in transportation, particularly as we move towards autonomous vehicles. Dealing with standardization in this area will require joint work between different Standards Developing Organizations (SDOs).

autonomous vehicles - Smithers Apex infographics
Autonomous vehicles - Image sensor markets to 2027 (Infographics: Smithers Apex)

Sensors, cameras and automotive

In some sub-systems, sensors have already become a pervasive technology in the modern-day road vehicle. They are now an integral part of engine management, safety systems and climate control. Many of them use MEMS; International Standards for these are prepared by IEC SC 47F: Micro-electromechanical systems.

We are now moving into new fields of automotive sensors, based on imaging. We could consider these to be extensions of the proximity sensors currently fitted to automotive products to sense the presence of objects during manoeuvring. As we progress towards the concept of smart traffic, these sensors need to be replaced by full image sensors, making these vehicles machine vision platforms.

Machine vision in smart transportation

Machine vision applications are not new to transportation. At the most basic level, proximity sensors built into the road detect the presence and transit of a vehicle for traffic tolling. Based on inductive loops and magnetometers, these systems are fine for detecting large iron-based vehicles but rather poorer for sensitivity to modern methods of transportation such as carbon fibre bicycles. Consequently, an alternative solution, designed to enhance the safety of cyclists at junctions and based on thermal imaging, is under test by the cities of Liverpool and Utrecht.

 A more recognizable machine vision application is the Automatic Number Plate Recognition (ANPR) approach used in parking areas and mounted above roads. Traffic cameras take an optical image of the car number plate, often at near IR wavelengths, and pattern recognize the image to “read” the characters.

Inward and outward facing image sensors

As we move towards Connected and Autonomous Vehicle (CAV) technology (see article in e-tech April 2016), there will be an increasing need for these machine vision systems. In some ways we can consider the image sensors for a smart vehicle to be developing in parallel with those of a smart phone, with cameras facing both the operator and the outside world. In automotive applications, sensors fitted to seats, safety belts and steering wheels are already of interest for monitoring driver fatigue. Image-based sensors may well be positioned to produce the next generation of sensor platforms, looking within vehicles.

Currently it is outward-facing sensors that are attracting the most attention. Advanced Driver Assistance Systems (ADAS) are becoming available, enabling features such as automatic parking, lane keeping assistance, lane departure warnings and emergency braking. These features are also obvious requirements for the next generation of smart autonomous vehicles.

Various imaging-based technologies are required to facilitate ADAS. From the vision side, input will come from a mixture of radar, Light imaging detection and ranging (LIDAR) and infrared/visible wavelength image sensors. There will also be a substantial software systems need, from mapping technology to critical decision-making systems.

The image sensor space is increasingly well served by academic and industrial conferences. For example, 2017 was the inaugural year for the IS&T Autonomous Vehicles and Machines Conference, sponsored by ON Semiconductor. But what of standardization in this field?

Standards for machine vision in transportation

This is an important area that may well need a central home within standardization. There is also a case to be made for this being across both IEC and the International Organization for Standardization (ISO), as the technology bridges the two areas.

The contribution of IEC TC 69: Electric road vehicles and electric industrial trucks, may be more pertinent longer term, along with ISO TC 22: Road vehicles, particularly its SC 32: Electrical and electronic components and general system aspects.

However, none of these would appear to have the background in machine vision that this work requires. A more interesting option for this could be ISO TC 42: Photography. Formed in 1947 to provide standardization of silver halide-based photographic cameras and materials, it has successfully made the transition into electronic imaging. For example, ISO TC42/JWG20: Digital still cameras, is a Joint Working Group between ISO and IEC groups and this may be a useful model to follow. Since the establishment of JWG20 in 1999, this working group, together with others in ISO TC 42, has provided a gathering place for expertise in the area of electronic imaging standardization. Other ISO TC 42 JWGs with the IEC include ISO/TC 042/JWG 22: Colour management, which incorporates the work of IEC TC 100: Audio, video and multimedia systems and equipment.

The digital imaging technology currently used for machine vision and that will be used for autonomous vehicles relies heavily on image sensors for detecting and conveying the information that constitutes images. International Standards for a multitude of sensors used for imagery, motion and distance detection, such as those used in ADAS and Lidar, are developed by IEC SC 47E: Discrete semiconductor devices.

Increased coordination between SDOs is a must

Many IEC International Standards are applied or referenced by other SDOs in the automotive industry, such as ISO or the Society of Automotive Engineers (SAE). The ISO 26262, Road vehicles – Functional safety, series of Standards, for instance, is the adaptation of IEC 61508, Functional safety of electrical/electronic/programmable electronic safety-related system, to the specific requirements of passenger cars and light utility vehicles.

Likewise the SAE J1772-2009 Standard for electrical connectors for electric vehicles, has been added as Type 1 to IEC 62196-2:2016, Plugs, socket-outlets, vehicle connectors and vehicle inlets – conductive charging of electric vehicles Part 2: Dimensional compatibility and interchangeability requirements for a.c. pin and contact-tube accessories.

Coordination between various SDOs in the automotive sector is likely to strengthen in the future as the overall share and value of electrical and electronic components in vehicles increases in line with the rollout of increased numbers of autonomous vehicles.