Boeing

Computer vision in China

A collaborative effort in machine perception research aids airplane inspection.

By Ally (Ying) Liu, Electrophysics Engineer | Elisabeth Martin, Aerospace Engineer | Fong Shi, Technical Fellow

China is rapidly becoming the largest aviation market outside the United States. According to Boeing forecast, China’s aviation industry needs at least 5,000 airplane technicians each year for the next 20 years. However, given that it takes five to seven years to train an airplane maintenance technician, a shortage of qualified airplane technicians becomes inevitable; on current trends, only 3,000 might be licensed.

An automated system to assist maintenance inspection could help mitigate this challenge in technician training and represents a significant business opportunity for aviation services.

The rapid development of combining immersive technologies such as augmented reality (AR) with artificial intelligence is now enabling computer vision-based applications, also known as machine perception technology, that could play an important role in airplane ground inspection.

Aiming to improve airline operation efficiency and support business development in aviation services, Boeing researchers have been collaborating with research partners from Chinese universities to develop an automated system for airplane inspection. Computer-vision research and development efforts across China have achieved major progress in recent years, driven by a need for practical applications such as this in light of the country’s aviation system growth.

At the front-end image capturing stage, ground crews may scan an airplane with an AR headset along with other necessary hardware, and capture images of all possible regions of an airplane. Captured images are transmitted to a back-end processing platform where computer-vision techniques are deployed to identify if certain abnormalities exist.

To achieve detections with higher accuracy, state-of-the-art computer-vision architectures and algorithms of deep neural networks are being designed and developed to recognize and analyze the captured airplane images to identify anomalies and initiate early warnings. Furthermore, for the purpose of training computers through intelligent machine learning, over 20,000 airplane images containing damages of various types have been collected.

Applying automated detection for a comprehensive airplane inspection—including airplane crown, tails and components that are hard to reach—will also facilitate shorter aircraft turnaround times on the ground.

Boeing’s research center in China—established in 2009—has research collaborations with 35 major universities, 21 national institutes and 24 industrial partners in China as of 2018, making it a nexus of research, development and innovation for the Chinese research community.

Computer-vision-aided airplane inspection is an example of one of the research applications advancing with this effort, creating practical technical solutions for the burgeoning market.

Infographic

Infographic

SYSTEMS ARCHITECTURE

  • Capturing images of airplanes with an AR headset
  • Employing deep learning techniques to identify anomalies and initiate early warn
  • Transmitting the captured images wirelessly to a back-end workstation

Graphic: Boeing