Deep learning is robo-cars’ Achilles heel, experts say

Article By : Junko Yoshida

U.S. regulators fail to tangle head-on with fundamental difficulties in testing machine learning.

Self-driving cars are nearing reality, thanks to advancements in machine learning. But when the issue comes down to “safety testing,” machine learning is self-driving’s Achilles heel, according to safety experts.

Philip Koopman, professor of Carnegie Mellon University, believes the biggest hole in a federal automated policy published late September is in the regulators’ failure to tangle head-on with fundamental difficulties in testing machine learning—a problem already known to the scientific/engineering community.

“Mapping Machine Learning‐based systems to traditional safety standards is challenging,” Koopman said, “because the training data set does not conform to traditional expectations of software requirements and design.”

In Koopman’s opinion, the Fed’s policy “should say that Machine Learning is an unusual, emerging technology.” This acknowledgement would prompt regulators to ask more pointed questions on Machine Learning in their safety assessment.

“I’m not saying how to test the Machine Learning (ML)’s training data set,” said Koopman. Rather, “I’m proposing that the DoT should demand from a carmaker or autonomous car platform vendor a written document that justifies why their ML-based autonomous vehicle is safe,” he said.

__Figure 1:__ *Philip Koopman teaches embedded systems to undergraduates and safety-critical embedded systems to grad students at CMU.*

Koopman has been involved in autonomous vehicle safety for 20 years. His experience ranges from participating in the Automated Highway System (AHS) program early in his career to working at the National Robotics Engineering Centre with funded projects on autonomous vehicle safety and robotic system dependability.

Challenges in Machine Learning

Asked by EE Times how to test machine learning systems, Luca De Ambroggi, principal analyst, Automotive Semiconductor, Technology at IHS Markit, told us, “This is the biggest challenge. There is no answer there yet.”

R&D veterans of machine learning field are familiar with its brittleness. The DoT should probe “the completeness and correctness of the ML data set, training process, and validation process,” explained Koopman.

The DoT rolled out what it calls a “15 Point Safety Assessment” for manufacturers, developers and other organisations to follow in the design, development, testing and deployment of automated vehicles. Under the proposed guideline, regulators are asking automakers to provide the National Highway Traffic Safety Administration (NHTSA) with a safety assessment. While praising DoT for “a good job at proposing a baseline for discussing how an appropriate level of safety can be achieved,” Koopman noted several gaps in the guidelines especially when it comes to Machine Learning.

Followings are a few topics Koopman believes that regulators should cover in assessing safety in ML-based autonomous cars.

 
Next: Knowing the limitations of machine learning »