Designers Could Soon Adhere to UL’s Safety Standards

Article By : Junko Yoshida

By the end of this year, designers of self-driving vehicles, mining robots and a plethora of other industrial systems featuring autonomy could be signing on to the emerging UL 4600 standard.

Underwriters Laboratories (UL) is doing a cannonball into the deep end of the “autonomous standards” pool.

By the end of this year, designers of self-driving vehicles, mining robots, unmanned aerial vehicles (UAVs), and a plethora of other industrial systems featuring autonomy could be adhering to an emerging UL Standard, dubbed UL 4600, that would cover autonomous product safety.

At least this is the goal set by the UL and the Edge Case Research (ECR) – who are collaborating on the still embryonic UL4600.

The emerging standard is designed to “address the ability of autonomous products to perform the intended function without human intervention based on their current state and sensing of the operating environment,” according to ECR.

It’s important to note that a bigger international organization, ISO, is driving a similar initiative, known as “Safety of the Intended Functionality (SOTIF).” The group is reportedly pursuing SOTIF development to reduce risks in ADAS-assisted and autonomous vehicles (AVs) that might cause trouble on the road even without a hardware or software malfunction.

So, how does UL 4600 differ from SOTIF (also known as ISO 21448)?

Phil Koopman, professor in Electrical and Computer Engineering at Carnegie Mellon Univ. and CTO of ECR, made it very clear, “UL 4600 is not being developed to supplant SOTIF.”

Riccardo Mariani, Intel Fellow and functional safety technologist, who is active in SOTIF development, agreed. Although he hasn’t seen details of the UL 4600 spec, he is hopeful that SOTIF and UL 4600 will find ways to collaborate.

Mariani told EE Times, “Yes, I heard in some conferences and blogs that the UL 4600 effort is under way. As also I have inquired with Phil [Koopman] about details and I plan to give my contribution to it, either directly or indirectly.” He added, “I respect every standardization effort because I extremely value interoperability and cooperation between industries and experts. Standards are helping to consolidate and share the state of the art.”

Nevertheless, there are differences in the two groups’ approaches to defining safety in highly automated systems.

Philosophical vs. Prescriptive

On one hand, SOTIF is taking a more philosophical approach to define “unknown, unsafe” scenarios for ADAS and AVs. The UL 4600, on the other hand, prefers a goal-based approach that prescribes topics to be addressed in creating a safety case, according to Koopman.

Michael Wagner, co-founder and CEO at Edge Case Research, added that UL standards have always been “very prescriptive… UL safety certification was fomented to assess the compliance of products to recognized requirements. UL can tell you that if you do this, the following (bad) things could happen. Therefore, don’t do that.”

The idea of collaborating with UL hit Wagner while he was talking to ECR’s clients. “Many of our clients told us they wanted to know a systemic way of building an autonomous system,” said Wagner. A range of clients also wanted a standard not just for autonomous vehicles but something that can be adapted by different autonomous systems that operate in different situations.

Machine learning is another area where the two groups take varying approaches. Many members working within the SOTIF feel more comfortable limiting the scope of SOTIF development to lower-level autonomous vehicles. Koopman said, “We will go after full autonomy head on.” UL 4600’s authors intend to specifically cover validation of any machine learning based functionality and other autonomy functions used in life-critical applications.

At this point, however, neither UL 4600 nor SOTIF is recognized as an autonomy standard by the engineering community at large. Asked about UL 4600, Phil Magney, founder of VSI Labs, told EE Times, last week, “Any buzz? Not yet. I just came back from SAE WCX (World Congress Experience), and mentioned it a couple of times to my colleagues and it was new to most of them.”

Asked to compare UL 4600 with SOTIF, Magney said, “UL 4600 is complementary in my opinion. In fact, UL 4600 was designed in the interest of SOTIF.”

He explained, “SOTIF calls out the need for methods and processes while UL4600 is a methodology and procedure for getting it done.” He suggested that with SOTIF, the industry is trying to understand the limits of the technology, so they can better define the operating domains.

Asked about the UL standard’s complementary facets, Magney said, “UL 4600, for example, addresses validation of AI trained methods, something that is not covered in ISO 26262” or SOTIF.

Why UL?

Without question, UL comes with a great heritage and reputation. But UL might not necessarily be the first industry organization that comes to mind as developer of standards for technology as futuristic and software-intensive as autonomous systems.

UL’s legend derives from its creation of well-established standards for wires and cables, heating and cooling equipment, industrial control equipment and life-safety devices like smoke detectors and smoke alarms.

Asked about UL, Magney said, “When it comes to safety, UL is the ultimate mark of safety in the eyes of consumers. I was a kid in the ’60s and my father always told me to look for the UL seal on products, and especially electronics. They have been around for more than 100 years, with their stamp of approval related to the safety of the products.”

Magney added, “But I don’t recall or have ever heard of UL being applied to cars. I think it makes pretty good sense because of UL’s background ensuring the safety of electronics products.”

It turns out that UL is already familiar with the automotive industry. It helps automotive manufacturers around the world attain electromagnetic compatibility (EMC) certification. UL can assess components from electric motors and ABS brake modules to complex infotainment modules with integrated wireless technologies. UL has heft in the automotive market, because its full-service EMC laboratories are accredited or recognized by numerous regulatory agencies.

Setting that aside, though, there are two specific reasons ECR’s Wagner and Koopman identified UL as a good place to start developing standards for autonomous products.

Being Nimble

First, UL is a “very nimble” organization, Koopman said.

In developing UL 4600, for example, Koopman said that ECR began its work last summer, “and I’ve been doing all the typing” to write up “a straw man” for the standard.

By mid-May, a first draft will be shown to “a balanced panel with representatives from relevant stakeholder groups and subject matter expertise,” according to Koopman. While no participant names have been unveiled, Koopman promised that there will be OEMs, full-stack vendors, first tier vendors, chip makers, government folks and others on the standard technical panel. The STP will go through three rounds of comment before a vote on the standard before the end of this year.

This fast-track process presents a sharp contrast to ISO national standards bodies, who spend more time building consensus among stakeholders.

Build-in Feedback Loop

A second advantage of working with UL, according to Koopman, is that “UL comes with a built-in feedback loop,” noted Koopman.

Recommended
Improving Safety Standards for AV

Inevitably, anything like UL 4600 faces a laundry list of what should or shouldn’t be done with autonomous systems to ensure safety. In developing safety systems, system designers will encounter stuff that poses the question, as Koopman notes, “Did you think of that?”

When each autonomous system vendor reports incidents – describing what broke or which bugs were found, what led to a crash— the data can be fed back into UL 4600, leading to as short as six-month updates of the specs. Koopman is fully aware that not every autonomous vehicle vendor, for example, might be willing to share “raw data,” but UL 4600 offers a forum where companies can inform one another, “Hey guys, if we did this, bad things happened,” much like airlines reporting hazards to the FAA. Based on the reports on hazards and its checklist, UL has a mechanism to improve the standard and close the loop inside the UL, explained Koopman.

While Koopman is the lead writer of the UL 4600 draft, he told EE Times that he has elicited help from numerous industries so he can incorporate existing best practices. He mentioned Uma Ferrell, who is a qualified Federal Aviation Administration (FAA) Designated Engineering Representative (DER), and a systems engineer at MITRE Crop.; and Frank Fratrik, lead engineer at Edge Case Research, who has 15 years of experience in defense system safety. Koopman cited additional contributions from a team of Underwriters Laboratories safety standards experts led by Deborah Prince, UL standards process manager.

Asked about chapters the UL 4600 draft standards will cover, Koopman listed the following:

• Safety case sufficiency
• Hazards and risk mitigation
• Development process
• Machine learning and “AI”
• Verification & Validation
• Dependability & security
• Data safety
• Non-driver human interface
• Metrics & assessment
• Tool qualification
• Maintenance
• Other lifecycle concerns
• Field data feedback
• Dealing with unknowns

SOTIF going forward

Mariani maintains that standards-setting activities are not a zero-sum game. He said, “I believe standardization bodies shall make every reasonable effort to regularly discuss with the other bodies in order to cross-check about standardization overlaps and avoid creating competing or even worse conflictual standards on same topic.”

He noted, “For this reason, in my current role of vice president for IEEE Computer Society standardization activities board and member of IEEE-Standard Association, I am strongly supporting the ongoing continued links with ISO and IEC, so that we make sure we don’t ‘over-standardize’ or create confusion in the community with overlapping standards.”

Mariani added, “I have no problems to push back on some new standards request if the topic is already covered by other bodies. This is my suggestion to UL as well, and I am sure this will be done, as already happened in the past with some other topics. In fact, I think nobody has the interest to create confusion in the complex and critical world of autonomous products.

Mariani, acknowledging that he has not been exposed yet to any UL 4600 details yet, said it is too soon to make any judgements. He defended SOTIF’s position by noting that SOTIF/PAS has been prepared through a rigorous and participatory process. The same, he said, is happening with the development of a formal international standard (ISO 21448 IS). He described SOTIF as “very comprehensive, and the state of art,” including “some of the emerging controversial issues of ADAS and automated driving functionalities.”

But that doesn’t exclude areas that “may require a more focused or prescriptive standardization effort, especially for cross-domains areas,” he pointed out. This applies especially to formal methods, use of artificial intelligence, tooling for automated dependable systems analysis and crowdsourcing techniques for automated products.

Mariani noted that the UL 4600 fact sheet does not mention ISO 21448 (while acknowledging ISO 26262 and other standards) “concerns me a bit.” However, he added, “Knowing Phil’s open attitude and competence, I am sure the standardization teams will interact to fix any critical divergence. I will promote that, and I look forward to cooperating with Phil to manage that risk.”