On-Chip Analytics to ‘See’ into Complex ML, AI Chips

Article By : Nitin Dahad

Heterogeneous multicore chips are becoming increasingly common, particularly in enabling the ML and AI technologies required in leading-edge applications such as driverless cars.

LONDON — UltraSoC has extended its on-chip analytics architecture to provide monitoring and analytics of complex machine learning (ML), artificial intelligence (AI), and parallel computing chips comprising up to 65,536 elements.

The new architecture, to be introduced in the company’s UltraDevelop 2 integrated development environment (IDE), allows system-on-chip (SoC) designers to build on-chip monitoring and analytics systems with up to 65,536 elements, allowing seamless support for systems with many thousands of processors. This makes it suitable for monitoring a massive number of the internal building blocks that make up the most complex SoC products — and analyzing the impact on system-level behavior of the interactions between them.

Such heterogeneous multicore chips are becoming increasingly common, particularly in enabling the ML and AI technologies required in leading-edge applications such as driverless cars. In addition to the sheer size of modern SoCs, the ML and AI algorithms are often inherently non-deterministic: Because they devise their own ways of solving problems by “learning,” it is impossible for the system’s original designer to predict how they will behave in the final application. In-life monitoring of the chip’s behavior is, therefore, the only way of getting a true picture of what is going on inside the chip and the wider system.

UltraSoC said that its system-level monitoring and analytics capabilities extend beyond the chip’s core processing components to all parts of the system — which may include thousands of IP blocks and subsystems, buses, interconnects, and software. The new features within the UltraSoC architecture allow chip designers to deploy tens of thousands of monitoring and analytics modules within a single infrastructure. By providing an integrated, coherent analysis of the system’s behavior, this reduces the development burden for next-generation ML and AI applications and allows the implementation of product features such as hardware-based security and functional safety.
Future iterations of UltraSoC’s architecture aim to allow even higher numbers of processors for exascale systems. In addition to this scaling capability, the company has added new system memory buffer (SMB) IP to allow the embedded analytics infrastructure to handle the high volumes of data generated by multicore systems and cope with “bursty” real-world traffic.

UltraSoC’s debugging and in-life monitoring now scales to systems with up to 65,536 IP blocks and elements. (Image: UltraSoC)

UltraSoC CEO Rupert Baines said, “Our solutions are unique in the market in their ability to deal with multiple heterogeneous processors, standard and proprietary bus structures, and even custom logic. This dramatic extension of our architecture takes us even further ahead of traditional solutions, both in the debug and development arena, and in allowing our customers to incorporate in-life monitoring capabilities to ensure security, functional safety, and real-world performance optimization.”

Dave Ditzel, founder and CEO of Esperanto, developer of energy-efficient high-performance computing systems for ML and AI applications, said that with their products incorporating over a thousand RISC-V processors and AI/ML accelerators on a single chip, UltraSoC’s ability to match that level of scaling with monitoring, analytics, and debug capabilities is a vital enabler for its business. Esperanto is the lead customer for its solution at present, with a system that has 4,000 cores with one architecture and 64 cores with another, all based on RISC-V.

The complex interactions between multiple hardware blocks, firmware, and software within SoCs have already made real-time in-life monitoring an indispensable tool for SoC designers. Changes in design approaches are also making system-wide monitoring more necessary than ever. Agile software development and ad hoc programming practices inherently require high-granularity visibility of the real system. Similarly, system hardware and software may not be “architected” in the traditional sense; again, engineers need clear visibility of the run-time behavior of their systems.