Employing network-on-chip (NoC) technology in system-on-chip (SoC) designs has been proven to reduce routing congestion and lower power consumption. Now, a new NoC-enabled tiling methodology helps speed development, facilitates scaling, participates in power reduction technology and contributes to increased design reuse for SoCs targeting artificial intelligence (AI) applications.
For these discussions, we will assume that AI encompasses use cases such as machine learning (ML) and inferencing.
Soft and hard tiles
One challenge in engineering is that the same term may be used to refer to different things. The term “tile,” for example, has multiple meanings. Some people equate tiles with chiplets, which are small, independent silicon dies, all presented on a common silicon or organic substrate or interposer. Chiplets may be thought of as “hard tiles.”
By comparison, many SoCs, including those intended for AI applications, employ arrays of processing elements (PEs), which can be considered “soft tiles.” For example, refer to the generic SoC depicted in Figure 1.
Figure 1 High-level block diagram shows SoC containing a neural processing unit (NPU). Source: Arteris
In addition to a processor cluster comprising multiple general-purpose central processing units (CPUs), along with several other intellectual property (IP) blocks, the SoC may also contain specialized processors or hardware accelerators. These units include an image signal processor (ISP), a graphics processing unit (GPU) and a neural processing unit (NPU), designed for high-performance, low-power AI processing.
In turn, the NPU comprises an array of identical PEs. In the not-so-distant past, these PEs were typically realized as relatively simple multiply-accumulate (MAC) functions, where MAC refers to a multiplication followed by an addition. By comparison, today’s SoCs often contain PEs with multiple IPs connected via an internal NoC.
Implementing soft tiling by hand
In the common SoC scenario we are considering here, NoCs may be employed at multiple levels in the design hierarchy. For example, a NoC can be used at the top level to connect the processor cluster, ISP, GPU, NPU and other IPs. NoCs may be implemented in various topologies, including ring, star, tree, mesh and more. Even at the top level of the SoC hierarchy, some devices may employ multiple NoCs.
As has already been noted, each PE in the NPU may consist of multiple IPs connected using an internal NoC. Furthermore, all the PEs in the NPU can be connected using a NoC, typically implemented as a mesh topology.
The traditional hand-crafted approach to implementing the NPU starts by creating a single PE. In addition to its AI accelerator logic, the PE will also contain one or more network interface units (NIUs) to connect the PE to the main mesh NoC. This is illustrated in Figure 2a.
Figure 2 This is how designers implement soft tiling by hand. Source: Arteris
If we assume that the NPU specification calls for a 4×4 array of PEs, the designer will replicate the PE 16 times using a cut-and-paste methodology (Figure 2b). Next, NoC tools will be used to auto-generate the NoC (Figure 2c). During this process, the NoC generator automatically assigns unique identifiers (IDs) to each of the NoC’s switching elements. However, the NIUs in the PEs will still have identical IDs; that is, the default ID from the PE’s creation.
For the NoC to transfer data from source nodes to destination nodes, the NIU in each PE must have a unique ID. This requires the designer to hand-modify each PE instance to provide it with its own ID. In addition to being time-consuming, this process is prone to error, which can impact downstream testing and verification.
This hand-crafted tiling technique poses several challenges. For example, changes to the PE specification are often made early in the process. For each change, the designer has two options: (a) manually replicate the change across all PE instances in the array, or (b) modify only the original PE and then repeat the entire hand-crafted soft tiling process. Both options are time consuming and error prone.
Also, performing soft tiling by hand is not conducive to scaling. If it becomes necessary to replace the original 4×4 array with an 8×8 version, such as for a derivative product, the process becomes increasingly cumbersome and problematic.
NoC-enabled tiling
The phrase “NoC-enabled tiling” refers to an emerging trend in SoC design. This evolutionary approach uses proven, robust NoC IP to facilitate scaling, condense design time, speed testing and reduce design risk.
NoC-enabled tiling commences with the designer creating a single PE as before. In this case, however, the NoC tools can be used to automatically replicate the PEs, generate the NoC and configure the NIUs in the PEs, all in a matter of seconds. The designer only needs to specify the required dimensions of the array.
Figure 3 This is how NoC-enabled tiling is carried out. Source: Arteris
In addition to dramatically speeding the process of generating the array, this “correct by construction” approach removes any chance of human-induced errors. It also enables the design team to quickly and easily accommodate change requests to the PE early in the SoC development process. Furthermore, it greatly facilitates scaling and design reuse, including the creation of derivative designs.
An evolving market
Based on an analysis of AI SoC designs currently under development by their customers, the Arteris team has determined the relative use of soft tiling in key verticals and horizontals for AI today. This is illustrated in Figure 4, where the areas of the circles reflect the relative number of application use cases.
Figure 4 NoC-enabled tiling is shown in key verticals and horizontals for AI today. Source: Arteris
Designing multi-billion-transistor SoCs is time-consuming and involves many challenges. Some SoC devices, such as those intended for AI applications, may include functions like NPUs that comprise arrays of PEs. Here, NoC-enabled tiling is an emerging trend and it’s supported only by leading NoC IPs and tools.
Andy Nightingale, VP of product management and marketing at Arteris, has over 37 years of experience in the high-tech industry, including 23 years in various engineering and product management positions at Arm.
Related Content
- SoC Interconnect: Don’t DIY!
- What is the future for Network-on-Chip?
- SoC design: When is a network-on-chip (NoC) not enough
- Network-on-chip (NoC) interconnect topologies explained
- Why verification matters in network-on-chip (NoC) design
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>
The post Speeding AI SoC development with NoC-enabled tiling appeared first on EDN.