Applications in the AI era are memory starved. They need more capacity and, in many cases, more effective memory bandwidth than traditional server designs. Generative AI and large language models (LLMs) store trillions of parameters that must be accessed frequently. And real-time inference for translation, chatbots, and similar services demands low-latency memory paths.
Larger memory footprints enable bigger batch sizes, improving training and inference efficiency. However, while the fastest memory sits on die—in CPU caches—capacity there is inherently limited. For decades, systems have relied on double data rate (DDR) interfaces for high-capacity, relatively low-latency off-chip memory.
Adding more DDR to a CPU, however, runs into hard limits: each interface consumes scarce I/O pins (288 pins on modern DDR4/DDR5 modules), demands strict signal integrity at ever-higher data rates, and carries significant cost. An alternative that has gained real traction is Compute Express Link (CXL) Type 3 memory expanders—devices that attach over a CXL link and present additional system memory to the host.
CXL is a cache-coherent interconnect that lets a CPU access device-resident resources with semantics closer to memory than traditional PCI Express (PCIe) peripherals. A Type 3 memory expander exposes DRAM (and, in some designs, other media) as byte-addressable memory that firmware and the operating system can map and allocate like conventional RAM—after discovery, decode programming, and policy setup.
The critical implication for bring-up and validation is that this memory is logically host-visible yet physically and administratively distinct from local dual inline memory modules (DIMMs). That distinction affects discovery, non-uniform memory access (NUMA) topology, performance, and error-handling paths through firmware and the OS.
Figure 1 The memory-latency pyramid describes relative ordering and validation implications. Source: arXiv
The memory latency–capacity pyramid
System memory is best understood as a latency–capacity pyramid. Small, fast, most expensive structures—CPU caches—sit at the top. Larger, slower, progressively cheaper tiers sit below; local DRAM, then expansion memory, and then I/O-backed storage. Absolute nanoseconds vary by CPU generation, CXL version, link width, topology (direct attach versus retimer or switch), firmware tuning, and contention; the pyramid describes relative ordering and validation implications, not a single fixed latency table.
Local DDR, typically attached near the CPU socket, offers the lowest DRAM access times the OS sees for general-purpose allocation. CXL Type 3 expander memory is DRAM-class and byte-addressable from software’s perspective, but it’s reached across a CXL fabric hop (often with additional buffering and coherency handling).
It therefore sits below local DDR—higher average and tail latency, sometimes behaving like “far memory” in NUMA terms. In other words, imaging CPU 0 accessing DDR memory is attached to CPU 1 in the system, as shown in Figure 2.
Figure 2 Here is a typical two-socket system with a CXL memory expander device attached to CPU 0 via a Gen5, x16 CXL bus. Source: Author
For bring-up, that placement matters as correctness tests may pass while performance and quality-of-service (QoS) tests fail. Workloads with pointer chasing, fine-grained random access, or strict tail-latency budgets are the first to expose suboptimal placement, interleaving, or contention.
Storage and networked memory (NVMe, RDMA, and similar) form the broad base of the pyramid with much higher latency and usually block or page semantics. CXL memory is not in the same tier as SSDs, but it’s meaningfully different from local DIMMs for latency-sensitive software. On a typical two-socket system, the access latency of DDR behind a CXL device can be comparable to accessing DDR attached to the adjacent CPU—a useful mental model when setting performance expectations.
Platform prerequisites: A cross-layer contract
Whether CXL Type 3 memory becomes reliably visible, addressable, and serviceable depends on aligned support across the stack: CPU CXL capability and enablement; system BIOS/firmware support for discovery, decode, and ACPI tables; kernel CXL enumeration and memory management; and expander device firmware for DRAM training, HDM reporting, and mailbox/DOE services. All layers must agree.
Consider CXL Integrity and Data Encryption (IDE); it requires CPU support, BIOS enablement, and device firmware support to be usable end to end. Similarly, the kernel needs a CXL-aware path to recognize the device class, bind memory resources, and transition capacity to an online state the allocator can use.
Reliability, availability, and serviceability (RAS) matter equally. Corrected and uncorrected error notifications must propagate from hardware through firmware to OS subsystems that can log, isolate, or offline affected regions. Because behaviors evolve quickly across kernel releases, validation plans should treat OS version, configuration (huge pages, numactl policies, memory mode), and boot/firmware settings as explicit test variables. Failures are often misattributed to the expander when the root cause is a policy or enablement gap.
Host-managed expander memory generally relies on the in-kernel CXL/memory management stack rather than a monolithic device-specific driver, though platform integrations may include monitoring agents, telemetry exporters, or hardware management interfaces that affect how engineers observe link state, temperature, power, and error counters during bring-up.
Linux and the NUMA story
On Linux, a Type 3 memory expander normally appears as a PCI/CXL function. In upstream kernels with CXL support enabled, the in-tree cxl_pci module is the default bind target. A stock Type 3 host-managed device (HDM) endpoint typically comes up under cxl_pci rather than a vendor-specific host driver for basic enumeration.
The cxl_pci module is PCI-facing glue: it attaches to the device, brings up CXL.io access (including the configuration mailbox), and registers the endpoint with the CXL core so the rest of the stack can expose memory devices to the OS.
In a NUMA machine, the operating system groups CPUs and memory into nodes and treats local memory as cheaper than remote memory. DRAM next to a socket is usually the lowest-latency memory for CPUs in that socket, so the scheduler and allocator try to keep threads and pages on nearby nodes (subject to policy).
CXL Type 3 expander memory is still host-coherent and byte-addressable, but it’s physically and topologically distinct from local DIMMs. Platforms and operating systems therefore commonly expose expander memory ranges as a distinct NUMA node, or as memory with different affinity and distance metadata in ACPI proximity hints. The same application binary can run correctly while performance changes sharply depending on where pages are allocated and whether threads migrate across sockets.
For CXL bring-up and validation, the NUMA story is central. Issues often appear as unexpected remote access or imbalanced bandwidth rather than hard functional failures. Engineers must verify not only that memory is online, but that placement and distance metadata match the intended system topology.
What comes next
Part 2 of this series introduces the user-space tooling ecosystem—cxl/libcxl, ndctl, daxctl, numactl, and topology helpers—and traces the full boot sequence from slot power and DRAM training through DVSEC discovery, decode programming, CDAT delivery, ACPI table handoff, and OS driver binding. Part 3 turns to practical test and debug: interpreting lspci output, validating HDM ranges, exercising CXL-attached memory with numactl, and selecting bandwidth and stress tools for validation gates.
Together, these three parts provide a vendor-neutral, OS-focused playbook for engineers, bringing CXL Type 3 memory expanders from first power-on to production-ready validation.
Ameet Sanghavi works in post-silicon validation for PCIe and CXL at Nvidia with a focus on interface bring-up and validation on shipping products. He has worked on PCIe since 2005 (from PCIe 1.1 onward) and on CXL since 2020 (from CXL 1.1 onward).
Editor’s Note
This is Part 1 of the mini-series on CXL Type 3 memory technology. Part 2 of this series introduces the user-space tooling ecosystem. And Part 3 turns to practical test and debug work.
The views and content of the article are the author’s own and not affiliated to any of his current or previous employers.
Related Content
- Stripped Down CXL Scales Memory Wall
- CXL Efforts Focus on Memory Expansion
- CXL Adds Port Bundling to Quench AI Thirst
- CXL: The key to memory capacity in next-gen data centers
- From GPUs to Memory Pools: Why AI Needs Compute Express Link (CXL)
The post Why CXL Type 3 memory matters, what your platform must provide appeared first on EDN.