Meeting future workload demands: the case for emerging memory technologies (Part 2)
Author : Federica Monsone, A3 Communications
11 September 2025
Federica Monsone, A3 Communications
In the second part of this two-part article, Federica Monsone of A3 Communications continues to garner the viewpoints of key players in the memory space about the evolution of memory technologies.
No new architectures
One thing that seems clear is that there will not be a new server architecture for HPC and AI workloads that replaces what we have today. Advances in CPU and GPU technology, and large investments in such platforms, still make general-purpose computing the best fit for most jobs.
As such, some emerging memory technologies are likely to be more of niche interest, for custom jobs that require the fastest speeds. Computational-RAM (CRAM), where computations can take place directly in RAM, is a good example of this.
“Although CRAM offers compelling advantages for AI inference and acceleration in theory, it suffers from very limited programmability and restricted workload flexibility. As a result, CRAM is unlikely to replace the traditional server architecture for general HPC. Instead, it will at most be deployed selectively for niche applications,” said JB Baker, VP Marketing and Product Management, ScaleFlux.
Effective scaling and higher density
Irrespective of this, AI and HPC are pushing the requirements for more memory and require more flexible ways of using it. In that regard, continuing to push the boundaries of today’s memory technologies makes sense, as it can help maximise investment in current computing architecture.
At the core of memory development are two technologies that can help: 3D DRAM for increased capacity and Compute Express Link (CLX) for improved scaling and memory pooling.
“HPC and AI require both 3D DRAM for capacity and bandwidth, and CXL for scalable, cost-effective memory expansion. 3D DRAM such as HBM3, is ideal for on-package, high-speed tasks like training large AI models due to its fast data access and energy efficiency. CXL will provide pooled memory for flexibility and persistent workloads,” said Arthur Sainio and Raghu Kulkarni, Persistent Memory Special Interest Group co-chairs, SNIA (The Storage Networking Industry Association). “A hybrid approach that combines these technologies is essential for efficiently meeting the growing demands of modern HPC and AI applications.”
Emerging technologies also promise to maximise the investment in existing storage, which is particularly important given the need for a tiered approach to modern workloads.
Martin Kunze, CMO, Cerabyte, gives an example: "Emerging technologies such as ceramic storage can release expensive high performing storage like HDDs which today is used for storing cold data, for a better use.”
Emerging memory technologies also promise improved caching and access to data available on traditional storage technologies, such as flash and hard disks.
“Advanced caching strategies leveraging faster memory types - such as HBM or stacked DRAM - can significantly accelerate access to hot data, improving the performance of existing storage systems. Using persistent memory for metadata acceleration or tiered caching layers will continue to enhance storage efficiency without fundamentally redesigning architectures,” said Baker.
Software is critical
While hardware may steal the limelight, software is essential in provisioning and managing data tiers. Crucially, software has to make life easier and work with what’s available, rather than changing how systems work.
This is a valuable lesson learned from Intel Optane, as Joseph Lynn, Executive VP of Operations, Tarmin, explained: “A final hurdle for Optane was the slow adaptation of the software ecosystem. While, in theory, Optane DIMMs expanded memory transparently to the OS, in practice, optimising databases and file systems to take full advantage of its unique persistence and performance characteristics proved to be complex and time-consuming, further hindering its widespread and effective use.”
Software is critical to the success of any technology, particularly in a future where resources must be efficiently combined across different platforms.
“Software optimises workloads across CPUs, GPUs, TPUs, and CRAM by managing resources, scheduling tasks, and improving memory use. Tools like Kubernetes and TensorFlow ensure efficient hardware utilisation, while future innovations in AI-driven orchestration, unified APIs, and real-time monitoring will enhance performance and energy efficiency across heterogeneous platforms,” said Sainio and Kulkarni.
Barriers to uptake
While the AI explosion may make adoption of emerging memory technologies a forgone conclusion, there are still many risks, particularly around the investment in existing memory technologies. Demand for new technologies can be limited by what’s currently working.
A general resistance to new technology is something noted by David Norfolk, Practice Leader: Development and Governance, Bloor Research, who highlighted that one of the biggest barriers to adoption is “The amount of legacy tech still in use and working ‘well enough’ in many applications. Plus, general mistrust of anything too new unless there is no alternative.”
In a similar vein, new technology has to be demonstrably better than what’s available now. As Baker said, “New memory technologies must not only outperform but also offer acceptable economics compared to DRAM or NAND to achieve widespread adoption.”
These are factors that we’ve seen time and time again, but failure to invest in emerging technologies poses a risk of its own. As Kunze explained, “100x more money is invested in computation than in memory and storage. But without investment in newly scalable technologies, billions of investments in AI could be squandered due to lack of storage. This looming risk should be exposed to and explored in the AI and AI-Investor community.”
The future is coming
Despite these warnings, the requirements of demanding computing workflows are only exacerbating the memory wall problem, increasing the need for novel solutions. Emerging memory technologies are required now more than ever, and wider adoption is only a matter of time.
"Looking five years ahead, the confluence of ever-increasing data intensity and the scaling of datasets suggests we are indeed on the cusp of a transformative period in memory technology, arguably the most significant in a generation. This relentless growth in data demands will necessitate radical advancements and new architectural approaches to overcome the limitations of current memory systems,” said Lynn.
Developments in scalability and density must be priorities for any new technology looking to successfully tackle the memory wall challenge. Thankfully, the building blocks of these technological advancements are already available.
“Breakthroughs in CXL-based memory and Racetrack memory could transform the industry. CXL will enable scalable, low-latency persistent memory integration, while Racetrack memory offers ultra-high density, faster speeds, and energy efficiency. These advancements can revolutionise AI, HPC, and edge computing performance,” said Sainio and Kulkarni.
It’s important to think about how data will be used to understand the future of emerging memory technologies, as Kunze explained: "There will be ‘hot storage’ and ‘not so hot storage.’ The distinction between hot and cold storage/data will disappear; rather, data will be classified by the need to make it immediately accessible or not.”
As a result, the future looks set to be based on multiple technologies, with tiering used to hit different requirements at different points in a system. That means emerging memory technologies, but also continuing to push the limits of what today’s technology can offer.
“We expect there will be more flavours of persistent and volatile memory. They will be based primarily on DDR cell but also NAND cells,” said Baker. “The objective of DDR based memory will offer lower power, slower performance vs DRAM and cost vs a standard DRAM. It will reside between DRAM and NAND in the compute hierarchy. The innovation on NAND memory will target to expand the bandwidth in the overall compute hierarchy to meet the needs of AI and in-memory databases.”
Conclusion
You’ll probably be disappointed if you are expecting a new, emerging memory technology to become standardised in the near future. For the time being, the traditional tiered memory architecture isn’t going anywhere, and will continue to see iterative improvements to boost speed and capacity.
But, equally, the ever-growing demands of AI and HPC workloads mean that there’s a sense of urgency to solve the performance bottlenecks with current memory designs.
Held back by issues such as high costs, limited software support and a general resistance to technological change, emerging technologies have not caught on quite yet.
That said, there is clearly a sense that change is inevitable, sooner or later, and various approaches could be adopted to address the bottlenecks of current memory in the future.
Contact Details and Archive...