Meeting future workload demands: the case for emerging memory technologies (Part 1)
Author : Federica Monsone, A3 Communications
11 September 2025
Federica Monsone, A3 Communications
It often feels as though memory is an outlier in the technology world. While we’ve seen significant changes in compute power (both with CPU and GPU) and storage, memory development has been iterative rather than revolutionary.
While that approach has worked in the past, current memory technology is starting to cause challenges, due to an issue known as the “memory wall problem”. This occurs when a processor’s speed outpaces memory’s bandwidth and, as a result, the processor has to wait for data to be transferred from memory, introducing a bottleneck.
The performance restrictions caused by the memory wall problem are only getting worse, as CPU and GPU advancement continues to outpace improvements in memory architecture. And it’s being exacerbated by the growth of demanding, memory-intensive workloads such as high-performance computing (HPC) and AI, which didn’t exist at the same scales until relatively recently, but are now seeing rapid adoption.
This issue is creating the need for new memory technologies, designed for modern workloads.
“Emerging memory technologies are being driven by the explosive growth in AI and machine learning, big data analytics, scientific computing, and hyperscale cloud datacentres,” said JB Baker, VP Marketing and Product Management, ScaleFlux. “Traditional memory technologies like DRAM and NAND flash are reaching scaling, speed, and density limits that restrict the performance required for next-generation workloads.”
With AI demands highlighting the flaws in existing systems, Baker believes that new memory technologies are required.
“The needs for higher bandwidth, lower latency, greater capacity, and energy efficiency in AI and HPC applications are exposing the inadequacy of traditional solutions. We are indeed approaching the physical and economic end of the road for conventional memory scaling, making new architectures and technologies essential,” he noted.
While traditional memory technology still has a role to play, other factors are pushing the demand for emerging technologies.
David Norfolk, Practice Leader: Development and Governance, Bloor Research, explained, “AI hype is driving things - that and the need for vendors to sell something new with higher margins. I very much doubt that we are at the end of the road yet, but people always want more speed, scaling, and density. “What may be a driver, is more energy efficiency, less heat and more reliability - less waste.”
Defining the problem
High-performance workloads have numerous requirements, so no single emerging memory technology works across the board. For example, AI workloads require a significant volume of data, both for fresh processing and longer-term storage. That applies to both typical Generative AI (Gen AI) services, such as ChatGPT, and to a growing number of physical machines that collect data using sensors for decision-making. But it isn’t always clear what data is needed, when, and for how long it should be stored.
Martin Kunze, CMO, Cerabyte, explained, “It is not yet defined how much raw sensor data is needed for decision making, and how long it needs to be retained for when it comes to machine-human interaction. There were already legal consequences for companies that didn’t keep enough data to reconstruct accidents that were caused by false AI-decisions.”
Legal reasons, rather than purely technological ones, will have their part to play in how emerging memory technologies are provisioned and used.
“The ‘audit trail data’ will be one of many drivers that lead to the surging demand for data storage,” Kunze continued. “Current storage technologies are approaching their limits; analysts are forecasting a scenario where mainstream technologies can deliver only 50 percent of the required demand – a looming supply gap could put AI-related investments at risk.”
A tiered approach
Universal memory, which combines persistent memory and storage into a single unit, would seem to be the panacea, providing fixed storage for vast amounts of data and high speeds for processing on demand. However, that is unlikely to be a realistic proposition for some time, so tiered data using a variety of technologies will be the default in the short-to-medium term.
Arthur Sainio and Raghu Kulkarni, Persistent Memory Special Interest Group co-chairs, SNIA (The Storage Networking Industry Association), said, “Universal memory such as PCM and ULTRARAM promises to merge RAM speed with persistence, but faces manufacturing complexity, high costs, and scaling barriers. Tiered architectures will dominate short-to-medium term due to cost efficiency. Universal memory may see niche use (edge AI, aerospace) but requires material breakthroughs to displace tiered systems, likely post-2030. Hybrid solutions like CXL PMEM + DRAM + SSDs, remain the pragmatic path.”
While technological issues impede such a technology, there’s also a concern from some that this memory type might inhibit performance.
“While the concept of universal memory is intellectually appealing, in practice we are likely to maintain a tiered storage architecture for the foreseeable future,” said Baker. “The technical gap between DRAM-class speeds and persistent storage-class latencies remains too large to collapse into a single layer without major compromises.”
While universal memory may not be an immediate solution to the memory wall problem, emerging memory technologies still have a big role to play, particularly in how tiers interoperate.
“Emerging memory technologies may narrow this gap, but they are more likely to create new tiers rather than eliminate the concept of tiered architectures altogether,” continued Baker.
Most experts agree that tiering with AI workloads will be different from traditional ones.
"In a future coined by AI, the typical segmentation of hot - warm - cool - cold data will very likely be increasingly blurry. Large chunks of cold data need to be warmed up quickly, and then after being processed, put back in cold storage. For example - to enhance AI training or AI-assisted search, for patterns in scientific data or to present sensor data was the basis for AI decisions in a liability case in court,” said Kunze.
When considering emerging memory, or indeed any future technology, it’s natural to expect higher performance will be one of the most significant new features being offered.
But that may not be the case, since the demand for greater performance, higher capacity, and reduced overall costs, means that vastly different technologies have their part to play, as Erfane Arwani, founder and CEO, Biomemory, explained: "DNA storage isn’t fast, but it’s insanely dense and lasts forever. Perfect for archiving AI models and massive datasets you don’t need to access often.”
Persistent memory
Fully fledged universal memory that can do everything required of traditional storage and memory may be a long way off, but there are alternative technologies, such as persistent memory, which is designed to retain data without requiring constant power (i.e. it’s non-volatile). Persistent memory promises to bridge the gap between storage and memory. But while this idea sounds great, it has been a rocky road for this technology so far, with the best-known example, Intel Optane, being abandoned.
As Joseph Lynn, Executive VP of Operations, Tarmin, explained, “Several factors contributed to this. Optane faced challenges due to its relatively high cost per GB compared to NAND flash, making it less appealing for capacity-sensitive applications. Further, when used in DIMM configuration, its performance, while better than NAND, did not fully match DRAM, so the performance/cost-benefit could not always be justified, when considering memory capacity and latency.”
These kinds of issues seem to be universal with current persistent memory technologies, preventing them from mass uptake and universal appeal, as explained by Baker. “Major limitations include: high bit cost, which makes scaling economically challenging; long read/write latencies, which cannot match the speed requirements of latency-sensitive applications; and low read/write throughput, which bottlenecks throughput-intensive applications,” he said. “Emerging alternatives that offer better density, faster access, and lower energy per operation are increasingly attractive for AI and HPC workloads.”
So, again, we come back to the need for tiering, with emerging persistent memory technologies able to work in specific tiers. That includes slower, data-rich tiers, as Arwani noted: “DNA shows that tiered storage still makes sense. It’s ideal for the coldest layer - super dense, low-energy, and long-lasting.”
Faster persistent memory technologies have their place, although there are still hurdles that need to be overcome.
“Emerging alternatives like MRAM and ReRAM provide advantages such as near-SRAM speed, zero standby power in the case of MRAM, and analogue compute capabilities like ReRAM, but face scalability and manufacturing hurdles. They are gaining some traction as they promise better scalability, energy efficiency, and performance for future HPC demands, but have hurdles to overcome,” said Saino and Kulkarni. “CXL NV-CMM types of products offer DRAM-like speed and persistence, making them valuable for caching and checkpointing functions in HPC applications. High density hybrid CXL solutions are likely as well.”
Contact Details and Archive...