Analysis
Stored attention state was part of SK hynix’s pitch for shared inference memory at the October 12–15 OCP Global Summit in San Jose. Its MARS demonstrations explored transferring and reusing that cache across servers and holding it across DRAM and SSD tiers.
The company’s lineup also included second-generation CMM-DDR5 and CMM-Ax. Hosik Kim, vice president and head of Memory Systems Research, presented Introduction to Memory-centric AI Rack Systems in the summit executive session. SK hynix describes proof-of-concept work with partners, including efforts to reduce repeated transfers when work moves between models.
The appeal is reuse: retaining useful state could avoid some repeated work. Whether sharing pays off depends on how that state is reached. How would access across the fabric affect latency? Would competing hosts strain capacity, and how much overhead would the software introduce?
Recovery poses another question. If several workloads depend on a shared tier, its loss could have implications beyond one server. The scope would depend on the design; the demonstrations do not establish how eviction, inconsistent state or node loss would be handled.
Independent production benchmarks and broad availability remain unestablished. A comparison with additional local memory would help show where reuse justifies the extra coordination. For now, the demonstrations leave that tradeoff open.
Source: “Memory-Centric Server Systems Are the Future”: SK hynix Presents Its Next-Generation AI Infrastructure Vision at OCP Global Summit 2026 · Primary source context · attributed account