Memory

The AI memory squeeze: DRAM prices, cloud costs and what it changes for JVM teams

Memory · Published 2026-10-24

Since late 2025, memory makers have sent a growing share of their output to AI datacentres, and DRAM prices have climbed at a pace the industry had not seen before. Cloud list prices for ordinary instances have not followed so far, but the cost of memory now drives hardware budgets. This guide gathers the sourced figures, each with its date, and then turns to what JVM teams control: how many gigabytes a service actually needs.

Where the memory goes

AI accelerators use high-bandwidth memory (HBM), stacked DRAM packaged next to the GPU, and the servers around them use large registered DIMMs (RDIMM). Both are made on the same wafers as the DRAM that ends up in laptops and general-purpose servers. In a press release of 25 August 2026, TrendForce estimated that HBM and RDIMM combined will account for 51% of DRAM bit supply in 2026. The same release projects that DRAM and NAND flash combined will account for 47% of the major cloud providers' total capital expenditure in 2026 and 68% in 2027.

The makers' own results tell the same story. SK hynix, reporting on 29 July 2026, posted second-quarter revenue of 79.3187 trillion won, up 257% year on year, with an operating margin of 76%. It began mass shipments of HBM4 in the quarter and said that customer demand exceeds its supply capabilities. Samsung's second-quarter 2026 release (30 July 2026) reports KRW 127.5 trillion of revenue and KRW 89.2 trillion of operating profit for its DS (semiconductor) division and expects the memory market to stay undersupplied. Micron's fiscal fourth quarter, which ended on 3 September 2026 and was reported on 30 September 2026, brought revenue of $54.23 billion against $11.32 billion a year earlier, with a GAAP gross margin of 86.8%.

Contract and spot prices

Most DRAM is sold on quarterly contracts; the spot market is smaller and moves first. The TrendForce figures below are as published, forecasts as forecasts:

On the spot market, TrendForce's weekly update put the average price of the mainstream chip it tracks, DDR4 1Gx8 3200MT/s, at US$7.931 in its update of 22 October 2025 and at US$46.357 in the week of 20 October 2026. The later update adds that DDR5 supply remains tight and that suppliers are reluctant to sell. The direction is clear: prices are still rising, but each quarter's increase is smaller than the one before.

What a gigabyte costs in the cloud

The three large providers price general-purpose and memory-optimised instances side by side, so at a fixed vCPU count the price steps show what extra memory costs. On-demand Linux prices from the public price pages, read on 24 October 2026:

Azure's Retail Prices API gives 1 October 2024 as the effective start date of both Azure rates, so for those two sizes the list price predates the rise in memory prices. Committed-use and reserved prices are lower than these figures. For a JVM team, the useful point is simpler: going from 16 to 32 GiB on the same vCPUs is a separate, visible step on every price list. A service that fits in the smaller size, with headroom, never pays it.

Spend fewer gigabytes per service

Most JVM services are given far more memory than their live data needs, because the safe answer to an OOM kill is a bigger limit. The tools to size them properly are already in the JDK.

Measure before cutting

Run the service under realistic load with -XX:NativeMemoryTracking=summary and read the committed values per category at peak, as described in Native Memory Tracking. Take the heap's live set from the GC log (the occupancy after a full or mixed collection; see reading GC logs). These two numbers, plus a margin, are the budget. Anything above them is memory you pay for and do not use.

Size the heap from the live set, not the limit

The default heap is 25% of the container's memory, and many teams answer that by raising MaxRAMPercentage and then raising the limit as well. Do it the other way round: decide the heap from the live set and the allocation rate, add the non-heap budget, and set the container limit to the total. The JVM in containers has the full budget table. If -Xms equals -Xmx, the whole heap is committed from the start; leave them different when you want an idle service to use less.

Cap off-heap memory explicitly

Direct buffers default to a limit roughly equal to the maximum heap, which means a container sized for the heap alone can be killed during a burst of I/O. Set -XX:MaxDirectMemorySize, -XX:MaxMetaspaceSize and, where relevant, -XX:ReservedCodeCacheSize from measured values. Explicit caps turn a silent container kill into a diagnosable OutOfMemoryError; see direct buffer memory.

Shrink every object: compact object headers

JEP 450 (JDK 24) reduced the object header in HotSpot from between 96 and 128 bits to 64 bits on 64-bit platforms, as an experimental option. JEP 519 made it a product feature in JDK 25, so -XX:+UnlockExperimentalVMOptions is no longer needed:

# JDK 25 and later
-XX:+UseCompactObjectHeaders

JEP 519 cites results from experiments: in one SPECjbb2015 setting, 22% less heap space and 8% less CPU time; in another, 15% fewer garbage collections with both G1 and Parallel. It is not the default layout. Services with many small objects (maps, DTOs, boxed values) gain most; measure your own live set before and after, on a canary.

Share class metadata: CDS

Class Data Sharing maps a pre-built archive of class metadata into the process. The JDK documentation explains that part of the archive is mapped read-only and shared among JVM processes on the same host, which reduces memory footprint when several JVMs run side by side, and that a default archive ships with the JDK since JDK 12. On a node running many replicas of the same image, an application archive (or the AOT cache from JEP 483, JDK 24, which also speeds up startup) is cheap to adopt.

Choose a collector that gives memory back

A heap that grew during a peak stays committed unless the collector returns it. JEP 346 (JDK 12) lets G1 return unused committed memory when idle through periodic collections set with -XX:G1PeriodicGCInterval (in milliseconds); the feature is disabled by default, and the JEP's motivation is precisely containers where resources are paid by use. JEP 351 (JDK 13) made ZGC uncommit unused memory, enabled by default. For small, single-CPU containers the JVM already picks Serial, which has the smallest footprint; choosing a collector covers the trade-offs.

Checklist

• Record NMT committed totals and the GC live set under load; write the budget down.
• Set the container limit from the budget, not the heap from the limit.
• Cap direct memory, Metaspace and code cache explicitly.
• On JDK 25 or later, test -XX:+UseCompactObjectHeaders on a canary and compare the live set.
• Use a CDS archive or AOT cache where many replicas share a host.
• Let idle services return heap: G1 periodic collections, ZGC uncommit, or Serial in small containers.
• Check whether the service still fits the next size down on the price list.

Sources

• TrendForce press releases: 31 March 2026, 1 June 2026, 9 July 2026, 25 August 2026, 30 September 2026
• TrendForce spot price updates of 22 October 2025 and 21 October 2026
• SK hynix: 2Q26 financial results (29 July 2026)
• Samsung Electronics: second quarter 2026 results (30 July 2026)
• Micron: fiscal Q4 and full year 2026 results (30 September 2026, Form 8-K exhibit)
• Price pages read on 24 October 2026: Amazon EC2 On-Demand pricing, Google Compute Engine general-purpose pricing, Azure Linux virtual machines pricing and the Azure Retail Prices API; sizes from the Dsv6 and Esv6 series pages
• JEP 450, JEP 519, JEP 483, JEP 346, JEP 351
• JDK documentation: Class Data Sharing

Read next

More in Memory

Production

Keeping AI products affordable

Batching, quantisation and caching on the model side; heap, GC and container sizing for the Java services around it.

Production

The JVM in containers

How limits are detected, what MaxRAMPercentage really sets, and a budget that keeps the OOM killer away.

Memory

Native Memory Tracking

Heap is only part of the process: read the NMT summary, diff against a baseline, and know what it cannot see.