Chapter 8
Memory: HBM, DRAM, NAND
Chapter 8: Memory: HBM, DRAM & NAND
8.1 Overview
Memory is the fuel that feeds AI accelerators. A GPU without sufficient memory bandwidth is like a jet engine without air intake. The compute exists but cannot be utilized. High Bandwidth Memory (HBM), stacked vertically using through-silicon vias (TSVs) and bonded directly adjacent to GPU dies via advanced packaging (Chapter 9), provides the TB/s-class bandwidth that makes AI training and inference at scale possible. HBM ranks alongside CoWoS packaging (Chapter 9) as one of the two binding constraints on GPU output volume. The cross-cutting analysis (Chapter 20) scores both in Tier 1.
This chapter covers three product categories: HBM (the AI-critical segment, growing fastest), DRAM (the broader market that HBM cannibalizes), and NAND flash (important for AI data pipelines but not a direct bottleneck). The memory industry is an oligopoly: three companies, SK Hynix, Samsung, and Micron, produce essentially all of the world’s DRAM and HBM. There is no fourth supplier.
The AI buildout’s impact on memory is transformative. HBM production requires roughly twice the wafer area of standard DRAM for the same number of bits, and current yields hover between 50-60% 1. Every HBM chip manufactured effectively cannibalizes the capacity for 3-4 standard PC DRAM chips. SK Hynix and Samsung are reallocating up to 40% of their advanced wafer capacity toward HBM 1. This reallocation is creating shortages in standard DRAM, driving up prices across the entire memory market and forcing Micron to exit the consumer memory business entirely (December 2025) to focus on AI data center customers 2.
8.2 Market Sizing & Growth
HBM market: Valued at approximately $35-38 billion in 2025, projected to reach $58 billion in 2026 and potentially $100 billion by 2028 (~40% CAGR) 234. HBM3E is the current mainstream product (used in NVIDIA H200/Blackwell, AMD MI350), with HBM4 entering production in 2026. Analysts forecast 2026 HBM revenue mix at approximately 55% HBM4 and 45% HBM3E 5, though other sources project HBM3E at two-thirds of shipments with HBM4 gradually increasing share 6.
HBM pricing: HBM3 costs approximately $200 per 24GB stack ($8-10/GB). HBM3E costs approximately $300 per 36GB stack. HBM4 is estimated at ~$500 per stack 3. Samsung and SK Hynix plan approximately 20% HBM3E price hikes for 2026 as demand continues to exceed supply 5.
Total DRAM market: Approximately $100-110 billion in 2025, with HBM representing roughly one-third of total DRAM value despite being a fraction of total bit volume. SK Hynix overtook Samsung as the world’s largest DRAM manufacturer by revenue for the first time in Q1 2025 (36% share vs Samsung’s 34%), a reversal of a nearly 40-year Samsung leadership position, driven entirely by HBM 24.
Agentic AI DRAM demand multiplier: Morgan Stanley (April 2026) projects that agentic AI workloads will drive 15-45 exabytes of additional DRAM demand by 2030, equivalent to 26-77% of 2027 annual DRAM supply 11. The mechanism is distinct from the HBM demand described above: agents require persistent context memory across sessions, shared state between cooperating agents, and continuous learning from past interactions. This makes DRAM an active system component rather than passive storage. The 70% of incremental DRAM bit shipments Morgan Stanley attributes to the agentic theme represents a second demand vector on top of HBM, which could stress DRAM supply even as HBM cannibalizes conventional capacity 11. A counterweight to this projection: software optimizations in KV-cache management, prompt caching, model distillation, and context compression could substantially reduce per-agent memory requirements. The 15-45 exabyte range assumes current agent architectures scale without efficiency gains, which is unlikely. The directional thesis (agents increase DRAM demand) is sound; the magnitude is uncertain.
NAND flash: Approximately $70-80 billion market. Less directly relevant to the AI accelerator buildout but important for data storage in AI training pipelines (large datasets must be stored and accessed rapidly). Enterprise SSD demand is growing as AI clusters require high-capacity, high-throughput storage.
8.3 Supply Chain Flowchart
SILICON WAFERS (Chapter 3)
| 300mm wafers from Shin-Etsu, SUMCO, GlobalWafers
|
v
DRAM FABRICATION
| SK Hynix: Icheon, Cheongju (Korea)
| Samsung: Pyeongtaek, Hwaseong (Korea); Xi'an (China)
| Micron: Hiroshima (Japan); Boise (US); Singapore; Taichung (Taiwan)
|
| Process nodes: 1α (1-alpha), 1β (1-beta) DRAM
| Equipment: ASML DUV/EUV, AMAT, Lam, TEL, KLA
|
v
WAFER THINNING & TSV FORMATION
| Each DRAM die thinned to ~30-40μm
| Through-silicon vias drilled for vertical interconnect
| Equipment: Disco (thinning), Lam (etch for TSV)
|
v
HBM STACKING
| 8-high (HBM3E): 8 DRAM dies + 1 logic base die
| 12-high (HBM4): 12 DRAM dies + 1 base die (foundry-logic process)
| 16-high (HBM4E): planned for late 2026-2027
|
| Bonding: TCB (current) → hybrid bonding (HBM4)
| Equipment: Besi, ASMPT (bonding); Advantest (test)
| SK Hynix: MR-MUF (Mass Reflow Molded Underfill) process
|
v
TESTED HBM STACK
| Known-Good-Stack (KGS) testing before integration
|
v
TO ADVANCED PACKAGING (Chapter 9)
| HBM stacks placed adjacent to GPU die on CoWoS interposer
| NVIDIA Blackwell: 8x HBM3E stacks per GPU
| NVIDIA Rubin: HBM4 (288-384GB VRAM)
| AMD MI350: up to 288GB HBM3E
| Google TPU v7p: 8x HBM3E stacks
|
v
FINISHED AI ACCELERATOR → SERVERS (Chapter 18)8.4 Key Companies
8.4.1 HBM / DRAM
| Company | Ticker | Exchange | Approx. Mkt Cap | Role | Key Metric |
|---|---|---|---|---|---|
| SK Hynix | 000660 | KRX | ~$817B | #1 HBM supplier (~50-62% share); #1 DRAM manufacturer by revenue | Q2 2025 HBM share 62%; overtook Samsung as #1 DRAM maker (Q1 2025, 36% vs 34%); first to mass-produce HBM3E; first HBM4 development completed; NVIDIA’s primary HBM supplier (~90% of SK Hynix HBM goes to NVIDIA) 2346 |
| Samsung Electronics | 005930 | KRX | ~$1.2T | #2 HBM (~17-35% share, recovering); #2 DRAM; also foundry, NAND, displays | HBM share fell from 41% (Q2 2024) to 17% (Q2 2025) due to NVIDIA qualification delays; recovering with HBM3E qualification; HBM4 development completed; Pyeongtaek P5 fab resuming; 6th-gen 10nm DRAM for HBM4 147 |
| Micron Technology | MU | NASDAQ | ~$842B | #3 HBM (~11-21% share); #3 DRAM; NAND flash | HBM share rose to 21% (Q2 2025), overtaking Samsung; began shipping HBM4 samples at 11 Gbps; exited consumer memory market (Dec 2025) to focus on AI; FY Q1 2026 revenue $13.64B (+57% YoY); gross margins above 50% 2348 |
SK Hynix is the most important memory company for the AI buildout. It was first to mass-produce HBM3E, first to complete HBM4 development, and holds the primary supply relationship with NVIDIA (approximately 90% of SK Hynix’s HBM output goes to NVIDIA) 26. UBS predicts SK Hynix will achieve approximately 70% market share in HBM4 for NVIDIA’s Rubin platform in 2026 6. Goldman Sachs projects SK Hynix will maintain over 50% total HBM share through at least 2026 6. The company reported fiscal Q1 2026 revenue of $13.64 billion (+57% YoY) with gross margins above 50%, doubling from ~22% in the prior year, reflecting the structural shift toward high-margin HBM 2.
SK Hynix is 20.07% owned by SK Telecom/SK Inc. 9, part of the SK Group conglomerate. The company’s Icheon and Cheongju fabs in South Korea are the primary HBM production facilities.
Samsung’s HBM stumble is one of the most significant competitive shifts in memory history. Its share collapsed from 41% in Q2 2024 to 17% in Q2 2025 because Samsung’s HBM3E failed NVIDIA’s qualification tests, leaving it dependent on older-generation HBM3 4. Samsung co-CEO Jun Young-hyun acknowledged the challenge but stated customers have said “Samsung is back” on HBM4 7. Samsung’s 3.3D packaging and its 6th-generation 10nm DRAM are positioned as stepping stones to regain share. Counterpoint Research forecasts Samsung will lift its HBM share above 30% in 2026 as HBM3E qualifications complete and HBM4 enters full-scale supply 4.
Micron has emerged as the surprise gainer, rising from minimal HBM share to 21% in Q2 2025 by successfully qualifying HBM3E with major customers 4. Micron’s decision to exit the consumer memory market (shutting down the “Crucial” consumer brand in December 2025) signals a strategic pivot: the company is betting its future entirely on AI data center memory 2. Micron has begun shipping HBM4 samples at up to 11 Gbps and is working with foundry partners on HBM4E 4.
8.4.2 HBM Technical Specifications
| Generation | Interface Width | Bandwidth/Stack | Capacity/Stack | Stacking | Status |
|---|---|---|---|---|---|
| HBM3 | 1024-bit | 819 GB/s | 24GB (8-high) | TCB + MR-MUF | Production (mature) |
| HBM3E | 1024-bit | 1.18 TB/s | 36GB (8-high) | TCB + MR-MUF | Production (mainstream 2025-2026) |
| HBM4 | 2048-bit | ~2.0 TB/s | 48GB (12-high) | Hybrid bonding | Production ramp 2026 |
| HBM4E | 2048-bit | ~2.8 TB/s | 64GB+ (16-high) | Hybrid bonding | Samples late 2026, production 2027 |
The transition from HBM3E to HBM4 is architecturally significant. HBM4 doubles the memory interface from 1024-bit to 2048-bit, enabling bandwidth of approximately 2 TB/s per stack 23. For the first time, HBM4 uses a foundry-level logic process for the base die (rather than a memory process), which means TSMC or Samsung Foundry must fabricate the base die while the memory dies come from SK Hynix/Samsung/Micron DRAM fabs 1. This adds supply chain complexity and introduces new dependencies.
8.4.3 NAND Flash & Storage
| Company | Ticker | Exchange | Approx. Mkt Cap | Role | Key Metric |
|---|---|---|---|---|---|
| Samsung | 005930 | KRX | ~$1.2T | #1 NAND producer; also DRAM and HBM | Dominates NAND; 200+ layer V-NAND; enterprise SSD for AI storage |
| SK Hynix (Solidigm) | 000660 | KRX | ~$817B | Acquired Intel NAND (Solidigm) in 2021 for $9B | Solidigm enterprise SSDs for AI data center storage |
| Micron | MU | NASDAQ | ~$842B | #3 NAND producer | 232-layer NAND; pivoting to enterprise/AI storage |
| Kioxia | 285A | TSE | ~$10.0B | #4 NAND (JV with Western Digital); IPO’d Dec 2024 | Japanese NAND producer; enterprise SSD growing. Ticker corrected from 6600 to 285A. |
| Western Digital | WDC | NASDAQ | ~$165B | NAND (JV with Kioxia) + HDD | Spun off flash business (completed 2024); HDD remains important for data center storage |
| Seagate Technology | STX | NASDAQ | ~$186B | #1 HDD manufacturer; exabyte-scale drives for AI training datasets | 30TB+ HAMR drives shipping; enterprise nearline dominates AI data lake storage |
Hard disk drives remain economically essential for AI at scale despite the SSD narrative. Frontier model training datasets are measured in petabytes; at current NAND pricing, storing multi-petabyte corpora entirely on flash is cost-prohibitive. Seagate and Western Digital supply the high-capacity nearline drives (30TB+, moving to 40TB+ with HAMR) that form the bulk storage tier in AI training pipelines. Seagate’s heat-assisted magnetic recording (HAMR) technology is the primary capacity scaling mechanism for next-generation HDDs, enabling areal density increases that sustain the $/TB advantage over NAND for cold and warm data tiers.
8.5 Bottleneck Analysis
HBM supply (EXTREME): HBM supply is severely constrained, ranking alongside CoWoS packaging as a gating factor for GPU deployment (see Chapter 20 for the definitive severity ranking across all layers). Three companies produce all HBM globally. Supply is fully allocated through 2026, with SK Hynix and Samsung planning 20% price increases 5. SK Hynix management stated on the Q1 2026 call that “demand required of us over the next three years exceeds our capacity” and the company is “supplying HBM within the limits of our capacity” 12. In the same call, management noted that “customers are prioritizing securing volume over pricing, which is sustaining the current pricing strength” 12. NVIDIA’s GPU output is limited in part by HBM availability: each Blackwell GPU requires 8 HBM3E stacks, and each Rubin GPU will require even more HBM4. At roughly $300 per HBM3E stack (rising to an estimated $500 for HBM4) 3, HBM represents approximately $2,400-4,000 per GPU in memory cost alone, a significant fraction of total module bill-of-materials, module cost derived from rack pricing]. This is not a low-cost gating item like ABF film (Chapter 9); HBM is both high-cost and supply-constrained. The transition to HBM4 (with its new foundry-process base die) adds a new dependency on TSMC/Samsung Foundry that did not exist for HBM3E.
HBM qualification cycle (SEVERE): Getting a new HBM product qualified at a customer like NVIDIA takes 6-12 months. Samsung’s 2024-2025 HBM3E qualification failure with NVIDIA demonstrates that this process is non-trivial. A company cannot simply increase HBM production; the product must pass rigorous validation for electrical performance, thermal behavior, and reliability under the specific workload conditions of each GPU platform.
Wafer capacity reallocation (HIGH): HBM requires 2x the wafer area of standard DRAM per bit, and yields run 50-60%. As memory makers shift wafer capacity toward HBM, standard DRAM and consumer memory face shortages. Micron’s exit from consumer memory, NVIDIA’s 30-40% cut in gaming GPU production due to GDDR7 constraints, and rising DDR5 prices are all downstream consequences of HBM prioritization 2.
HBM test capacity (HIGH, hidden demand multiplier): Every HBM stack requires Known Good Die (KGD) testing before bonding and system-level test (SLT) after assembly. Test time per HBM stack runs 2-3x longer than conventional DRAM due to the need to test each die in an 8-high or 12-high stack individually, then validate the assembled stack’s thermal behavior and electrical performance under load. Advantest holds approximately 58-65% of the global semiconductor test market (up from ~58% in 2024, with share gains driven by AI-related test demand) and dominates HBM testing with its T5503HS2 platform (16,256 channels for maximum parallelism). Teradyne (TER, NASDAQ, ~$56.3B) is the duopoly partner, with memory test revenue reaching $128M in Q3 2025 (75% DRAM, 25% flash), up 30% YoY 10. Together they control approximately 95% of the ATE market. As HBM volume scales, test equipment capacity becomes its own constraint. The current draw required for HBM testing limits how many devices can be tested in parallel, meaning test throughput scales slower than production volume.
The test bottleneck has a hidden consumable layer: probe cards. FormFactor (FORM, NASDAQ, ~$13.6B) is the #1 probe card supplier with 32-34% global market share, providing specialized HFTAP Matrix Probe Cards for KGD testing of HBM wafers. Every HBM wafer requires KGD probing before the bonding step; probe cards are the limiting consumable. Kulicke & Soffa (KLIC, NASDAQ, ~$5.4B) provides the die bonding and wire bonding equipment for HBM assembly, though hybrid bonding from Besi and ASMPT is capturing HBM4 business.
This is a bottleneck that almost no sell-side analysis covers because it sits inside the memory maker’s fab, invisible to external observers. FY2025 Advantest revised guidance: approximately ¥1,070B revenue with ¥454B operating income (revised upward from initial ¥950B / ¥374B guidance, reflecting stronger-than-expected AI test demand).
DRAM supply gap (HIGH): Micron CEO Sanjay Mehrotra quantified the shortfall on the Q1 2026 call: “In the medium term, we are only able to meet about 50% to two-thirds of our demand from several key customers,” adding that “the aggregate industry supply will remain substantially short of the demand for the foreseeable future.” Micron’s entire 2026 HBM supply is fully allocated, with pricing locked in under multiyear contracts 13.
Three-company oligopoly (STRUCTURAL): The memory industry consolidated to three companies through decades of brutal cyclical downturns that eliminated weaker players. Building a new DRAM fab costs $15-20 billion and takes 3-4 years. No new entrant has succeeded in DRAM in over 20 years. China is attempting (ChangXin Memory Technologies, or CXMT, is the leading Chinese DRAM effort), but remains far behind in technology and volume. This oligopoly gives the three incumbents structural pricing power, particularly in HBM where demand far exceeds supply.
8.6 Risks
HBM oversupply risk (medium-term): All three memory makers are investing aggressively in HBM capacity. If AI demand growth slows or efficiency improvements reduce HBM-per-GPU requirements, the industry could face overcapacity by 2028-2029. The memory industry has a long history of boom-bust cycles, and HBM is not immune.
Samsung’s recovery compressing SK Hynix margins: If Samsung successfully ramps HBM3E and HBM4, the competitive dynamic shifts from “SK Hynix monopoly” to “oligopoly with price competition.” This would benefit GPU buyers (NVIDIA, AMD) but compress memory maker margins.
Alternative memory architectures: Processing-in-memory (PIM), compute-near-memory, and novel memory technologies could reduce the need for HBM bandwidth by moving computation closer to data. Samsung’s HBM-PIM and various startup approaches are exploring this. These are 3-5 year risks, not near-term.
Chinese DRAM competition at mature nodes: CXMT is expanding capacity at DDR4 and older-generation DRAM, which could compress pricing for non-HBM memory. This does not threaten the HBM oligopoly (Chinese firms are years away from HBM capability) but could affect the broader DRAM market.
HBM4 base die complexity: The requirement for a foundry-logic base die in HBM4 introduces new supply chain dependencies and potential yield issues. If base die yields from TSMC or Samsung Foundry are low, HBM4 production ramps could disappoint.
First principles check: Does the HBM bottleneck make sense? Yes. Stacking 8-16 ultra-thin DRAM dies with TSVs, bonding them with sub-10μm precision, and achieving yields of 50-60% 1 on these complex 3D structures requires process expertise that has taken SK Hynix a decade to develop. The qualification process at NVIDIA adds another barrier. A new entrant would need to master DRAM fabrication, advanced stacking, and pass NVIDIA qualification, a multi-year journey even with unlimited capital.