High-bandwidth memory (HBM)
Also known as: HBM, HBM3, HBM3E, HBM4, stacked memory
High-bandwidth memory (HBM) is a type of memory made by stacking several DRAM chips on top of each other and placing them right next to a processor, connected by thousands of tiny wires.
While AI discussions focus on GPUs, the memory beside them is often the real bottleneck. HBM's supply has become one of the most constrained and profitable corners of the chip industry.
High-bandwidth memory (HBM), explained at three levels
At home
Think of a chef whose ingredients sit in a pantry down the hall, reached through one narrow door. However fast the chef works, meals wait on that door. HBM moves the pantry right next to the stove and adds a thousand doors. In practice, it stacks memory chips into a tower placed beside the processor, joined by more than a thousand tiny connections, so data flows in fast enough to keep the processor busy.
At work
High-bandwidth memory is a type of DRAM built as a stack of memory chips that sits inside the same package as a processor. It's used mainly in AI accelerators, such as Nvidia's data centre GPUs, and in supercomputers. Businesses care because the amount of HBM sets how large an AI model one accelerator can hold, and its speed sets how fast the model answers.
At home
Regular PC memory sits on separate sticks a few centimetres from the processor, connected by a 64-bit-wide channel, and graphics cards use GDDR chips with 32-bit links. HBM takes the opposite approach: each stack connects through a 1,024-bit interface in HBM3, and 2,048 bits in HBM4. Each wire runs slower than GDDR's, but there are so many of them that a single stack can move more than a terabyte per second. Several stacks around one processor multiply that again.
At work
An HBM stack has two parts: a base, or logic, die at the bottom that talks to the processor, and four to sixteen DRAM dies stacked above it. Stack height sets capacity, so a 12-high HBM3E stack holds about 36 gigabytes. For AI, capacity decides how large a model fits on one accelerator, and bandwidth decides how fast it generates text, since every generated word requires reading the model's weights from memory. Compare accelerators on both gigabytes and terabytes per second.
At home
The dies in a stack are joined by through-silicon vias, vertical copper wires etched straight through each thinned die, plus tiny solder bumps between layers. The finished stack sits on a silicon interposer next to the processor, because a normal circuit board can't carry thousands of wires that close together. Every die and bond must work, so one flaw can scrap a whole stack. In my view, that tight integration is brilliant engineering, but it's also why HBM costs several times more per gigabyte than ordinary DRAM.
At work
Each HBM generation raises per-pin speed, stack height, or interface width. HBM4 doubles the interface to 2,048 bits and lets the base die be made on a foundry logic process, so customers can add custom logic close to memory. Supply depends on three things: memory makers' HBM wafer capacity, stacking yield, and foundries' advanced packaging capacity, so shortages can start at any of them. SK hynix, Samsung, and Micron are the only major suppliers. Power and heat limit how tall stacks can grow.
Real-world examples
At home
Jerome was shopping for a graphics card for running local AI models. He noticed that professional AI accelerators listed HBM with huge bandwidth figures, while gaming cards used GDDR memory. He realized memory bandwidth, not just processing power, set the speed limit for the models he wanted to run.
At work
Aisha manages infrastructure planning at a research lab buying AI servers. Lead times were driven by HBM supply rather than GPU availability, so she placed orders months ahead and chose a configuration with more memory per accelerator to fit the lab's largest models.
How it compares
| feature | High-bandwidth memory (HBM) | GDDR memory |
|---|---|---|
| Structure | Stacked dies next to the processor | Separate chips on the board |
| Interface width | Very wide (1024 bits or more per stack) | Narrow (32 bits per chip) |
| Bandwidth | Highest | High |
| Cost | Much higher | Lower |
| Typical use | AI accelerators, supercomputers | Gaming graphics cards |
HBM delivers far more bandwidth than GDDR at a much higher cost and complexity.
For analysts
In AI hardware decisions, memory is often the constraint that matters most. I'd size workloads by memory capacity and bandwidth first, then choose compute, because an underfed accelerator is expensive silicon doing nothing.
History & origin
SK hynix and AMD developed HBM, and the JEDEC standards body adopted it in 2013. AMD's Radeon R9 Fury graphics cards in 2015 were the first products to use it. HBM2 arrived in 2016, and HBM3 followed in 2022, appearing in Nvidia's H100 AI accelerator. HBM3E arrived in 2024, and JEDEC published the HBM4 standard in 2025, doubling the interface width. SK hynix established a lead in HBM for AI, with Samsung and Micron competing hard. The AI boom made HBM scarce and expensive, and it diverted memory production away from conventional DRAM, contributing to wider memory price increases. In my opinion, HBM shows that the AI race isn't just about compute. It's about moving data fast enough to keep that compute busy.