// HACKER NEWS — CYBERSECURITY
Processing in Memory: DRAM Is About to Do Math
Samsung's LPDDR5X-PIM puts compute inside commodity memory, exposing 614 GB/s of bank-level bandwidth for local LLM inference. The silicon works. Quantization, memory layout, and runtime support now stand in the way.
At Hot Chips 2026, Samsung presented a 16 GB LPDDR5X memory package that delivers 614 GB/s of internal bandwidth to its own compute units.
For scale, 614 GB/s matches the memory bandwidth of a top-spec Apple M5 Max across its entire unified memory system: the 40-core GPU configuration in the $3,499 MacBook Pro. Samsung claims that number from inside one memory package, against the 76.8 GB/s that escapes through its external pins. Tom's Hardware reported the resulting eightfold difference.
That eight-to-one ratio makes the case for Processing in Memory. DRAM banks already provide most of the bandwidth; the external pins cannot expose it.
Bank. DRAM contains banks, independent arrays that can operate in parallel. A modern LPDDR5X die has dozens. Together they deliver enormous internal throughput, but they share a narrow external interface.
GEMV vs. GEMM. Matrix-vector multiply versus matrix-matrix multiply. Autoregressive decoding at batch size 1 uses GEMV: one token's activations multiplied against the entire weight matrix. Prefill and batched serving use GEMM. PIM helps most with GEMV.
PIM. Processing-in-memory hardware places compute units next to the banks, inside the memory die, where they can use bank-level bandwidth instead of interface bandwidth.
Arithmetic intensity. FLOPs performed per byte loaded. High intensity makes a workload compute-bound. Low intensity makes it bandwidth-bound, leaving matrix engines idle.
Generating one token requires reading every parameter from DRAM, multiplying it once, and discarding it. Each token depends on the preceding token, so batch-1 decoding cannot reuse the weights across tokens. Arithmetic intensity sits near the floor.
At 614 GB/s against a 20 GB model, the hard ceiling sits around 30 tokens per second. The real number is lower because the KV cache grows with context and competes for the same bandwidth. During generation, most of the matrix hardware you paid for waits on DRAM.