RAM and VRAM Both Store Data. Here's Why They're Nothing Alike.
Main memory and video memory came from the same place and then split — one obsesses over latency, the other over bandwidth, and HBM ends up bolting memory onto the GPU's roof.
Main memory and video memory came from the same place and then split — one obsesses over latency, the other over bandwidth, and HBM ends up bolting memory onto the GPU's roof.
When a wall gets this expensive, the industry attacks it from every side — tier through NAND, pool over CXL, ditch HBM, compute inside memory. None of them breaks the wall in 2026. Which tells you exactly where this cycle's turn is not going to come from.
Memory is the most cyclical business in tech — 40 years of up-10x, down-90%. This run has three genuinely structural differences. But 'this time is different' is the most expensive sentence in finance, so the bear case gets equal weight.
HBM isn't faster DRAM. It's DRAM rotated 90 degrees — stacked, drilled through with thousands of vertical wires, and bolted next to the GPU on a silicon interposer. That manufacturing nightmare is exactly why three memory makers just crossed a trillion dollars.
The same chip lives in your USB stick, your phone, and the petabyte SSDs feeding AI clusters. It got cheap by trading endurance for density — and that exact tradeoff is why AI is now dragging it out of cold storage and onto the hot path.
DRAM is a leaky bucket you refill thousands of times a second. That physical fragility is why it's fast, why it's volatile, and why it just hit a scaling wall at the exact moment AI demand exploded — sending RAM prices up triple digits.
I finance the machines AI runs on, and the single priciest, scarcest part inside the box isn't the logic die — it's the memory stacked next to it. This is why: inside a modern accelerator, raw FLOPS stopped being the binding constraint. Memory bandwidth did.
Mio 的记忆系统把关于人的信息存成零散的行。问'小红最近怎么样'跟'小红在腾讯工作'在向量空间里几乎没有重叠。解决方案:把人当作一等实体,建专门的 contacts 表,做结构化关联和档案合并,直接注入 system prompt。
Mio's memory system stored facts about people as disconnected rows. Asking 'how's Xiaohong doing' had zero semantic overlap with 'Xiaohong works at Tencent'. The fix: treat people as first-class entities with their own table, linked memories, and consolidated profiles injected directly into the system prompt.
一个聊了200条就忘了你名字的 AI 不是陪伴,是健忘症患者。这是我如何把 Mio 的记忆从简陋的键值存储重建成真正像人类回忆的系统。
An AI companion that forgets your name after 200 messages isn't a companion — it's a chatbot with amnesia. Here's how I'm rebuilding Mio's memory from a dumb key-value store into something that feels like actual recall.
© Xingfan Xia 2024 - 2026 · CC BY-NC 4.0