All articles.
The arithmetic of attention: why FlashAttention still matters
Inference & ServingMemory bandwidth, not FLOPs, is what bounds modern inference. A walk through the numbers behind a kernel that quietly reshaped the field.
Continuous batching, revisited
Inference & ServingThree years after the original paper, what does state-of-the-art serving actually look like? A field report from a team running 12B tokens a day.
What we've been getting wrong about MoE routing
ArchitectureTop-k routing has become a default. It shouldn't be. A look at the tradeoffs nobody's measuring and the experiments that change my mind.
Quantization-aware training, end-to-end
QuantizationFP4 is here, and the gap between PTQ and QAT has widened. What's actually working in production today, and why the recipe is messier than it looks.
A research-grade trainer in 400 lines
Training SystemsMost training frameworks are 50,000 lines of code in a trench coat. This is what falls out when you start from FSDP and a will to delete.
Embeddings as compression: the bitter lesson, retold
Retrieval & RAGWe've stopped treating embeddings like first-class data. A case for revisiting them, with measurements from a 200M-document corpus.
World models and the cost of imagination
ModelsLatent rollouts are cheap. World rollouts are not. What we learned trying to scale a JEPA-style world model on robotics data.
The hidden latency in agent loops
AgentsEvery tool call is a round trip. Every round trip is a context append. Why naive agent loops compound latency faster than you expect, and what to do about it.
RAG is not retrieval-augmented generation
Retrieval & RAGIt is a feedback loop, an index policy, and a re-ranker pretending to be a system. Why most RAG postmortems mistake the symptom for the disease.
Speculative decoding without the speculation
Inference & ServingDraft models work. They also fail in ways the original papers didn't surface. A small bag of tricks for keeping acceptance rates high in real workloads.
FSDP vs DeepSpeed, 2026 edition
Distributed TrainingThe choice used to be obvious. It isn't anymore. A side-by-side on training a 30B model across three clusters and four hardware generations.
The four evals that matter (and the dozen that don't)
EvaluationWe have too many benchmarks and too few signals. A framework for choosing evaluations that correlate with the thing you actually care about.
Testing Final Blog
ArchitectureBlog testing Write portal!
Test First Blog Post
ArchitectureTesting blog post from website!
Neural Networks From Zero: From a Single Number to a Billion Parameters
ArchitectureA neural network never sees a word, an image, or a sound — only a list of numbers. Starting from that one fact and a single neuron, this day-zero guide builds the whole machine: how any input becomes numbers, why weights, biases, and activations each exist, how neurons stack into layers and layers into a model, and how to compute a model's size and running cost by hand.
Notes on KV cache paging at scale
Inference & ServingPagedAttention is a good idea poorly understood. A primer, plus the second-order effects you only see at 10,000 concurrent requests.