Articles by Priya Raghavan

Three years after the original paper, what does state-of-the-art serving actually look like? A field report from a team running 12B tokens a day.

PagedAttention is a good idea poorly understood. A primer, plus the second-order effects you only see at 10,000 concurrent requests.