#inference
3 articles
- 2026-10-17
Tokens Per Second Is Two Benchmarks Wearing a Trench Coat
A single tokens-per-second score hides prompt ingestion, decoding, and the workload that actually matters. Benchmark both halves or stop comparing machines.
- 2026-10-14
Diffusion Language Models Have a Cache Problem. That Is Finally Getting Interesting
Diffusion language models can generate text in parallel, but repeated denoising makes them slow. New cache-aware work makes the trade-off worth watching.
- 2026-10-12
Diffusion Language Models Have a Cache Problem. That Is Finally Getting Interesting
Diffusion language models can generate text in parallel, but repeated denoising makes them slow. New cache-aware work makes the trade-off worth watching.