Inference · 8 minThe benchmark said one second. Production said ninety.
What theoretical load tests reveal—and why KV-cache locality decides whether an agent feels instant or broken.
Read field note ↗Platform · UpcomingWhat a private AI gateway actually needs
Keys and routing are the easy part. Ownership, fallbacks and observability decide whether teams trust the platform.
In the notebookReliability · UpcomingLong-running AI jobs without the chaos
Why leases, idempotency and atomic artifacts matter when inference lasts minutes.
In the notebook