Blog | Systems, AI Infrastructure, and Open Source

Writing & Research

Technical Blog

Long-form engineering notes on distributed systems, AI infrastructure, routing control planes, and the design trade-offs behind open-source libraries.

Open Source SystemsMar 21, 2026

kvfleet Internals: Building a KV-Cache-Aware Routing Control Plane for LLM Fleets

A deep technical walkthrough of kvfleet's routing engine, from prompt fingerprinting and KV-cache affinity to policy-aware model selection, fallback chains, and OpenAI-compatible gateway mode.

#kvfleet#llm-routing#kv-cache#python#distributed-systems#ai-infrastructure