Writing

Technical articles about distributed systems, AI infrastructure, routing, and open-source engineering.

Articles

Featured article

Subscribe via RSS
Open Source Systems11 min read

kvfleet Internals: Building a KV-Cache-Aware Routing Control Plane for LLM Fleets

A deep technical walkthrough of kvfleet's routing engine, from prompt fingerprinting and KV-cache affinity to policy-aware model selection, fallback chains, and OpenAI-compatible gateway mode.

#kvfleet#llm-routing#kv-cache#python#distributed-systems
Read article