Engineering work
AI infrastructureOpen-source package

kvfleet

A KV-cache-aware routing control plane for self-hosted and hybrid LLM fleets, built around explainable policy, locality, health, and fallback decisions.

PythonLLM routingKV cachePolicyOpenAI-compatible API

01

Problem

LLM routing is often reduced to choosing a model, while production fleets also need to preserve cache locality, respect tenant and compliance constraints, react to endpoint health, and recover without hiding why a route changed.

Constraints that shaped the design

  • Remain useful as both an embedded Python library and an OpenAI-compatible gateway.
  • Keep route decisions inspectable instead of hiding them behind a single score.
  • Balance CPU-side decision cost against much larger GPU-side recomputation cost.

02

Architecture

  1. 01

    Normalize and fingerprint

    Build stable request, conversation, and prefix signals from incoming work.

  2. 02

    Filter and rank

    Apply policy and health gates before ranking eligible endpoints across multiple objectives.

  3. 03

    Execute and explain

    Route with fallback behavior while exposing the factors behind the decision.

03

Design decisions

  • Keep conversation affinity and prefix affinity as separate signals.
  • Apply hard policy constraints before soft ranking so an attractive score cannot bypass eligibility.
  • Offer gateway mode to reduce migration cost for OpenAI-compatible clients.

04

Alternatives considered

  • A model-only router that ignores replicas, endpoint locality, and fleet health.
  • A single opaque routing score with no inspectable decision factors.
  • Assuming process-local route memory is sufficient for every horizontally scaled deployment.

05

Testing and results

  • The public package and repository expose the implementation and release history.
  • The technical article walks through the routing pipeline, gateway mode, design trade-offs, and current limits.
  • The article explicitly identifies process-local affinity and prefix memory as a scaling boundary.

Outcome

kvfleet packages cache-aware, policy-aware routing into a reusable control plane and pairs the code with a long-form architectural explanation.

06

Limitations

  • Route memory is currently process-local unless the deployer supplies a shared design.
  • Horizontal deployments still need deliberate ingress locality or shared affinity state.

07

Lessons learned

  • The best fleet route is constrained by locality, policy, health, economics, and recovery at the same time.
  • A router becomes easier to operate when every decision has a legible explanation.