knowledge base
Knowledge Base
Papers we've read carefully enough to redraw. Each entry starts with no assumptions at all and adds the technical layer only once the idea has landed — the version we wish had existed when we first opened the PDF.
Entries
-
entry 01 · arXiv:2406.18665
The model that decides which model answers
RouteLLM — how a small router sends each question to the cheapest model that can actually handle it, and how PGR, CPT and APGR tell you whether it worked. Nine illustrated figures, with toy examples for the metrics and the matrix factorization.
-
entry 02 · arXiv:2404.14618
The router that knows when cheap is good enough
Hybrid LLM — a small BERT-style router predicts whether the cheap model's answer would be good enough, learned from a measured quality gap and routed by one tunable threshold. Seven illustrated figures, with a worked soft-label example and the cost/quality curve.
-
entry 03 · arXiv:2305.05176
Run the cheap model first — then check its work
FrugalGPT — the canonical LLM cascade: run models cheapest-first and escalate only the answers a small checker judges unreliable, matching GPT-4 for up to 98% less. Seven figures, routing-vs-cascading laid out, and the cost funnel worked through by hand.
-
entry 04 · arXiv:2310.12963
A model that grades itself — and a judge that knows better
AutoMix — the small model self-verifies its own answer, and a noise-aware POMDP meta-verifier decides when to trust that shaky signal, learning to tell a hard question from a hopeless one. Seven figures, with the three hidden states and the belief-update worked out.
-
entry 05 · arXiv:2403.12031
One ruler for every router — and a humbling result
RouterBench — the shared testbed (405k precomputed outcomes) and the AIQ metric that put every router on one cost/quality plane, bookended by an oracle and a zero router. Seven figures, and the finding that today's routers barely beat the trivial baseline.
In the queue
That completes the routing arc — upfront routers (RouteLLM, Hybrid LLM), cascades (FrugalGPT,
AutoMix), and how to measure them all (RouterBench). Beyond routing, the papers under
papers/ in the repo are the wider reading list — distillation, lottery tickets, LoRA,
Chinchilla, Mamba, DeepSeek-R1. Each gets an entry once we've had to use it for something real.
Prefer a guided tour of the architecture itself? The Illustrated Decoder covers a GPT-style model one figure at a time, or run the forward pass in 3D.
Working on routing, evaluation, or sovereign deployment — or want to? hello@attention.sh