output layer · research

Research

We're an applied lab. Our findings come out of real deployments — evaluation harnesses, fine-tunes, and routing systems for domains where "approximately right" isn't good enough.

ls published/
(empty) — first artifacts are still compiling
cat in-progress.md
→ an evaluation benchmark for tax-law LLMs (Indian GST)
→ a parameter-efficient fine-tuning playbook for regulated domains
→ routing across model tiers in production
→ distillation recipes for sovereign deployment budgets

The stack we care about

  1. layer 01

    evaluation

    Before anything is tuned, measure it. Eval suites that surface real failure modes in specialized domains — the kind users hit, not the kind benchmarks flatter.

  2. layer 02

    fine-tuning & distillation

    LoRA, adapters, and teacher→student compression — the math of parameter efficiency, applied where a specialist genuinely beats a generalist.

  3. layer 03

    retrieval & routing

    Grounding specialized models in the right context, and routing each query to the smallest model that can answer it well.

  4. layer 04

    sovereign deployment

    Your data, your weights, your infrastructure. Everything above, running inside your boundary — because for tax, law, and finance that's a requirement.

What we're reading

The knowledge base holds our illustrated read-throughs of the papers this stack rests on — a full arc on routing each query to the cheapest model that can actually answer it (layer 03 above): RouteLLM and Hybrid LLM (upfront routers), FrugalGPT and AutoMix (cascades), and RouterBench (how to measure them). And a full arc on post-training — how a base model becomes an assistant: LIMA (the data bar), InstructGPT (the SFT→RM→RL template), DPO (preference tuning without the RL rig), QLoRA (the one-GPU hardware recipe), Tulu 3 (the full open pipeline), capped by GLM-5 & Kimi K3 — the recipe at the mid-2026 open frontier. And the frontiers arc — the bets that question the recipe itself: DeepSeek-R1 (reasoning learned from verifiable reward), Mamba (sequence modeling without attention), and HOPE (Google's nested-learning architecture that keeps learning after training ends).

Until the first paper lands: read the blog, run the forward pass, or trace the intellectual traditions behind the math. Working on something in this space — or want to work on it with us? hello@attention.sh