Learn · Track

Fine-tune Qwen for GST

Take a small open Qwen model and make it measurably better at reasoning about Indian GST than it is out of the box — on a free Colab GPU, with the data, the model and the scores published on Hugging Face.

You will use: Unsloth, Google Colab, Hugging Face

0 of 9 steps done
  1. Step 1 of 9

    Measure before you train

    1 Watch & read

    • Who Judges the Judge? 2026 — An LLM judge's verdict shifts with which model wrote the answer — validate it against an expert first. Hindi video coming soon Read the paper ↗

    2 Concepts explain each one in your own words before moving on

    • A fine-tune without a baseline is a story, not a result.
    • Held-out eval set: questions the model will never be trained on.
    • What fine-tuning changes (how the model reasons and formats) vs. what retrieval changes (which facts are in front of it).
    • An LLM judge is a measuring instrument: check it against an expert's verdicts before you trust it.
    • Tag every question by failure type, so a score tells you what improved.

    3 Quick quiz

    1. You fine-tune and the answers "look better". What is missing?

    2. A GST rate changed last month. The most direct fix is…

    3. Before using an LLM judge to score 100 answers, you should…

    4 Action

    Build the eval set: 50–100 GST questions, each with a reference answer, the exact authority it rests on, an as-of date, and a tag for the reasoning it tests — deeming fiction, authority hierarchy, specific-over-general, citation discipline, connected provisions. Start from the expert's corrections. Run an untouched 4-bit Qwen on all of them, score with a judge you have checked against the expert, and push questions and scores to a Hugging Face dataset repo.

  2. Step 2 of 9

    LoRA — what you are actually training

    1 Watch & read

    2 Concepts explain each one in your own words before moving on

    • The base weights W stay frozen; you learn a small update ΔW = B·A.
    • Rank r is the width of that update — small r, few parameters.
    • B starts at zero, so training begins from the unmodified model.
    • After training, B·A can be merged into W: no extra inference cost.

    3 Quick quiz

    1. During LoRA training, which weights receive gradient updates?

    2. Why is B initialised to zero?

    3. Does a merged LoRA adapter slow down inference?

    4 Action

    Open an Unsloth Qwen fine-tuning notebook in Colab and run it unmodified, end to end. Change nothing. Write down the percentage of trainable parameters it prints, then change r and watch it move.

  3. Step 3 of 9

    QLoRA — fitting it on a free GPU

    1 Watch & read

    • QLoRA — Freeze the base in 4-bit, train adapters on top — a fine-tune on the GPU you already have. Hindi video coming soon Read the paper ↗ How to read it

    2 Concepts explain each one in your own words before moving on

    • The frozen base is stored in 4-bit; the LoRA adapters stay in higher precision.
    • NF4: a 4-bit data type shaped for normally-distributed weights.
    • Gradients flow through the quantized base into the adapters — the base itself never changes.

    3 Quick quiz

    1. In QLoRA, what is quantized to 4-bit?

    2. What does that buy you?

    4 Action

    Load the same Qwen model twice in Colab — once in 4-bit, once not — and record peak GPU memory for each. Pick the largest Qwen that fits a T4 in 4-bit with room to train.

  4. Step 4 of 9

    Format — the chat template

    1 Watch & read

    2 Concepts explain each one in your own words before moving on

    • A chat model was trained on text with special tokens marking system, user and assistant turns.
    • The chat template turns your list of messages into exactly that text.
    • Use a different format in training than at inference and quality drops with no error message.
    • Training on responses only: the loss is counted on the assistant's answer, not on the prompt.

    3 Quick quiz

    1. Your fine-tuned model answers well in the notebook but badly in your app. The first thing to check is…

    2. What does a chat template produce?

    4 Action

    Take five of your GST examples as system / user / assistant messages. Render them with the tokenizer's chat template and print the result. Read the raw text until you can point at every special token and say what it marks.

  5. Step 5 of 9

    Synthetic data 1 — generate

    1 Watch & read

    • Self-Instruct — Bootstrap thousands of training examples from a handful of seeds — and inherit the generator's mistakes if you do not filter. Hindi video coming soon Read the paper ↗
    • LIMA — A thousand carefully chosen examples can be enough. Hindi video coming soon Read the paper ↗ How to read it
    • Textbooks Are All You Need — Better data beats a bigger model. Hindi video coming soon Read the paper ↗ How to read it
    • Quality, Diversity, Complexity — A survey of what actually makes synthetic data useful — three properties that pull against each other. Hindi video coming soon Read the paper ↗

    2 Concepts explain each one in your own words before moving on

    • Seeds: every generated example starts from real material — a statute section, an expert correction — never from nothing.
    • The loop is generate → filter → verify. Self-Instruct showed the loop, and that unfiltered output repeats the generator's errors.
    • Three properties to balance: quality (is it right), diversity (does it cover the domain), complexity (is it hard).
    • LIMA: a small, consistent, curated set beats a large noisy one.
    • Generate training data with an open model you are allowed to train on.

    3 Quick quiz

    1. Why seed generation from statute text and expert corrections, instead of asking a model to "write GST questions"?

    2. You generate 5,000 examples and most are rewordings of the same forty questions. Which property is missing?

    3. What did Self-Instruct's own analysis find about its generated data?

    4 Action

    Build the smallest working pipeline. Input: 30 statute sections and your expert corrections. For each seed, generate a question, the reasoning, the exact citation and the answer in one fixed format. Deduplicate. Read 30 at random yourself and record how many are actually correct.

  6. Step 6 of 9

    Synthetic data 2 — make it hard, keep it right

    1 Watch & read

    • Evol-Instruct (WizardLM) — Make a question harder on purpose — add a constraint, a reasoning step, a rarer case. Hindi video coming soon Read the paper ↗
    • RAFT — Train the model with the right document mixed in among distractors, so it learns to pick. Hindi video coming soon Read the paper ↗
    • Agentic Adversarial QA 2026 — Generate the questions where a weak model and a strong model disagree — a small set aimed at interpretive reasoning. Hindi video coming soon Read the paper ↗
    • QueST — Train the question generator itself to produce difficult problems. Hindi video coming soon Read the paper ↗

    2 Concepts explain each one in your own words before moving on

    • Evolve: rewrite a question to add a constraint, a reasoning step, or a rarer case.
    • Distractors: put near-miss provisions in the context so the model has to choose the governing one.
    • Disagreement mining: keep the questions where a weak and a strong model give different answers.
    • Model-in-the-loop filter: a question the base model already answers correctly teaches it little.
    • Verify every kept item — a hard question with a wrong gold answer is worse than no question.
    • For GST, 'hard' has five shapes: a deeming provision that overrides the facts; a circular that conflicts with the Act; a specific rule behind a general one; a question whose honest answer is 'no authority supports this'; a connected provision that changes the outcome.

    3 Quick quiz

    1. Which of these makes a GST question harder in a useful way?

    2. What does RAFT put in each training example alongside the question?

    3. Every model you test fails one of your new questions. Before calling it "hard", you should…

    4. Why keep only the questions the base model gets wrong?

    4 Action

    Take the 300 examples from the last step. Evolve each one along one of the five GST difficulty shapes. Add distractor provisions to the context. Run the base Qwen and a stronger model on every question and keep the ones where they disagree or the base model fails. Verify each kept item with your judge, and have the expert check a sample. Confirm zero overlap with the eval set.

  7. Step 7 of 9

    The run — supervised fine-tuning

    1 Watch & read

    2 Concepts explain each one in your own words before moving on

    • SFT is stage one of every recipe: show the model demonstrations of the behaviour you want.
    • Rank, alpha, learning rate, epochs: the four knobs worth understanding before touching the rest.
    • Enough training to learn the pattern, not so much that it memorises a few hundred rows.
    • Training loss going down is not the result. The held-out eval score is.

    3 Quick quiz

    1. In InstructGPT's three steps, what comes first?

    2. Training loss keeps falling but the eval score is getting worse. This is…

    4 Action

    Fine-tune Qwen on your verified set with the Unsloth notebook. Re-run the eval from step 1. Push the adapter to the Hub with a model card that states both scores — base and tuned — broken down by failure type.

  8. Step 8 of 9

    Preferences — learn from corrections

    1 Watch & read

    2 Concepts explain each one in your own words before moving on

    • DPO learns from pairs: a preferred answer and a rejected one, with no separate reward model.
    • An expert correction is already a pair — the model's wrong answer is the rejected one, the expert's is the preferred one.
    • Distillation: a student learns from a teacher's outputs, which carry more information than a bare right/wrong label.

    3 Quick quiz

    1. What does a DPO training example look like?

    2. Why do a teacher's soft targets teach more than hard labels?

    4 Action

    Turn the expert's corrections into preference pairs: the question, the answer the model gave, the answer the expert wrote. Run DPO on top of your SFT adapter. Re-run the eval. Then export to GGUF and run the model on your own machine.

  9. Step 9 of 9 · recent research

    Frontier — reward the reasoning with a rubric

    1 Watch & read

    • DeepSeek-R1 — Reasoning trained by reinforcement — the model is rewarded for results, not shown answers. Hindi video coming soon Read the paper ↗ How to read it
    • Rubric rewards with GRPO 2026 — Reinforcement fine-tuning of a quantized Qwen with LoRA, scored criterion by criterion against a rubric. Hindi video coming soon Read the paper ↗
    • Rethinking Rubric Generation 2026 — A vague rubric gives a vague reward — how to split criteria until they actually separate good from bad. Hindi video coming soon Read the paper ↗
    • Small Models as Rubric Judges 2026 — The judge runs thousands of times during RL — a small one can be good enough and far cheaper. Hindi video coming soon Read the paper ↗
    • Judge-R1 2026 — A legal system that pairs agentic retrieval of statutes with rubric-guided RL to cut invented citations. Hindi video coming soon Read the paper ↗

    2 Concepts explain each one in your own words before moving on

    • SFT and DPO show the model answers. Reinforcement fine-tuning scores the answers the model produces itself.
    • GRPO: sample a group of answers to one question, score each, and push the model toward the ones that scored above the group's average.
    • Law has no automatic checker, so the score comes from a rubric: did the answer identify the deeming provision, rank the authorities, cite something real?
    • The rubric is the whole game. Criteria that overlap or are vague give a reward the model can satisfy without reasoning better.
    • Reward hacking: the model finds the cheapest way to score well. Expect it and look for it.
    • The usual order is an SFT warm-up first, then RL.

    3 Quick quiz

    1. How does reinforcement fine-tuning differ from supervised fine-tuning?

    2. GST answers cannot be checked by a program the way maths can. What provides the reward?

    3. After RL, rubric scores rise sharply but the expert says the answers are no better. The likely cause is…

    4 Action

    Write a rubric from your five failure types, one checkable criterion each. Have the judge score 20 answers with it and compare against the expert, criterion by criterion. Only if they agree, run a GRPO notebook on top of your SFT adapter with the rubric as reward, and put the result beside the DPO score on the same eval.

← all tracks · home · progress is saved in this browser only