The intellectual traditions behind ML

ML didn't borrow metaphors from physics — it inherited the math

The usual story is that machine learning "borrows metaphors" from physics. The truth is stronger and more interesting: in several places, ML literally is physics, applied to information instead of matter.

Boltzmann worked out how gas molecules distribute across energy states: the probability of finding a system in a state with energy E at temperature T is proportional to e^(−E/T). It answers a simple question — given many possible configurations, which does nature favour? Low-energy states are more probable, and temperature controls how strictly that preference is enforced.

Hopfield (1982), a physicist, recognized that a neural network is the same mathematical object: it has a state (its activations), an energy (its loss), and it seeks low-energy configurations (good solutions). He imported statistical mechanics into neural networks wholesale. Hinton then built Boltzmann machines — named for Boltzmann, using his exact distribution, temperature and all. Training one is essentially simulating a physical system cooling toward its lowest-energy state.

And the punchline: softmax is the Boltzmann distribution with the sign flipped — high logit means high probability, where physics had low energy meaning high probability. The "temperature" you set when sampling from an LLM, or when distilling a teacher into a student, isn't an analogy borrowed from thermodynamics. It is the same equation, transplanted. That's why raising temperature "softens" a distribution in exactly the way heating softens the occupancy of energy states.

The energy framing isn't decoration — it's the actual mathematical skeleton. Every time you tune a sampling temperature, you're turning Boltzmann's dial.

Why not chemistry?

Because chemistry is downstream of physics. Bonding, reaction rates, equilibrium constants — all derive from the same statistical mechanics. A chemist's equilibrium constant K = e^(−ΔG/RT) is the Boltzmann distribution again, wearing different notation. Chemistry never developed an independent mathematical machinery for ML to borrow; it inherited physics'. Had ML drawn from chemistry, it would have arrived at the same equations under different variable names.

A few ML ideas do echo chemistry as genuine analogies: activation energy resembles the loss barrier between two local minima (you need enough "energy" — learning rate, noise — to escape a poor solution), and catalysis resembles a good initialization or curriculum (it doesn't change the destination, but it speeds the route there). But those are analogies layered on top. The structural debt is to physics.

Why not biology?

Here the answer flips: early neural networks did borrow heavily from biology. McCulloch and Pitts modeled artificial neurons on biological ones; Hebb's rule — "neurons that fire together wire together" — inspired the first learning rules; the very word neural is biological. The clean division is this: biology gave ML its architectural intuition (layers, connections, signals — what to build), while physics gave ML its mathematical framework (energy, optimization, probability, temperature — how to analyze what you built).

The more literally biological approaches — evolutionary algorithms, genetic programming, swarm intelligence — exist and work, but they never achieved the mathematical elegance or the practical dominance of the physics-derived methods. Part of the reason: biological evolution is hard to formalize cleanly, whereas statistical mechanics arrived with centuries of polished mathematics ready to be repurposed.

The tradition ML didn't inherit

Physics gave ML optimization and probability: how to find the best answer, how to quantify confidence. What physics-derived math does not provide is an interpretive framework — how to decide which source of authority governs, how to recognize when a text creates a legal fiction that overrides literal reality, how to weigh intent against letter. Those are the questions of hermeneutics.

Hermeneutics means, literally, "the art of interpretation" — from Hermes, the Greek messenger-god who carried meaning between gods and humans. It began as the systematic interpretation of scripture, which faced exactly the problems a tax-law AI faces today: when a text contradicts observable reality, is it metaphor or a binding fiction? When two passages conflict, which governs? Does the author's intent matter, or only the literal words? These are not optimization questions. No amount of gradient descent discovers them, because they aren't in the statistical structure of text — they are principles about how to read.

And this tradition is not foreign to Indian law — it is foundational to it. Mimamsa, one of the six schools of Hindu philosophy (codified around 200 BCE), developed rules for interpreting Vedic texts, and several of its principles became cornerstones of Indian jurisprudence — among them that a specific provision overrides a general one and a later enactment overrides an earlier one. When an LLM answering a question of Indian tax law fails to apply specific-over-general, it isn't merely missing a software feature. It's failing to apply a 2,200-year-old hermeneutic principle that Indian courts inherited from Vedic exegesis.

The model optimizes beautifully and computes perfect distributions; what it lacks is the interpretive layer — and that layer was never in ML's mathematical inheritance to begin with. Fine-tuning on expert corrections is, in effect, installing the hermeneutic layer by example.

The deeper point: ML is inference under uncertainty

Why does ML draw on so many disparate traditions — physics, cryptography, linguistics, information theory, and now, implicitly, hermeneutics? Not because it is secretly one discipline. Because ML is fundamentally about inference under uncertainty — determining what is true when you cannot observe truth directly. Every intellectual tradition that ever wrestled with that problem developed tools, and ML absorbs them all.

1. tradition 01

   physics

   Wrestled with inferring the state of vast systems from limited observations. It gave ML energy, temperature, and the Boltzmann/softmax form.

2. tradition 02

   cryptanalysis

   Wrestled with recovering hidden messages from intercepted signals. It gave ML the hidden-state inference of HMMs — the engine of classical speech recognition.

3. tradition 03

   linguistics

   Wrestled with the structure of sound and meaning. It gave ML phone units and the noisy-channel view of language.

4. tradition 04

   information theory

   Wrestled with transmitting messages through noisy channels. It gave ML entropy, cross-entropy, and KL divergence — the loss functions of modern training.

5. tradition 05

   hermeneutics

   Wrestled with recovering correct meaning from ambiguous text. Exactly the tradition legal AI needs — and the one ML's math omits.

To make it concrete, take one specialized domain as a worked example — Indian GST law. A system answering GST questions sits squarely in this lineage. Its hidden states are legal realities (is this a supply? is input-tax credit eligible?); its observations are the text of statutes, circulars, and fact-patterns; its decoder is a reasoning pipeline searching for the interpretation that best fits both. The work is the same as every entry above — principled inference of an unobservable truth from ambiguous evidence — with one twist: here the rules of inference are hermeneutic, and must be taught rather than derived.

Glossary

This essay is one chapter of the study curriculum behind attention.sh. The research page lists what we're building in this space, and the story walks a real prompt through a transformer — softmax, temperature, and all. Questions, disagreements, additions — hello@attention.sh.