Scaling laws
Also known as: neural scaling laws, AI scaling laws, Chinchilla scaling
Scaling laws are patterns researchers found showing that AI language models improve in a predictable way as they're trained with more computing power, more data, and more parameters.
They've guided billions of dollars in AI investment, because they suggest that bigger training runs reliably buy better models. While scaling laws held remarkably well for years, data shortages, rising costs, and diminishing returns on some benchmarks have pushed labs toward new approaches, such as scaling reasoning at the time of use.
Scaling laws, explained at three levels
At home
Think of learning to play piano. More practice hours and better instruction make you improve, but each new level takes more hours than the last. Scaling laws describe the same pattern in AI: give a model more data, more computing power, and a bigger network, and its mistakes shrink in a predictable way, with each improvement costing more than the one before.
At work
Scaling laws are measured relationships showing that AI models improve predictably as training uses more computing power, data, and parameters. They explain why AI companies spend billions on chips and data centres: they can forecast roughly how good a bigger model will be before building it. For businesses buying AI, they explain why top models keep improving and why they cost so much.
At home
Researchers measure a model's 'loss,' how wrong its predictions of the next word are. Plotting loss against training compute on logarithmic scales gives a nearly straight line, called a power law. That means each tenfold increase in compute cuts errors by a similar fraction. Doubling resources doesn't double intelligence; it buys a steady, shrinking improvement, which is why each new generation of models costs so much more to train.
At work
Loss falls as a power law with three inputs: parameters, training tokens, and compute. Labs fit these curves on small runs, then predict the result of a run hundreds of times larger before spending the money. DeepMind's Chinchilla work put the compute-optimal balance at roughly 20 training tokens per parameter. Many labs now deliberately 'overtrain' smaller models far beyond that ratio, because a smaller model is cheaper to serve to millions of users.
At home
Scaling laws predict loss, a measure of next-word prediction error, not usefulness. A small drop in loss can suddenly unlock a skill, like solving a new kind of puzzle, or change nothing you'd notice. That's why benchmark gains don't always map onto everyday experience. While the curves are smooth, what people perceive is lumpy. In my opinion, that gap explains why some new models feel like big leaps and others don't.
At work
Pre-training scaling now faces limits: high-quality human text is running short, synthetic data needs careful verification, and power and chip supply constrain the largest runs. Labs have added two more scaling axes, reinforcement learning after pre-training and test-time compute, where models think longer per query, shifting cost from training to inference. Whether 'emergent' abilities are real or artefacts of the chosen metrics is still debated, so strategic plans that assume endless gains carry real risk.
Real-world examples
At home
Marcus read that a new AI model cost hundreds of millions of dollars to train. A friend explained that scaling laws give companies confidence that spending more will produce a better model.
At work
Fatima is a technology strategist at a Canadian bank. She cites scaling laws when explaining why AI vendors keep raising prices for top models while cheaper, smaller models improve quickly too.
Common confusions
- Scaling laws are laws of nature that will always hold. — They're observed patterns from experiments. They've held for years, but nothing guarantees they continue.
- Bigger models are always better. — Chinchilla showed many models were too big for their data. Balancing size, data, and compute matters more than size alone.
For analysts
Scaling laws explain the AI arms race, but I wouldn't plan around one vendor's next model being dramatically better. I'd design systems that can swap models as the market shifts.
History & origin
Researchers had noticed power-law patterns in machine learning before, including a 2017 Baidu study. In January 2020, Jared Kaplan and colleagues at OpenAI published 'Scaling Laws for Neural Language Models,' showing smooth improvements with size, data, and compute. In 2022, DeepMind's Chinchilla paper by Jordan Hoffmann and colleagues found that many models were too big for the data used, and that a 70-billion-parameter model trained on 1.4 trillion tokens beat larger ones. After OpenAI's o1 in 2024, labs described a second scaling law for 'test-time compute,' where letting models think longer improves answers. Debate over whether pre-training gains are slowing grew through 2025. In my view, scaling laws are real, but they describe a trend, not a guarantee.