Temperature (LLM sampling)
Also known as: LLM temperature, sampling temperature, AI temperature setting
Temperature is a setting that controls how much randomness an AI language model uses when picking each next word.
At low temperature, the model sticks to its most likely choices, giving focused and repeatable answers. At high temperature, it takes more chances, giving varied and creative but sometimes odd results. While temperature is often explained as a creativity dial, even a temperature of zero doesn't always guarantee identical answers.
Temperature (LLM sampling), explained at three levels
At home
Think of a jukebox that usually plays the most popular song, but sometimes picks something unexpected. Temperature sets how often it takes that chance. When an AI writes, it chooses each word from a list of likely options. A low temperature makes it nearly always pick the top choice, while a higher one lets less likely words through, making text more varied and sometimes stranger.
At work
Temperature is a setting that controls how much randomness an AI language model uses when choosing each next word. Low values give focused, repeatable answers; higher values give more varied, creative ones. Businesses set it per task, low for data extraction, classification, and code, and higher for brainstorming or marketing copy, and document it alongside the prompt so behaviour stays consistent.
At home
For each next word, the model gives every possible token a score, and a formula called softmax turns those scores into probabilities. Temperature divides the scores before that step. A low temperature exaggerates the gaps, so the top choice dominates; a high temperature shrinks the gaps, so unlikely words get a real chance. At zero, the model simply picks the top option, which is why answers become nearly repeatable.
At work
Temperature usually works alongside two cut-off settings. Top-k keeps only the k most likely tokens, while top-p, or nucleus sampling, keeps the smallest set whose probabilities add up to p, such as 90 percent. Typical temperature ranges are 0 to 1 or 0 to 2 depending on the provider, and some reasoning models don't let you change it at all. Changing several sampling settings at once makes results hard to interpret, so most teams adjust one.
At home
Temperature doesn't change what the model knows; it only changes how boldly it picks from its existing guesses. If the model's top guess is wrong, low temperature repeats that error reliably, and high temperature just adds variety, some of it worse. While raising it can make writing feel fresher, it can't fix a misunderstanding. In my opinion, if answers are wrong, better instructions usually help more than tweaking temperature.
At work
Even at temperature zero, outputs can vary. GPU floating-point arithmetic isn't perfectly associative, and the batch your request shares on a provider's servers changes how calculations are grouped, which can flip a near-tie between tokens and send the text down a different path. Silent model updates add more drift. For reproducibility, pin model versions, log every sampling setting, and use evals that tolerate wording variation rather than assuming determinism.
Real-world examples
At home
Nadia used an AI to brainstorm names for her bakery. Turning up the temperature gave quirky, unexpected ideas, while a low setting kept repeating safe names.
At work
Carlos is a data engineer at a Saskatoon agriculture company. He set temperature to zero for an AI tool that pulls figures from invoices, reducing inconsistent results.
How it compares
| feature | Low temperature | High temperature |
|---|---|---|
| Word choices | Most likely only | More varied |
| Repeatability | High | Low |
| Best for | Facts, extraction, code | Brainstorming, fiction |
| Risk | Bland or repetitive | Incoherent or off-topic |
Low temperature is steady. High temperature is surprising.
For analysts
For business processes, I'd set and document temperature with the prompt and model version. Treating it as part of the specification avoids mysterious changes in behaviour.
History & origin
The term comes from statistical physics, where temperature controls how evenly energy spreads across states, following Ludwig Boltzmann's work in the 1800s. Machine learning borrowed the idea for the softmax function, which turns model scores into probabilities; Geoffrey Hinton and colleagues used temperature in their 2015 work on knowledge distillation. As text generators improved, developers used temperature to balance coherence and variety. In 2019, Ari Holtzman and colleagues introduced nucleus sampling, or top-p, as a complementary control that limits choices to the most probable words. AI APIs from OpenAI, Anthropic, Google, and others exposed temperature as a standard parameter, though some reasoning models restrict changing it. In my view, temperature is the most misunderstood setting in AI, often blamed for problems that better prompts would fix.