Context window
Also known as: context length, context limit, token limit, context size
A context window is the limit on how much text an AI language model can work with at one time, counting your instructions, any documents you attach, the conversation so far, and the model's own answer.
It's measured in tokens, which are pieces of words. While context windows have grown from a few pages to the length of several novels, a bigger window doesn't guarantee the model pays equal attention to everything inside it.
Context window, explained at three levels
At home
Think of a context window as the size of the desk an AI works on. Everything it needs for your conversation, your messages, its replies, and any files you add, has to fit on the desk at once. When the desk fills up, older pages get pushed off the edge, which is why a very long chat can 'forget' what you said at the start.
At work
A context window is the maximum amount of text an AI model can take in for one request, counting instructions, attached documents, chat history, and its own answer. It is measured in tokens, which are pieces of words. For a business, it decides whether a tool can read a whole contract or only part of it, and since providers charge per token, it also shapes cost.
At home
AI models read text as tokens; in English, a token is about three-quarters of a word, so 100,000 tokens is roughly a 300-page book. Each time you send a message, the app sends the model the whole conversation again, plus any uploaded files, because the model keeps no memory between requests. When the total nears the limit, the app has to drop or summarize older messages, which is why details from early in a long chat can disappear.
At work
Every request bills for the input tokens sent plus the output tokens generated, so a long context costs more and takes longer before the first word appears. Applications therefore manage the window actively: summarizing older turns, retrieving only relevant passages with RAG instead of whole documents, trimming verbose tool outputs, and putting stable instructions first so prompt caching can reuse them. As of 2026, limits range from about 128,000 tokens to over a million, depending on the model.
At home
Inside the model, attention lets every token look at every other token, but that attention is spread thin across a huge input. Studies such as 'Lost in the Middle' found models recall facts near the start and end of a long prompt better than facts buried in the middle. While a bigger window lets you paste a whole book, the model doesn't read it the way you would. In my opinion, pasting just the right page often beats pasting the entire book.
At work
Standard attention cost grows with the square of sequence length, and the KV cache grows with every token, so long contexts strain GPU memory. Providers counter with efficient attention variants, KV cache compression, and prompt caching. Quality also degrades unevenly with length, so 'needle in a haystack' tests, which find one planted fact, overstate real performance; benchmarks such as RULER test harder multi-fact tasks. Context engineering, deciding what goes into the window and in what order, is now a core skill.
Real-world examples
At home
Marcus pasted a long rental agreement into an AI chatbot and asked about pets. The answer was right, but when he later added three more documents, the AI started mixing up details.
At work
Nadia builds internal AI tools for a Halifax insurance company. She found that sending only the five most relevant policy sections, rather than the whole 300-page manual, gave cheaper, faster, and more accurate answers.
Common confusions
- A bigger context window means the AI remembers everything perfectly. — Models can still overlook details, especially in the middle of long inputs. More room isn't the same as better attention.
- The context window is the AI's long-term memory. — It only covers the current request. Memory features that persist across chats are stored separately and added back into the context.
For analysts
When evaluating AI tools for long documents, I'd test with your real files and questions. Advertised context size says little about whether answers stay accurate at that length.
History & origin
OpenAI's GPT-2 in 2019 handled 1,024 tokens, and GPT-3 in 2020 handled 2,048, roughly a few pages of text. GPT-4 launched in March 2023 with 8,000- and 32,000-token versions. Anthropic expanded Claude to 100,000 tokens in May 2023, and Google announced a one-million-token context window for Gemini 1.5 in February 2024. Research such as the 2023 'Lost in the Middle' paper from Stanford and others showed models often overlooked information buried in the middle of long inputs, prompting new long-context benchmarks. By 2025 and 2026, windows of hundreds of thousands to a million or more tokens were common among leading models. In my view, context windows grew faster than our ability to use them well.