Context window
Also known as: context length, context limit, token limit, context size
A context window is the limit on how much text an AI language model can work with at one time, counting your instructions, any documents you attach, the conversation so far, and the model's own answer.
It's measured in tokens, which are pieces of words. While context windows have grown from a few pages to the length of several novels, a bigger window doesn't guarantee the model pays equal attention to everything inside it.
Context window, explained at three levels
At home
Think of the context window as the AI's short-term memory for one conversation. If a chat gets very long, the earliest parts may fall out of that memory.
At work
The context window decides how big a document or how long a conversation an AI tool can handle in a single request.
At home
Context is measured in tokens; in English, a token is about three-quarters of a word. Your messages, the AI's replies, and any files you upload all count toward the limit.
At work
Longer contexts cost more, since pricing is per token, and slow responses. Applications manage context by summarizing older messages, retrieving only relevant passages (RAG), and trimming tool outputs.
At home
Even with huge context windows, AI models can miss details hidden in the middle of long documents. In my opinion, pasting a whole book and asking about one page isn't always better than pasting just the right page.
At work
Attention cost grows with sequence length, so providers use techniques like efficient attention, KV caching, and prompt caching. Quality should be tested with realistic long-context tasks, not only 'needle in a haystack' tests. Context engineering, deciding what goes into the window, is now a core skill for AI applications.
Real-world examples
At home
Marcus pasted a long rental agreement into an AI chatbot and asked about pets. The answer was right, but when he later added three more documents, the AI started mixing up details.
At work
Nadia builds internal AI tools for a Halifax insurance company. She found that sending only the five most relevant policy sections, rather than the whole 300-page manual, gave cheaper, faster, and more accurate answers.
Common confusions
- A bigger context window means the AI remembers everything perfectly. — Models can still overlook details, especially in the middle of long inputs. More room isn't the same as better attention.
- The context window is the AI's long-term memory. — It only covers the current request. Memory features that persist across chats are stored separately and added back into the context.
For analysts
When evaluating AI tools for long documents, I'd test with your real files and questions. Advertised context size says little about whether answers stay accurate at that length.
History & origin
OpenAI's GPT-2 in 2019 handled 1,024 tokens, and GPT-3 in 2020 handled 2,048, roughly a few pages of text. GPT-4 launched in March 2023 with 8,000- and 32,000-token versions. Anthropic expanded Claude to 100,000 tokens in May 2023, and Google announced a one-million-token context window for Gemini 1.5 in February 2024. Research such as the 2023 'Lost in the Middle' paper from Stanford and others showed models often overlooked information buried in the middle of long inputs, prompting new long-context benchmarks. By 2025 and 2026, windows of hundreds of thousands to a million or more tokens were common among leading models. In my view, context windows grew faster than our ability to use them well.