Definition
The context window of a large language model is the total amount of information the model can "see" at once when generating a response. It is measured in tokens — small chunks of text roughly equal to three-quarters of an English word — and it includes everything in the current exchange: the system instructions, your prompt, any documents you paste or upload, the earlier messages in the conversation and the model's own answer as it is being written.
Think of the context window as the model's working memory or the size of its desk. Whatever fits on the desk can be read, compared and referenced. Whatever falls off the edge is invisible. If a conversation or document exceeds the limit, the tool must drop, truncate or summarise some of it, and the model may lose track of details you mentioned earlier.
How big are context windows?
Context windows have grown dramatically. GPT-3 in 2020 handled 2,048 tokens — a few pages. By 2024 and 2025, mainstream models offered 128,000 tokens (roughly a 300-page book), and several, including Google's Gemini and Anthropic's Claude, offered windows of 200,000 to over a million tokens. The exact limits change frequently and differ between free and paid plans, so check your tool's documentation.
A bigger window does not mean unlimited attention. Research such as "Lost in the Middle: How Language Models Use Long Contexts" (Liu et al., Stanford University, 2023) found that models tend to use information at the beginning and end of a long context more reliably than information buried in the middle. Longer inputs also cost more and run slower.
Why the context window matters at work
- Long documents. Whether you can analyse an entire contract, report or transcript in one go depends on the window.
- Long conversations. In a lengthy chat, early instructions can be forgotten or diluted.
- Comparisons. Comparing several documents requires all of them to fit at once.
- Quality. Stuffing in irrelevant material can distract the model and lower accuracy.
- Cost. In paid APIs and some enterprise plans, you pay per token processed.
Context window versus memory
The context window is temporary: it covers one conversation or request. Some tools also offer a "memory" feature that stores facts about you across sessions, and retrieval systems (RAG) can fetch relevant snippets from large document libraries. These techniques work around the window's limits by putting only the most relevant information into it at the right moment.
Example
A project manager wants to prepare for a steering-committee meeting. She has a 70-page project status report, three months of meeting notes (about 40 pages) and a 25-page risk register. She pastes everything into a chat with a model that has a modest context window and asks, "What are the top risks and which ones were discussed but never resolved?"
The answer is vague and misses an important risk that appeared in the middle of the meeting notes. The combined material exceeded what the tool could reliably handle, and some content was silently truncated.
She changes approach. First, she asks the model to summarise each document separately into one page of key points, risks and open actions. Then she starts a fresh conversation, pastes in the three one-page summaries plus the full risk register, and repeats her question. This time the model identifies five unresolved risks, each traced to a specific meeting date. She checks two of them against the original notes and builds her briefing. By managing what went into the context window, she got a sharper and more reliable result.
Practical tips
- Front-load what matters. Put the key instruction at the start and repeat critical constraints at the end.
- Trim the input. Remove boilerplate, appendices and duplicates before pasting.
- Chunk and summarise. Break huge material into parts, summarise each, then work from summaries.
- Start fresh. Begin a new conversation when switching tasks or when the model starts forgetting instructions.
- Restate the brief. In long chats, briefly restate the goal and constraints.
- Use the right tool. For very long documents, choose a model with a large window or a tool with built-in document search.
Key takeaways
- The context window is the model's working memory, measured in tokens.
- Everything — instructions, files, history and the reply — shares the same budget.
- Bigger windows help, but relevance and placement still drive quality.
- Summarising, chunking and fresh conversations keep results accurate.