What a million-token context window actually means

A couple of years ago, the idea that a language model could "hold in mind" an entire book seemed like science fiction. Today it is a working reality. GPT-5 and Gemini 2.5 Pro allow you to load hundreds of thousands, even millions, of tokens into context.

In raw numbers: GPT-5 can process roughly 256,000 tokens per request. Gemini 2.5 Pro handles approximately 1,000,000 tokens.

To make that tangible, consider Bulgakov's The Master and Margarita, a novel most readers know well. Depending on the edition, it contains 150,000 to 200,000 words. Using OpenAI's rough ratio of 100 tokens per 75 words:

GPT-5 can hold the entire novel, first page to last, in a single request.

Gemini 2.5 Pro can hold three to four such novels simultaneously, without needing to "finish" one before starting the next.

This is no longer processing a couple of documents or a long chat thread. This is working with entire text corpora, legal case files, scientific reports, or large codebases in a single session.

The myth of flawless long context

At first glance, more context seems strictly better. Reality is more nuanced.

Research from Stanford and Anthropic, Lost in the Middle: How Language Models Use Long Contexts, revealed a telling effect: models use available context unevenly.

Place relevant information at the beginning or end, and the model almost always finds it. Place the same information somewhere in the middle of a massive text, and the probability it influences the output drops significantly. The resulting accuracy graph has a characteristic U-shape: high performance at the edges, a dip in the centre.

This is why "just dump everything into the model" is not always an effective strategy.

How to prepare data for long context

Working with large context windows requires attention to structure, not just capacity. A few approaches that yield consistently better results:

Remove noise. If the source document is a PDF or DOCX, convert it to Markdown or plain text before loading. This strips away formatting tags, styles, binary data, and other invisible elements that can consume hundreds or thousands of tokens without contributing anything useful.

Structure the material. Divide text into semantic blocks. Add brief annotations and a table of contents. This helps the model orient itself within large information sets.

Duplicate what is critical. Place key data closer to the beginning and end, minimising the risk of losing it in the "middle valley" of long-context processing.

Prioritise. Even a million tokens is not infinity. Sometimes it is more effective to load a carefully curated extract rather than the entire archive.

Does a typical user need a million tokens?

For specialised scenarios like legal analysis, scientific literature reviews, or comprehensive audits of large codebases, long context can be a decisive advantage. Insurance firms reviewing lengthy policy wordings, claims histories, or regulatory documents across multiple jurisdictions are a clear use case.

For the vast majority of tasks, whether marketing copy, scripts, short document analysis, or correspondence review, a context the size of one Master and Margarita is more than sufficient. The key is not simply having a large window, but filling it intelligently.

The real takeaway

Growth in context size is an important stage in AI development, but not a cure-all. Real effectiveness comes when a large context window combines with smart data organisation and an understanding of how the model actually reads your text.

More room does not mean better answers. Better structure does.