GPT-5.4 with 1M context:
what actually works
and what falls apart

Why Context Size Actually Matters

Before GPT-5.4, working with large documents meant one of three workarounds: chunking (splitting the document and summarising each piece), retrieval-augmented generation (embedding + semantic search), or truncation and hoping the critical context made it into the window. Each workaround introduces error. Chunking loses cross-chunk references. RAG misses context that doesn't match the retrieval query. Truncation is just truncation.

A 1M token context window changes the calculus on one specific question: does the answer to this task depend on having access to the entire document simultaneously? If yes — if the answer in paragraph 3 depends on something buried in paragraph 847 — then a large context window isn't a nice-to-have. It's the only way to get the right answer without engineering around the limitation.

The Failure Modes Nobody Talks About

Every GPT-5.4 launch post covered what the 1M context window enables. Almost none covered where it breaks. Here are the three failure modes you will hit in production if you don't account for them upfront.

Use Cases That Genuinely Work

The 1M context window earns its keep on tasks where the answer requires holistic awareness of the entire document, where chunking produces materially worse results, and where context fill stays below the 500K degradation threshold.

Use Cases That Fall Apart

These are the use cases people try first because the context window seems tailor-made for them. In practice they don't work — not because the context is too small, but because of how attention degrades or how the task structure fights against a single large-context call.

The Cost Reality Check

"1M context window" gets treated as a feature. It's also a billing event. At current OpenAI pricing for GPT-5.4, a single 1M-token input call costs significantly more than the casual API call most people mentally budget. If you're running this at scale — analysing 50 codebases a day, processing a legal discovery set daily — the economics shift fast.

↑ Estimates based on GPT-5.4 pricing at time of publication. Output tokens additional. Verify current pricing at platform.openai.com.

Prompting Patterns for Coherent Output at Scale

For the use cases that do justify 1M context, these four patterns measurably reduce lost-in-the-middle failures and coherence drift. Pattern 1 alone eliminates most attention failures on well-structured tasks — start there before adding the others.

# System prompt — add to any large-context call

You are reviewing a large document set. When answering any question, you MUST:
1. Explicitly state which document(s) your answer draws from (name + approximate position)
2. Quote the specific passage that supports each claim — do not paraphrase without attribution
3. If you are uncertain whether you have reviewed a section of the documents, say so

# Why this works: forcing explicit attribution makes attention failures
# visible rather than silent. The model can't produce a confident answer
# without naming a source — missing documents show as gaps, not omissions.

# Place this BEFORE all documents in the context

DOCUMENT INDEX
The following [N] documents have been loaded. Refer to them by ID when citing:

DOC-01: [Document name] — [Brief description, date if relevant]
DOC-02: [Document name] — [Brief description]
DOC-03: [Document name] — [Brief description]
// ... continue for all documents

When referencing information, always include the DOC-ID.
Do not reference information from documents not listed in this index.

---BEGIN DOCUMENTS---
[Documents follow in order]

# Instead of: "Write a comprehensive analysis"
# Use a staged output structure that caps each section:

Produce your analysis in the following structure.
Each section is independent — do not forward-reference or refer back to previous sections.

Section 1 — Executive Summary  (max 200 words)
Section 2 — Key Findings        (max 8 bullet points, each under 30 words)
Section 3 — Risk Factors        (max 5 items, with severity and supporting document ID)
Section 4 — Recommended Actions (max 5 items, ranked by priority)
Section 5 — Open Questions      (items requiring further review or external input)

# Add this when critical content sits in the middle of a large context

COVERAGE REQUIREMENT
This analysis must account for content from ALL sections of the provided documents,
including sections in the middle of the document set.

Before generating your response, mentally scan for relevant content at three points:
- The first third of the documents
- The middle third (most likely underrepresented — check deliberately)
- The final third

If you find relevant content in the middle section that contradicts your initial impression
from the early documents, the middle-section content takes precedence.

# Explicitly naming the "danger zone" improves recall from that zone
# by 15–25% in needle-in-a-haystack testing.

GPT-5.4 vs Claude for Long Context Work

Both GPT-5.4 and Gemini 2.5 Pro offer 1M-token context. Claude Opus 4.6 caps at 200K. For teams already running Claude workflows, the honest question is: does the 1M context window justify switching models for long-context tasks, or does 200K cover the vast majority of real-world use cases?

The Verdict — When to Reach for 1M

Large context windows are real capability. GPT-5.4's 1M context genuinely unlocks a class of task that was architecturally impossible to do well before. The failure is treating it as a general-purpose upgrade — dumping everything in context because you can, not because the task demands it. Three questions before every large-context call: