Context Engineering: What’s on the Model’s Desk Matters

Prompt engineering teaches the model how to think; context engineering decides what’s on the table in front of it. With the same model, the quality of how the context is arranged can shift output quality by a whole grade. This step sits more fundamentally than writing prompts.

How it relates to prompt engineering

The prompt is “the instructions you give the model”; the context is “all the material you put into the window at the same time”: system settings, conversation history, retrieved material, tool returns. Even with perfect instructions, if the material is all noise the model can’t answer well. The two are layers: context is the foundation, the prompt is the direction on top. A lot of the blame aimed at “bad prompts” is actually wrong context assembly — what should be there isn’t, and what shouldn’t be is stuffed in. Check the context first, then blame the prompt.

What you put in matters more than how much

Beginners love stuffing every findable piece of material into the window, and the key points get diluted. The first principle of context engineering is relevance: put in only what the current task actually uses — less but clean beats more but messy. One highly relevant piece beats ten marginal ones. To decide whether something goes in, ask “if this were missing, would the model answer wrong?” If yes, put it in; if no, drop it. Treat the window like a desktop where every inch is precious, and only spread out the files you’re working on. Like a reader, a model is efficient when the desk is clean.

Order matters too

The model is more sensitive to the beginning and end of the window and tends to overlook the middle. Put the most critical material — hard constraints, core data — at the head and tail, secondary material in the middle, and the hit rate visibly differs. Ordering is really attention allocation. Also arrange the relationship between “instructions” and “material” correctly: if the instruction says “answer only from the following material,” the material should follow it closely, not be separated by a long stretch of history. Keep the instruction right next to what it constrains and the model won’t drift.

When it overflows the window, compress

When a task gets long, the raw conversation and material inevitably overflow the window. The approach is step-by-step summarization: after each step, compress that step into key points and keep them, discarding the raw stream. Compression preserves key information without bursting the window. Compression must preserve structure: don’t keep only the conclusion and lose “which material the basis came from,” or later tracing becomes impossible. Good compression is like writing meeting minutes — points plus sources, not smearing the process into one sentence. Compression with measure keeps the chain intact.

Maintain it dynamically, don’t set it once

Context isn’t assembled once and left alone. As the task advances, stale information must be deleted and new critical material added. Maintenance is continuous: every few steps clear expired content and add the new focus, so the window is always “currently relevant.” Unmaintained context rots: early wrong temporary conclusions keep occupying space and the model keeps referencing them into error. Make maintenance a habit rather than cleaning up after something breaks. Dynamism is the lifeline of context engineering.

How to arrange retrieved results

The chunks RAG pulls back are an important source of context. The arrangement has rules: put the most relevant few in key positions with source markers; when multiple sources conflict, put the authoritative one first and mark “defer to this.” Only then does the model have evidence to rely on. Also control the count — don’t pull ten chunks and stuff them all in. Truncate to three to five by relevance, enough without drowning the model. Retrieval and context are upstream and downstream: when retrieval is accurate, the context is easy to assemble; when assembled well, retrieval isn’t wasted.

How to handle tool returns

Raw data returned from function calls is often verbose, like a whole page of API response. Putting it all in the context directly blasts the noise level. The approach is extracting key fields first, putting in only the part the model needs for its next step. When extracting, keep the necessary structure, like “status code + key fields + summary,” rather than raw JSON. A structured summary the model can read is more useful than the original big object. Handling tool returns is the most easily overlooked link in context engineering.

Three common pits

Pit one: more is better, stuffing noise that buries the key points. Pit two: random order, key information buried in the middle and ignored. Pit three: no compression and no maintenance, the window rots and steers the model wrong. All three are resolved by “weight relevance, mind order, compress often, maintain constantly.” There’s no showy trick to context engineering — it’s the basic skill of keeping the model’s desktop clean.

How it works with memory

Context manages “what’s in the current window”; memory manages “what accumulates across tasks.” They connect: relevant entries pulled from memory get filtered and arranged by context engineering, then fed to the model; after use, the model writes new experience back into memory. Done well together, the model is neither limited by the window (thanks to compression) nor loses history (thanks to memory). Treat context as the short-term workbench and memory as the long-term archive, keep the duties separate, and an agent is both stable and coherent.

Measuring whether context engineering is good

Watch two signals: with the same model and same prompt, how much does output quality differ across different context arrangements — a big difference means this step is very valuable; and whether the share of irrelevant information is low and key information sits at head or tail. More practically, run controlled experiments: fix the model and prompt, change only how the context is assembled and ordered, and watch the task success rate move. Let the comparison speak, and context engineering goes from mysticism to an optimizable step.

Start with one change, don’t aim for perfect

Don’t expect to make context engineering perfect in one pass. Pick the one link that most affects quality and start there — say, putting key information at head and tail first, and see whether output improves immediately, then snowball into the rest. Optimize by comparison rather than feel: change only one arrangement at a time, and compare answers before and after. With few variables, you can tell which move actually works. Context engineering is slow work, tuned round after round, getting more accurate each time.