Harness Theory Part 04 | Context Engineering: Where Do Context, Memory, and RAG Belong?

Every iteration of an agent loop has to answer the same question:

What should the model see this time?

The answer is rarely everything the system currently holds.

A coding agent repairing a payment module may have access to conversation history, a repository map, hundreds of files, API documentation, previous incidents, test logs, user preferences, the last tool result, and several passages returned by RAG.

Any of those may matter. They do not all belong in the same model call.

Too little context hides constraints. Too much context dilutes them with repeated, stale, or low-value material. A larger context window increases capacity; it does not identify authority, remove obsolete claims, or prevent material from another tenant entering the prompt.

Anthropic describes context engineering as the recurring discipline of curating the context most likely to produce the desired behaviour at inference time. It covers far more than prompts, including tool definitions, message history, external data, and runtime observations. Anthropic also treats context as a finite resource with diminishing marginal returns. The objective is not brevity for its own sake, but the smallest high-signal set that is sufficient for the task.¹

For a harness, context engineering is a governance process:

Discover
→ Qualify
→ Select
→ Assemble
→ Observe
→ Persist or discard
→ Invalidate when sources change

It decides what the model sees now while ensuring that omitted material remains in the appropriate authoritative system and can be retrieved later.

Context Builder and bounded model context

Figure 4-1 | The context builder selects material for the current turn, retaining its source, version, scope, and inclusion reason before assembling a bounded model context.

Context is neither the chat history nor everything the system knows

The word context is often used for everything available to an agent. An engineering design needs at least five separate categories.

Category	Primary purpose	Canonical authority?	Loaded by default?
Model context	Input to the current inference	No	Only what the turn needs
Runtime state	Run lifecycle and state transitions	Yes for the run lifecycle	Usually a selected view
Durable artefacts	Versioned specifications, plans, and evidence	Depends on the artefact type	Task-scoped
Long-term memory	Reusable information across sessions	Usually not the final authority for business or run state	Selected by scope and task
Retrieved evidence	Candidate evidence for the current question	Authority comes from the original source	On demand

Model context

Model context is the set of tokens sent for the current inference, such as:

stable instructions
the user request
a task-contract summary
selected conversation history
tool definitions
retrieved evidence
the latest observation
an output schema

A model-readable policy summary may be included, but permissions, budgets, sandbox boundaries, and approval requirements still need runtime enforcement. Writing “do not access the production database” in the prompt does not remove credentials or close a network route.

Model context is temporary inference input. It is not the system of record.

Runtime state

Runtime state records execution facts, including:

run, turn, and task status
current state version
pending approvals
tool request/result receipts
consumed budget
active background work
workspace and tenant identity
cancellation state
side-effect operation identifiers

The context builder may turn part of this into a model-readable summary. Canonical state must not exist only as prose in the conversation.

Durable artefacts

Durable artefacts are versioned and reviewable work products that survive sessions, such as:

project context
specifications
plans
architecture decision records
progress ledgers
test evidence
diffs, build logs, and review findings

They may live in a repository, document store, object storage, or another authoritative system. They do not need to remain in every prompt, and material should not be copied into memory merely because it is not currently loaded.

Long-term memory

Memory retains information worth reusing across sessions, such as confirmed preferences, approved decisions, or stable working background.

It should not replace canonical task progress, business records, or repository facts. The current position of a task belongs in run state or a progress ledger. Memory may retain a governed summary and pointer, but it should not become the final status authority.

Memory needs:

scope
schema
provenance
verification status
retention
consent
supersession
revocation and deletion

“The model said this earlier” is not a sufficient write policy.

Retrieved evidence

Retrieved evidence is acquired for the current question through a vector index, SQL, document search, the web, code search, or another tool.

A retriever returns candidate content. A high similarity score does not convert it into truth. Authority comes from the originating system and its governance, not from vector distance.

The harness still has to determine:

whether the material belongs to the correct tenant and scope
whether the actor may use the source
whether its version and freshness suit the task
whether it contains untrusted text that must not become instructions
whether it supports the eventual claim
whether another source is required

Combining these five categories in one undifferentiated prompt makes three questions difficult to answer: where did this statement come from, is it still valid, and why did the model see it in this turn?

The context builder: who decides what the model sees?

The context builder is a runtime assembler. It obtains candidate sections from several sources, applies a selection policy, and emits the bounded structure accepted by the model API.

Typical inputs include:

agent configuration and stable instructions
user request and task contract
canonical run state
tenant, identity, and permission decisions
project and work-track artefacts
retrieved evidence
tool catalogue
tool results
verifier feedback
remaining token, time, and cost budgets
output schema

It does more than concatenate strings. It must address at least seven conditions:

Condition	Question
Relevance	Does this section help the current step?
Source authority	Is it policy, user input, an artefact, an external system, or unverified model output?
Freshness	Is the derived material valid after source changes?
Isolation	Does it belong to the correct tenant, workspace, task, and actor?
Instruction boundary	Is it an instruction or untrusted material that may only be read as data?
Boundedness	Are tokens, tool outputs, and duplication controlled?
Traceability	Can its source, version, digest, and inclusion reason be reconstructed?

The context builder should not promote low-trust material into authority. It preserves source labels, applies defined precedence, and refuses assembly or requests clarification when a conflict cannot be resolved safely.

The following architecture-level manifest is illustrative rather than a provider API:

{
 "run_id": "run_204",
 "scope": {
 "tenant_id": "tenant_acme",
 "workspace_ref": "checkout-fix@a81c4f2"
 },
 "sections": [
 {
 "id": "task-contract",
 "role": "instruction",
 "authority": "approved_spec",
 "source_ref": "spec-221",
 "source_version": "spec-v7",
 "tokens": 842
 },
 {
 "id": "retrieved-evidence",
 "role": "untrusted_data",
 "authority": "project_docs",
 "source_ref": "docs-query-481",
 "source_version": "docs-index-93",
 "tokens": 2140
 },
 {
 "id": "latest-test-result",
 "role": "runtime_evidence",
 "authority": "test_runner",
 "source_ref": "artifact-test-881",
 "source_version": "sha256:...",
 "tokens": 1260
 }
 ],
 "budget": {
 "context_tokens": 18000,
 "reserved_output_tokens": 4000
 }
}

The token figures are illustrative, not universal budgets.

What this example establishes

A section can carry scope, role, authority, source reference, version, and token accounting.
Different sections may have different owners and authoritative origins.
Retrieved material can be labelled untrusted_data rather than being treated as a system instruction.
Output space should be reserved instead of filling the whole window before generation.
A trace can record included and omitted sections with their source versions.

What it does not establish

It does not prescribe a prompt format or provider API.
It does not show that the illustrative budget suits a real workload.
It does not fully solve prompt injection, secrets, masking, or tenant authorisation.
It does not show that retrieved evidence is correct or sufficient.
It does not demonstrate that the selection policy has been evaluated.

A manifest makes context observable. It does not make the selection policy intelligent by itself.

Runtime prompt assembly: keep stable and dynamic sections separate

A system prompt should not become one ever-growing string. A more maintainable design gives each section an owner, version, priority, token budget, and loading condition.

Stable sections

Material that may belong in a stable prefix includes:

identity and operating principles
model-readable summaries of invariant policy
output conventions
stable tool guidance

Dynamic sections

Material that may change by run or turn includes:

workspace, branch, tenant, and actor
current tool catalogue
task, plan, and run state
memory-index generation
connected external capabilities
current budgets
retrieved evidence

Dynamic state should not contaminate a reusable stable-prefix cache. Changes to tools, permissions, memory, workspace, or policy should invalidate the relevant assembly.

Every model call should record at least:

loaded section identifiers
omitted section identifiers and reasons
section digests and versions
total token count
cache-hit scope
stable/dynamic boundary

Sections should be selected from real state and policy rather than guessed solely from query keywords.

Progressive disclosure: map first, detail for the task, evidence on demand

A coding agent rarely needs the whole repository on its first turn. A layered loading strategy is more reliable.

Progressive disclosure for agent context

Figure 4-2 | An always-on map provides navigation, task-scoped context supports current work, and on-demand evidence is acquired through governed tools.

Always-on map

The always-on map should be short, stable, and navigable. It commonly includes:

project purpose and major modules
build, test, lint, and run entry points
important safety and data constraints
an index of deeper documentation
an entry point to current task state

Its job is to tell the agent where to look, not to replace the knowledge base.

Task-scoped context

Load only what the current task needs:

relevant modules and real file paths
API, schema, and dependency boundaries
established implementation patterns
feature or defect specifications, plans, and acceptance criteria
allowed scope and required verification

When the task or run state changes, this layer should be selected again. Similar-looking tasks should not inherit one another’s context by default.

On-demand evidence

Large, infrequently used, or rapidly changing material should remain behind tools:

historical decisions
previous incidents
large logs and traces
external documentation
complete test output
large tables

Just-in-time retrieval depends on reliable identifiers, such as a file path, document ID, query handle, or artefact URI. Anthropic similarly distinguishes agentic search from loading all possible material in advance.¹

Repository-native context does not require all knowledge to be copied into Git. Code, build commands, and architectural rules often belong with the repository. Commercial, sensitive, cross-project, or live data may remain in other canonical systems and be exposed through controlled tools.

A fixed percentage of the context window is not a general design rule. The balance between map, task context, and evidence should be evaluated against task success, constraint recall, retrieval precision, tool errors, latency, and cost.

The context artefact lifecycle: govern intent, plans, progress, and evidence

Chat history records interaction. It is a weak sole specification for a complex project.

Work that spans turns, sessions, or operators needs durable artefacts.

Context artifact lifecycle

Figure 4-3 | Project context, specifications, plans, progress, and verification evidence have separate versions and review states. A change creates a new version rather than silently rewriting approved intent.

A useful artefact state graph is:

Bootstrap Project Context
→ Create Work Track
→ Draft Spec
→ Review / Approve Spec
→ Draft Executable Plan
→ Review / Approve Plan
→ Implement + Update Progress Ledger
→ Verify Completion
→ Promote Durable Knowledge / Retain Evidence / Archive Disposable Material

When an assumption, scope boundary, or architecture changes mid-flight, the system should not simply overwrite an approved document:

Detect change
→ create a new artefact version
→ record reason and impact
→ re-review the affected specification or plan
→ update verification requirements
→ resume from an explicit state

Google Conductor stores project context, specifications, and plans as persistent Markdown beside the code. The significant property is not Markdown itself. Intent, plans, and progress no longer exist only in an ephemeral chat session. Google’s workflow also requires plan approval before implementation.²

Project context

Relatively stable material across work tracks:

product users and goals
architecture and domain boundaries
technology stack
team workflow and testing rules
security and operational constraints

Work-track context

Material for one defect, feature, or migration:

problem and rationale
scope and out-of-scope work
specification
plan and phases
risks
acceptance criteria
progress, blockers, and decisions

Runtime evidence

Verifiable facts produced during execution:

test results
build logs
diffs
operation receipts
traces or screenshots
review findings
verification reports

Completion should not send all of this material into one knowledge folder:

reusable, verified decisions may become ADRs or maintained guidance
completion evidence follows its retention policy rather than being discarded as temporary material
superseded specifications and plans retain their version relationships
low-value exploratory drafts may be archived or deleted

Brownfield bootstrap needs more than one automatic summary

Context for an established system should combine:

code and dependency analysis
existing documentation
tests
Git history
runtime evidence
human interview and confirmation
architecture decision records

An automatically produced project description is a draft. Conflicting sources should produce an explicit conflict and unresolved assumption rather than whichever story reads most smoothly.

Context drift

Artefacts and the system will diverge without maintenance. Important artefacts need:

owner
version
status
last verified date
canonical sources
related code, schema, or policy
stale triggers
supersedes/superseded-by relationships
archive or retention policy

Changing a plan during execution is not the problem. Failing to record the reason, impact, re-approval, and new verification requirements is.

Compaction, clearing, and memory solve different problems

When context grows, teams often reach immediately for a summary or memory store. Either can be the wrong primitive.

Compaction, clearing and memory comparison

Figure 4-4 | Compaction compresses the working narrative, clearing removes safely reproducible payloads, and memory retains reusable cross-session material. Canonical progress remains in runtime state or a progress ledger.

Primitive	Main problem	Suitable retained material	Responsibility it should not own
Compaction	The transcript as a whole is too long	Goal, decisions, open issues, evidence references, next step	Lossless history or durable-state authority
Tool-result clearing	Re-fetchable tool payloads dominate context	Call record, source ID, digest, retrieval handle	Side-effect receipts, approvals, or irreproducible evidence
Memory	Information needs reuse across sessions	Confirmed preferences, verified decisions, curated background	Canonical task progress or business records

Compaction

Compaction summarises a long transcript so that work can continue in a new or reduced context.

A summary contract should retain:

current objective
user constraints
accepted decisions and reasons
modified resources
tests and evidence references
open failures
active operation identifiers
remaining work
next safe action

Compaction is lossy. Its prompt, required fields, evaluation probes, and recovery path need testing. A detail judged unimportant now may become the root cause later.

Tool-result clearing

File reads, search pages, API responses, and large logs often consume the most tokens. A safely reproducible payload can be removed while retaining:

tool-call identity
source identifier and version
digest and metadata
retrieval handle
required citation positions

Do not clear:

side-effect receipts
approval decisions
operation identifiers
irreproducible outcomes
evidence required by later verification

Memory

Memory is an external persistence layer. It should not save every model output or load every stored record into every turn.

A usable memory lifecycle separates:

Selection → load relevant memory
Extraction → propose candidate memory
Consolidation → approve, merge, supersede, expire, or delete

A candidate memory is not a fact. Before writing it, ask:

Does it need to survive the session?
Does a more authoritative source already hold it?
Is it verified fact, confirmed preference, or model inference?
Does it belong to user, project, tenant, or session scope?
Is it sensitive, and would the user expect it to be retained?
Who can inspect, correct, revoke, export, or delete it?
When should it be superseded or expire?

An illustrative record might be:

memory_id: mem-123
scope: project
subject_id: checkout-service
type: verified_decision_summary
content: "PostgreSQL remains the source of truth for payment state."
source_ref: adr-017
source_version: 4
verification_status: approved
approved_by: architecture-owner
created_at: 2026-07-08
expires_at: null
policy_version: memory-policy-4

The YAML does not make the record trustworthy. Its canonical source, version, approval status, and lifecycle do.

Task progress belongs in a progress ledger:

Canonical progress ledger
→ context builder selects a relevant view
→ model sees current progress

The reverse arrangement, in which a memory summary becomes task truth, creates split-brain state.

Anthropic’s cookbook also treats memory, compaction, and tool-result clearing as different techniques for different context bottlenecks. It notes that subagent isolation and programmatic tool calling can keep large data out of the main context entirely.³

Prefer low-loss reduction before semantic compaction

A safer degradation ladder is:

1. Persist oversized raw artefacts
2. Replace old, reproducible tool payloads with references
3. Remove low-value middle transcript without breaking protocol pairs
4. Create a semantic summary under a summary contract
5. Perform bounded emergency compaction
6. Stop safely when further compression cannot preserve required state

Order matters. Clearing a large result before persisting the only evidence it contains makes recovery impossible.

No cutting operation should separate:

a tool request and its correlated result
an approval request and decision
background acceptance and its work identifier
a side-effect operation and receipt
a task claim and ownership record

A full transcript may be retained in durable storage, but retained does not mean model-accessible. A later session still needs an artefact tool, retrieval path, or explicit reference to recover the detail.

Semantic compaction should have a bounded retry policy. Continuing to summarise after required state can no longer be preserved only spends more money while increasing loss.

Where RAG belongs in the harness

RAG is a knowledge-acquisition subsystem governed by the harness. It is not the whole harness, and it need not share a process with the harness runtime.

Harness and RAG responsibility boundary

Figure 4-5 | RAG plans and executes retrieval, returning candidate evidence with provenance. The harness still owns scope, policy, context assembly, fallback, completion, and tracing.

RAG commonly handles:

query planning
rewriting and decomposition
corpus or index routing
metadata-filter construction
retrieval
reranking
deduplication
candidate-evidence packaging

The harness still decides:

whether this turn needs retrieval
which corpora, tenants, and fields the actor may query
the retrieval profile, model, and budget
which scope and filters the planner may use
how to handle empty or weak retrieval
whether SQL, web, code search, or user clarification is permitted
how candidate evidence is qualified and inserted into context
whether a claim and citation are supported
when to retry, degrade, ask, or stop
how trace, cost, and evaluation evidence are retained

Therefore:

Good Retrieval ≠ Grounded Answer ≠ Good Production Harness

A retriever may find highly similar material while the system still produces an untrustworthy result through incorrect permission, stale content, broken citation mapping, or a weak stopping policy.

What should candidate evidence carry?

RAG should not return only plain text. A useful result retains:

source identifier and URI
source version or timestamp
tenant and scope
chunk location
retrieval query and profile
rank or score as a retrieval signal
content digest
data classification
freshness or validity metadata

A score is a ranking signal, not a truth score. The context builder may select evidence under policy and budget. Claim verification remains a separate step after an answer or action has been produced.

Query planning answers how to search

Query planning may:

rewrite an ambiguous query
split it into subqueries
route to different indexes
add metadata filters
run multi-step retrieval
merge and deduplicate results
decide that the retrieval plan has gathered enough evidence

That last decision stops retrieval, not the whole task.

Orchestration still decides whether to use SQL, ask the user, enter verification, or stop the run.

Do not confuse the context builder with synthetics

Area	Context builder	Synthetics
Timing	Every runtime turn	Before or around retrieval, training, or evaluation
Purpose	Assemble the context needed now	Generate auxiliary queries, documents, QA, or training data
Typical output	Messages, input items, section manifest	HyDE document, synthetic query, QA pair, hard negative
Main risk	Pollution, excess length, staleness, cross-scope data	Bias, hallucination, evaluation leakage

HyDE and synthetic queries may improve a retrieval problem. They are not memory and do not replace the context builder.

Derived-context caches also become stale

Context builders frequently produce expensive derived material:

repository maps
schema summaries
recent-commit digests
documentation indexes
tool-catalogue summaries
memory indexes

A cache hit reduces latency and cost. If the source changes without invalidation, the agent continues reasoning about an obsolete world.

A cached block should record:

tenant and workspace scope
source reference
dependency versions or generations
policy version
tool-catalogue version
memory-index generation
generation time
content digest

For example:

repository-map-v21
├── tenant: tenant_acme
├── workspace_ref: a81c4f2
├── file_tree_generation: 94
├── policy_version: policy-8
├── tool_catalog_version: tools-31
└── generated_at: 2026-07-08T10:30:00Z

Common mutation points include:

file writes, renames, and deletion
Git checkout, merge, and rebase
schema migrations
tool registration and plugin enablement
permission and policy changes
memory updates
dependency or environment changes

A mutation should emit an invalidation event. If dependencies cannot be tracked reliably, use a short TTL or do not cache.

Testing only that the second request hits the cache proves little. A correctness test is:

Build cache
→ mutate a declared dependency
→ verify invalidation event
→ rebuild context
→ confirm the old source version is absent

Provider stable-prefix caching, application section-assembly caching, and derived-data caching are separate layers. They should not share a vague cached=true state or one invalidation rule.

The fresh-session test: can a new session reconstruct the work?

A direct way to test context artefacts is to start a fresh session, provide no private verbal briefing, and ask the agent to reconstruct the work through the standard entry points.

The test should resemble real execution:

use production-like instruction precedence
use the standard tools and permission profile
begin from a clean checkout or known checkpoint
provide no tester-only hints
require citations to the artefacts used

The new session should be able to answer:

What is this system?
Where are the main modules and authority boundaries?
How is it installed, started, stopped, and checked?
What work is active, complete, blocked, or cancelled?
What is the next safe action?
Which operations are prohibited or require approval?
Which artefacts support those conclusions?

Record:

discovery time
invalid tool calls
false assumptions
human interventions
stale documents encountered
missing canonical artefacts

Failures can be classified as:

missing: required information does not exist
hidden: it exists but is not discoverable through the standard path
stale: artefact and system have diverged
ambiguous: several sources make conflicting authority claims
inexecutable: a documented instruction cannot be run
unauthorised: recovery requires permissions that should not be exposed

OpenAI’s Harness Engineering article describes repository knowledge as a system of record and agent legibility as an engineering goal. This does not require all information to live in the repository. It requires authoritative information to be discoverable, interpretable, verifiable, and traceable from the agent’s environment.⁴

Repeated failure should not be answered by adding more material to AGENTS.md. The repair may be a better index, an executable command, a corrected progress ledger, explicit ownership, or deletion of contradictory documentation.

Context needs gardening and an admission gate

Once agents generate specifications, summaries, memory, and documentation at scale, the problem changes from missing information to low-signal growth.

Before material enters core knowledge, ask:

What decision value does it add?
Does it duplicate or conflict with existing material?
What are its source authority and temporal scope?
Who maintains it, and which mutations make it stale?
Does it belong in core, reference, watchlist, or rejected/expired material?

Agents can collect, summarise, deduplicate, find broken links, and propose placement. Admission to canonical knowledge remains a governance decision.

A maintainable gardening loop is:

Scan
→ classify stale / orphan / conflict
→ open a bounded repair
→ verify against code or authority
→ update indexes
→ supersede, archive, or delete safely

A knowledge base that adds a document after every failure but never repairs or retires an old one will eventually offer several plausible answers to the same question.

Walking through a payment-state inconsistency

Continue the payment example. A provider has captured the payment, but the internal order remains pending.

A governed context flow might be:

The always-on map points to the checkout, payment, and order modules and their test entry points.
The task contract limits the work to state synchronisation, keeps the provider, and prohibits direct production-data changes.
The context builder reads workspace identity, run state, API contracts, and the latest test evidence.
RAG and code search find the webhook handler, state transition, and a previous incident while retaining source versions.
Policy applies tenant, corpus, and field scope before retrieval.
The context builder labels retrieved text as data so that it cannot override system instructions.
The agent reads only the required files and logs rather than loading the whole repository.
A large raw log is persisted as an artefact; only relevant lines, time range, digest, and artefact identifier enter context.
The agent proposes a patch and tests produce new runtime evidence.
The next context gives priority to the diff, failing tests, original acceptance criteria, and remaining work.
A webhook-schema change invalidates the old schema summary and dependent context caches.
Temporary search payloads are cleared; evidence follows retention policy; a confirmed design decision becomes an ADR rather than an unsourced memory.

An apparently complete context can still be wrong. Without step eleven, the next inference uses obsolete fields and reasons coherently about the wrong version of the system.

Ten questions for reviewing a context system

Who owns the context builder, and which state and policy drive its choices?
Are model context, runtime state, durable artefacts, memory, and retrieved evidence separate?
Does every section carry role, authority, scope, source version, digest, and inclusion reason?
Is retrieved or tool content treated as data rather than silently promoted into instruction?
Is the always-on map navigational, and is task-scoped context reselected for each task?
Is large raw data persisted or processed outside the prompt before a compact view is loaded?
Do compaction, clearing, and memory address measured bottlenecks in a low-loss-first order?
Does canonical progress remain in run state or a progress ledger rather than a memory summary?
Does the harness govern RAG permission, freshness, fallback, provenance, and citation verification?
Can cache-invalidation tests and a fresh-session test expose stale, hidden, ambiguous, or inexecutable context?

Good context engineering does not require the model to know everything. It gives the model enough trustworthy and actionable information at the right time, together with stable identifiers for reaching the next layer of evidence.

From theory to implementation: Part 04 checklist

Part 05 examines the capability surface that the context builder exposes to the model: tools, skills, hooks, plugins, and MCP, and why adding more tools does not necessarily make an agent more capable.

References

Anthropic, Effective context engineering for AI agents, 29 September 2025. ↩ ↩²
Google Developers Blog, Conductor: Introducing context-driven development for Gemini CLI, 17 December 2025. ↩
Anthropic, Context engineering: memory, compaction, and tool clearing, 20 March 2026. ↩
OpenAI, Harness engineering: leveraging Codex in an agent-first world, 11 February 2026. ↩