Harness Theory Part 03 | Agent Loops and Control Flow: How Does the Model Start Doing Work?

A structured tool call means that the model has proposed an operation. It does not mean that the operation has happened.

The system still has to determine whether the tool exists, whether the arguments can be interpreted safely, whether the current identity may perform the action, where the tool will run, how the outcome is paired with the request, whether failure should trigger another attempt, and whether a model that stops calling tools has completed the task or merely become blocked.

Those responsibilities form the agent loop.

The minimal flow looks short:

Input → Model → Tool Request → Tool Result → Model → Completion

Most of the engineering sits inside the arrows. A loop can work in a demonstration while lacking the authority, state transitions, and protocol contracts required to pause, resume, cancel, or prove completion.

OpenAI describes the agent loop as the core logic that coordinates interaction between the user, model, and tools. Anthropic describes tool use as a contract between the application and the model: the model emits a structured request, application or server code performs the operation, and the result flows back into the conversation. Producing JSON does not perform the external action.¹²

Agent Loop State Machine

Figure 3-1 | A reliable agent loop is a governed state machine. The model proposes actions; the harness parses, authorises, executes, records, and terminates them explicitly.

Tool calling supplies an interface; the agent loop supplies control

Tool calling normally provides two things:

a description of the available operations and their input schemas
a structured way for the model to request one of those operations

For an order lookup, the model might return:

{
 "tool_name": "get_order",
 "arguments": {
 "order_id": "ORD-1042"
 }
}

That JSON does not establish that:

ORD-1042 belongs to the current user or tenant
get_order is still the same enabled version that the model saw
the arguments have been parsed, canonicalised, and schema-checked
policy permits this operation
the tool completed successfully
the result contains no data from another tenant
the lookup satisfies the original task contract

The model usually owns the proposal. The harness turns that proposal into a controlled state transition by parsing the actual response items, fixing the tool and arguments, authorising the operation, executing it, recording the result, and deciding what happens next.

The distinction also applies to server-executed tools. A model provider may run the tool, but the product still needs a clear account of which execution controls and evidence the provider supplies, and which identity, tenant, task-contract, and completion responsibilities remain with the product harness.

Eight stages in a controlled agent loop

Frameworks use different class and function names, but the responsibilities can be arranged into eight stages:

Stage	Harness responsibility	Main output
1. Assemble context	Build the current model input from task contract, run state, policy, and visible capabilities	Bounded model context
2. Infer next action	Invoke the model and receive text, tool requests, plan updates, questions, or waiting proposals	Typed response items
3. Parse and canonicalise	Inspect actual blocks, resolve tool version, and fix arguments and resource identity	Canonical proposal
4. Authorise transition	Apply actor, tenant, scope, policy, risk, and state version	Grant, denial, or approval required
5. Execute in boundary	Run the capability under sandbox, timeout, cancellation, and other execution controls	Typed execution outcome
6. Commit correlated observation	Record request, result, operation identifier, evidence, and state transition together	Updated canonical run state
7. Evaluate terminal candidate	Check contract, evidence, pending work, and completion gates when no executable request remains	Continue, wait, accept, or reject
8. Continue, wait, or terminate	Start another iteration, pause, repair, cancel, fail, or complete with an explicit reason	Explicit run status and reason

This article places each responsibility without expanding all of its internals. Context construction belongs in Part 04, the tool-execution lifecycle in Part 05, and durable state and checkpointing in Part 06.

Why parsing and authorisation are separate

The harness must first convert model output into one unambiguous operation. Only then can policy decide what it is approving.

Suppose the model requests a write to:

./exports/../payments/config.json

A policy check performed against the raw string may approve a different resource from the path later resolved by the file API. A safer responsibility order is:

Parse structure
→ Resolve tool and version
→ Canonicalise arguments and resources
→ Authorise the canonical operation
→ Apply pre-execution gates
→ Execute

This does not require every business validation to occur before authorisation. It requires policy and execution to refer to the same canonical operation.

Dynamic tool catalogues also need version coherence. If the model saw deploy_service@v2 and the runtime silently resolves the request to a semantically different v3, the proposal is stale. It should not be upgraded invisibly.

Execution outcomes need more than success and error

A useful protocol should distinguish at least:

success
validation_failed
permission_denied
approval_required
timeout
cancelled
background_accepted
unknown_outcome
unknown_or_disabled_tool

These are not stylistic variants of an error message. They lead the controller to different state transitions.

A timeout, for example, does not prove that an external side effect did not happen. A payment API may have accepted the operation while its response was lost. The next step should reconcile the operation identifier before issuing another request. Idempotency and recovery are covered in Part 08.

What can a small `while` loop establish?

The following is architecture-level pseudocode:

while budget.allows(run):
 context = context_builder.build(run)
 response = model_adapter.generate(
 context=context,
 tools=capability_view.for_run(run),
 )

 items = protocol.parse_items(response)
 requests = protocol.executable_requests(items)

 if requests:
 for request in execution_policy.order(requests, run):
 operation = capability_registry.canonicalise(request, run)
 decision = policy.authorise(operation, run)

 if decision.requires_approval:
 result = approval_gateway.pause_for_decision(operation, run)
 elif decision.denied:
 result = tool_results.denied(operation, decision)
 else:
 result = capability_runtime.execute(operation, decision.grant)

 run = run_store.commit_correlated_result(
 run=run,
 request=request,
 operation=operation,
 result=result,
 )
 continue

 decision = completion_gate.evaluate(
 terminal_candidate=items,
 run=run,
 )
 run = run_store.commit_decision(run, decision)

 if decision.accepted:
 return delivery.from_committed_run(run)
 if decision.waiting:
 return delivery.waiting_receipt(run)

 run = controller.repair_replan_or_stop(run, decision)

raise BudgetExceeded(run.stop_reason)

It shows that:

the model requests and the harness authorises and executes
tool resolution, canonicalisation, and policy decisions are distinct
each request is committed with a correlated terminal result
an ordinary assistant message is only a terminal candidate
completion, waiting, repair, and budget exhaustion are separate transitions
delivery comes from committed run state

It does not prove that:

authentication, tenant isolation, and credential binding are correct
several tool requests can be executed safely in parallel
side effects have idempotency, compensation, and unknown-outcome reconciliation
distributed locking, worker crashes, and checkpoint recovery are handled
approval expiry, reconnection, and protocol versioning are complete
the completion gate or a model-based evaluator is reliable
the example is production-ready

A loop that runs proves that control can circulate. It does not supply safety, reliability, or quality by itself.

Four invariants the loop must preserve

Invariant 1: use actual response items and execution facts

A model provider may expose stop_reason, finish status, or another convenient field. Those fields belong to the provider contract, but they should not replace inspection of the actual response items.

The harness must avoid two failures:

a response still contains a tool request, partial item, or pending control item, but the controller terminates because of a stop signal
a tool outcome is unknown, but the transcript records success because that was the expected path

A control system must not overwrite what happened with what was expected to happen. If provider signals, typed content, and runtime evidence disagree, the run should enter a diagnosable protocol-error or reconciliation path.

Invariant 2: one tool request has one terminal result

One result means one terminal disposition at the protocol layer. Execution may produce many progress events, log deltas, and heartbeats, but each tool request should end with one correlated terminal result, such as:

success
denied
validation failed
timeout
cancelled
background accepted
unknown tool
unknown outcome

Tool Request and Result Integrity

Figure 3-2 | A request may produce many progress events, but only one terminal result. Background completion uses a separate work identifier and completion event.

Background work is where this invariant is most often damaged:

The original tool request receives a background_accepted result.
The result creates a separate work_id.
The worker emits progress events against that work identifier.
A later completion event records the background outcome.
The completion event does not masquerade as a second result for the original request.

When one model response contains several tool requests, each needs its own identifier and result. Sequential or parallel execution must be an explicit execution-policy decision rather than an assumption that the requests are independent.

Parallel execution needs answers to at least four questions:

Are the operations independent?
Does result ordering affect the next model context?
Do they compete for the same workspace or resource?
If one fails, should the others continue, cancel, or compensate?

Compaction, replay, and context assembly must preserve request/result pairs. Missing or duplicate terminal results cause the model, debugger, and evaluator to reconstruct different histories.

Invariant 3: runtime state is larger than the transcript

A transcript is useful for model-visible conversation and items. It is not a sufficient store for:

approval grants, denials, and expiry
tool request/result receipts
current state version
task ownership
active background work
cancellation state
budget use
workspace identity
side-effect operation identifiers
scheduled triggers
last acknowledged event sequence

A tool may have submitted a payment successfully while its response was lost. The transcript may show a timeout, but the runtime cannot infer that no payment happened.

Part 06 covers run state, checkpoints, and persistence. The boundary required here is simple: model-visible history is not the complete system state.

Invariant 4: continue, wait, and terminate require explicit reasons

The absence of a tool call produces a terminal candidate, not proof of success.

The controller must still consider:

task contract and required evidence
pending approvals, background work, and external events
maximum steps, tool calls, retries, tokens, cost, and deadline
human cancellation
non-recoverable failure
policy stop
loop detection
completion-gate acceptance or rejection

An approval decision is not automatically terminal. Allowing may resume the operation. Denial may prompt another safe plan, wait for a contract change, or end the run.

Every pause and termination should record an explicit reason. These states are not interchangeable versions of done:

completed
waiting_for_approval
waiting_for_external_event
cancelled
budget_exhausted
policy_stopped
failed

Workflow, agent, orchestration, and query planning own different decisions

These terms are frequently collapsed until every piece of control logic is called an agent.

Workflow, Agent, Orchestration and Query Planning

Figure 3-3 | Orchestration coordinates work and resources; workflow and agent are control patterns; query planning owns retrieval decisions inside RAG.

Concept	Main decision	Common characteristic
Workflow	Steps and transitions in a known path	Predictable, testable, suitable for stable processes
Agent	The next action and tool inside an allowed boundary	Flexible path with a higher control burden
Orchestration	Who acts, which resources are assigned, and where hand-off or failure goes	Routing, scheduling, ownership, stopping
Query planning	What to retrieve, how to rewrite or decompose, and which index to use	Retrieval decisions inside a RAG subsystem

Anthropic defines workflows as systems in which models and tools follow predefined code paths, and agents as systems in which the model dynamically directs process and tool use. It also recommends starting with the simplest arrangement that meets the need, because agentic systems exchange additional cost and latency for flexibility.³

Orchestration often coordinates workflows or agents, but it does not have to be a separate outer service that literally contains them. It is a set of cross-step, resource, and ownership responsibilities. Those responsibilities may live in the same process as the controller or in a higher-level system.

A useful design principle is deterministic first.

Decisions that usually belong in code include:

authentication and tenant scope
hard policy
schema and protocol validation
required verification sequences
payment, deployment, and deletion gates
budget accounting

Model judgement is more useful for:

selecting among already authorised tools
revising a plan from observations
exploration whose order cannot be enumerated in advance
classification, rewriting, and decomposition that require semantic judgement

Moving every fixed transition into the model does not create intelligence. It replaces testable programme logic with a more expensive and less stable decision.

Query planning also needs a firm boundary. Decomposing a question, selecting an index, and combining evidence remain retrieval decisions. They do not own task workspace, approval, tool execution, completion authority, or recovery.

Item, turn, and thread: a persistent interaction lifecycle

A chat API encourages a one-request, one-response mental model. Agent work rarely has that shape.

A request to run tests and repair failures may produce:

a user message
a plan update
a tool request
an approval request
tool execution
streamed logs
a diff
a verification result
a final agent message

The Codex App Server models this interaction through item, turn, and thread primitives and uses a bidirectional JSON-RPC protocol for events, approvals, and client integration. These names are a concrete Codex design rather than a standard every product must copy. The general requirements are typed events, nested lifecycles, stable identifiers, server-side state, and reconnection.⁴

Thread, Turn and Item Lifecycle

Figure 3-4 | A thread retains several turns, and each turn contains ordered typed items. An operation requiring approval pauses before execution and follows the client decision.

Item: the smallest typed unit

An item may represent:

user message
agent message
plan update
tool request
approval request
tool execution
tool result
diff
artefact reference

A typical lifecycle is:

item/started
→ optional item/delta or progress
→ item/completed | item/failed | item/cancelled

Different item types need not share identical states. A message may have streaming deltas, while an approval request needs pending, resolved, and expired semantics. The protocol should preserve type-specific payloads instead of compressing every event into prose.

Turn: one pausable unit of agent work

A turn begins with a user input or external trigger and may contain many model inferences, tool requests, and results.

turn/started
→ ordered typed items
→ turn/completed
 | turn/waiting
 | turn/failed
 | turn/cancelled

A turn is not one model API call. It may cycle between the model and tools several times or pause for an approval or external event.

Thread: a durable container across turns

A thread contains several turns and may be created, resumed, forked, archived, reconnected, or replayed.

Thread history should allow a new client session to obtain a snapshot, missing events, active-turn status, and pending approvals. Closing a browser removes a view, not the runtime state.

Streaming, approval, and reconnection require bidirectional control

An agent UI does more than stream model tokens.

The server may push:

item lifecycle events
tool progress
log deltas
artefact references
budget warnings
verification results
turn state changes

The server may also initiate an approval request:

server: approval/requested
 approval_id = apr_31
 operation_id = op_88
 state_version = 14
 reason = "Deploy to production"
 expires_at = ...

client: approval/resolved
 approval_id = apr_31
 decision = deny

The correct order is:

Proposed Operation
→ Approval Required
→ Turn enters waiting
→ Client allow / deny / expiry
→ Allow: execute the authorised operation
→ Deny or expiry: emit a terminal denied result, then replan or stop

Tool execution must not appear to start before approval. Approval should also bind to more than a vague instruction to “let the agent continue”. It needs at least:

approval identifier
operation identifier
canonical arguments
actor and scope
policy or state version
reason
expiry
decision

State version matters because the workspace, arguments, or policy may change while the request is awaiting approval. A stale approval should not silently authorise a different operation.

Reconnection is also part of control flow. The client supplies its last acknowledged sequence; the server returns a thread snapshot, missing event replay, active-turn status, and pending approvals. The client should not resend the whole conversation and ask the server to infer the current execution state. The OpenAI App Server similarly keeps thread state and event history on the server so that different clients can render a consistent timeline.⁴

Stopping policy: agents need legitimate waiting states as well as brakes

An agent loop should begin with a budget and stopping policy.

Limit	Failure it contains	Possible transition
Maximum steps or turns	Unbounded planning and repeated reasoning	Replan, escalate, or `budget_exhausted`
Maximum tool calls	Repeated exploration or a tool storm	Stop, narrow capabilities, or human review
Retry budget	Endless repetition of the same failure	Fallback, reconcile, or fail
Token budget	Context and output growth	Compact, degrade, or stop
Cost budget	Spend beyond authority	Approval or termination
Wall-clock deadline	Long-lived worker occupation	Checkpoint, cancel, or reassign
Loop detection	Repeated action without new evidence	Replan, rollback, or escalate
Human cancellation	Withdrawal of the task	Cancel pending work and persist
Policy stop	Entry into a prohibited area	Deny, wait, or terminate

The runtime must count and enforce these limits. Writing them into the prompt is not sufficient.

Waiting for human approval, a webhook, background work, or a scheduled time is not a failure. The system should persist a waiting reason and resume trigger, release unnecessary resources, and avoid a busy polling loop that continues consuming model tokens.

Goal drift: distinguish drift from an authorised change

A long task accumulates observations, errors, temporary repairs, and side paths. The agent can remain productive while moving away from the task contract.

Common forms include:

goal forgetting: the original acceptance criteria disappear inside a long context
goal substitution: building a tool, refactoring a module, or writing a report becomes the final objective
scope creep: the agent adds improvements that were never authorised
repeated loop: the same tool is called without new evidence
local optimisation: one metric improves while the overall task becomes worse

Not every goal change is drift. A user may legitimately alter scope or acceptance criteria. That should create a new contract version, state transition, and any required re-authorisation instead of silently replacing the old objective.

Text similarity alone is a weak detector. More useful signals include:

outstanding contract conditions
whether milestones and evidence are increasing
repeated tool calls and failure patterns
cost, time, step, and retry consumption
whether new work lies inside the allowed scope
repeated gaps from the completion gate
human or monitoring interrupts

Correction can escalate in stages:

Re-present the goal, scope, and acceptance criteria.
Produce a progress and gap summary with the smallest next step.
Replan while retaining verified evidence.
Return to a safe checkpoint.
Narrow tools, data, or budget.
Pause for a human decision to continue, amend the contract, or stop.

Asking the model to reflect may provide another signal, but reflection remains model output. It does not replace external state, budgets, scope gates, or completion evidence.

Walking through a test-and-repair task

Suppose the user asks a coding agent:

Run the checkout service tests, repair the current failures, and do not modify the payment-provider integration.

A controlled loop might proceed as follows:

Start a turn with the task contract, allowed scope, and budget.
Build context with checkout-service guidance and the capabilities visible for this run.
The model proposes running the specified test suite.
The harness canonicalises the command and workspace, applies policy, and executes it.
The command result reports two failures; the full log is retained as an artefact.
The model reads the relevant files and proposes a patch.
The write operation passes scope checking and leaves a diff and operation receipt.
The model requests another test run.
Unit tests pass, but the completion gate detects that the payment-integration contract test has not run.
The terminal candidate is rejected and the loop returns to the required verification step.
The contract test passes and a scope check confirms that the provider integration was untouched.
The run commits completed and delivers the diff, test evidence, and terminal reason.

If the model writes “fixed” at step nine, that is prose. Completion still depends on the task contract, policy, and evidence.

The boundary of a minimally reliable agent loop

At minimum, the loop needs:

typed input, output, and control items
explicit action authority
a canonical operation and tool version
tool-request and terminal-result correlation
explicit state transitions
budget, cancellation, and waiting semantics
terminal-candidate evaluation
reconstructable event history

The loop does not solve the whole harness:

context selection, compaction, and memory governance belong in Part 04
tool schemas, skills, hooks, plugins, and MCP belong in Part 05
thread persistence, checkpoints, and runtime lifecycle belong in Part 06
multi-agent delegation and long-running hand-off belong in Part 07
retries, unknown outcomes, and side-effect recovery belong in Part 08
permissions, sandboxing, and trust boundaries belong in Part 09
completion contracts, verifiers, and evaluation belong in Part 10
traces, replay, metrics, and SLOs belong in Part 11

The agent loop is the control engine, not the whole vehicle.

Ten questions for reviewing an agent loop

Does the controller parse typed response items or rely only on the provider stop signal?
Are the tool, arguments, and resource resolved into a canonical operation before authorisation?
Does every tool request receive one correlated terminal result?
Do progress, background completion, and tool results use distinct event semantics?
Is the sequential or parallel policy for multiple tool requests explicit?
Is the transcript separate from canonical runtime state?
Is an assistant message treated as completion or as a terminal candidate?
Are completed, waiting, failed, cancelled, and budget-exhausted states distinct?
Do approval, cancellation, reconnection, and replay have a bidirectional protocol?
Can each transition be reconstructed from event history?

An implementation containing only:

while response.has_tool_call:
 response = call_model(run_tool(response.tool_call))

is a reasonable demonstration starting point. A recoverable, auditable, and governable agent loop needs considerably more of the arrows.

From theory to implementation: Part 03 checklist

Part 04 turns to the hardest input problem before every inference: what belongs in a limited context window, what must remain in durable state, what should be retrieved through RAG, and why memory cannot be an ever-growing chat summary.

References

OpenAI, Unrolling the Codex agent loop, 23 January 2026. ↩
Anthropic, How tool use works, accessed 8 July 2026. ↩
Anthropic, Building effective agents, 19 December 2024. ↩
OpenAI, Unlocking the Codex harness: how we built the App Server, 4 February 2026. ↩ ↩²