Your agent made the right call — and your system still broke. The user clicked "confirm refund", the page refreshed, the approval banner vanished, and nobody knows whether the refund should run. Worse: the request reached the payment gateway, the backend restarted before writing local state, and recovery re-triggered the refund. Twice.
That's the gap between "an agent can run one loop" and "an agent can run a product". ReAct gives you Thought → Action → Observation, but production throws page refreshes, process restarts, multi-hour approvals and lost responses at you. This post breaks down how to turn agent judgment and actions into recoverable, auditable runtime facts — built on vivo's KDC engineering series (links at the end).
The core idea: three kinds of facts, don't mix them
- Domain reality — what actually happened (the refund landed in the user's bank). Software only observes it indirectly through interfaces, events and confirmations.
- Business judgment facts — what the system concluded and why (order is eligible, policy requires confirmation). "Fact" here means the judgment was actually formed, not that it was correct.
- Runtime facts — what this run has done so far (run started, approval pending, tool dispatched, checkpoint at X).
tool.call.completed ≠ money arrived. pendingApproval = null ≠ the user agreed. Keep them separate and connect them with stable IDs: runId, turnId, reasoningObjectId, capabilityId, approvalId, toolCallId, feedbackId. That's how a runtime knows which approval belongs to which judgment, and which tool call came from which capability.
Pattern 1: State / View / Control
The classic failure is the frontend inferring run state from chat text ("the agent said it needs confirmation → render a button"). That works in a demo and falls apart on refresh. Instead:
- State — persisted, authoritative runtime facts:
activeRun,pendingApprovals,toolCalls,checkpoint. - View — display derived from State:
isBusy,approvalBanner,canStop. It can change with your UI, but it's never the source of truth. - Control — commands, not state writes:
resume(approvalId, decision),stop(runId),retry(toolCallId).
Rule of thumb: Runtime writes facts, View reads facts, Control submits commands. The UI never fabricates runtime facts.
Pattern 2: event-sourced runtime facts
One append-only log of strongly-typed events, with projections per concern:
{
"eventId": "event-109",
"eventType": "approval.required",
"runId": "run-20260727001",
"turnId": "turn-003",
"producer": "policy-runtime",
"reasoningObjectId": "reasoning-021",
"capabilityId": "refund-order-v2",
"correlationId": "refund-intent-031",
"causationId": "event-108",
"schemaVersion": 2,
"occurredAt": "2026-07-27T16:42:10+08:00",
"payload": { "approvalId": "approval-017", "risk": "high", "expiresAt": "2026-07-27T18:00:00+08:00" }
}Approval state must be reduced from approval events — never because the model said "the user will probably agree". The same log can derive runtime state, user views, audit records and eval traces; each projection keeps its own schema, owner and usage rights.
Pattern 3: idempotency keys and result_unknown
High-impact calls need a stable idempotency key. And when a response is lost, result_unknown is a distinct recovery boundary — not a fancy way of saying "failed":
{
"toolCalls": [{
"toolCallId": "tool-call-031",
"capabilityId": "refund-order-v2",
"status": "result_unknown",
"idempotencyKey": "refund:order-001:intent-031",
"externalRequestId": "payment-request-8841",
"dispatchedAt": "2026-07-27T16:48:12+08:00",
"acceptedAt": null,
"resultRef": null,
"reconciliationStatus": "pending",
"lastError": "response_timeout"
}]
}If the payment channel may have accepted the request, state must not fall back to "not called" — keep the same toolCallId and idempotencyKey. On recovery: query the external system by externalRequestId, reconcile, then decide complete / compensate / hand to a human. Never blindly retry back to authorized.
Pattern 4: resume contracts (checkpoint ≠ durable execution)
A checkpoint only says where to resume. It doesn't guarantee side effects happened exactly once, or that replay is safe. LangGraph's Interrupt replays the node from the top, so pre-interrupt side effects must be safe to re-run; Temporal replays the event history and confines side effects to Activities. A Resume Contract turns the rules into something the runtime can execute and test:
- Stable idempotency keys for high-impact calls
- Distinguish command-sent / external-accepted / business-confirmed
- On unknown results: query or reconcile before retrying
- Transactional messages or an outbox to close the gap between local state and event publishing
- Approvals can't be double-consumed; re-validate policy, permissions and preconditions on resume
Also track execution status and outcome-verification status separately (proposed → authorized → dispatching → accepted → completed/failed vs not_observed → pending → confirmed_success/failure/inconclusive) — a run can end while the outcome is still pending.
Practice advice
- Start with State / View / Control separation and move approvals out of chat text — the biggest win per unit of effort.
- One event log, typed events, projections per concern. "One storage" is fine; "guess from prose" is not.
- Decouple Session (append-only record), Harness (loop, context, control) and the execution sandbox — a crashed harness can rebuild from the Session, and high-privilege credentials never enter the model context.
- For multi-day tasks, add a structured Progress Contract (verified items, open items, next safe step) so a fresh agent doesn't overestimate progress, redo work or break what works.
- Runtime state is temporary: archive checkpoints, close approvals, revoke short-lived credentials. Promote only verified, scoped lessons into Memory/Knowledge — persistence ≠ memory.
Resources
- Original: 如何把 Agent 的判断与行动变成可恢复的软件事实 (KDC engineering supplement, InfoQ, in Chinese)
- Anthropic: Managed Agents — stateless harness, durable session, sandboxed execution
- Anthropic: Effective harnesses for long-running agents
- LangGraph persistence and interrupts
- Temporal: event history
- 12-Factor Agents: unify execution state
