Customer-service agents that call tools across multiple turns fail in two predictable ways. They retrieve the right facts but ground decisions in stale or missing information. They execute syntactically valid tool calls that violate domain policies because the agent lost track of constraints observed three turns ago.
Standard tool-calling agents stuff everything into the prompt: conversation history, tool returns, policy instructions. The agent reconstructs task state from scratch every time it decides what to do next. This implicit state management breaks when the context window fills, when the model skips over a constraint buried in turn four, or when a tool output introduces a new rule mid-conversation.
LedgerAgent (arXiv 2606.20529v1) separates task state from the prompt. It maintains a structured ledger of facts, identifiers, constraints, and conditions observed through user interaction and tool calls. The ledger is rendered into the prompt at decision time and used to check state-dependent policy constraints before environment-changing tool calls execute.
Why Conversation History Is Not Enough
Conversation history is a linear log. A ledger is a normalized database. When a user says “I want to cancel my subscription,” the agent might call get_subscription(user_id) and learn the subscription is in a grace period. That grace period is a constraint. If the user later asks to downgrade instead of cancel, the agent must remember the grace period applies to both actions.
In a prompt-only design, the grace period lives in the tool return from turn two. By turn six, it may be outside the truncation window or the model may not re-extract it. In a ledger design, the grace period is written to a constraints field and checked before every policy-sensitive tool call.
State Components
LedgerAgent tracks four categories:
- Facts: user attributes, account balances, subscription tier.
- Identifiers: user ID, order ID, session token.
- Constraints: grace periods, approval requirements, rate limits.
- Conditions: observed states like “user verified identity” or “payment method on file.”
Each category is updated when a tool returns new information or the user provides a new fact. The ledger is versioned per turn so you can audit which state the agent saw when it made a decision.
Architecture
LedgerAgent runs at inference time. It wraps a standard tool-calling agent with two new components: a ledger manager and a policy checker.
Ledger Manager
The ledger manager parses tool outputs and user messages to extract state updates. When get_subscription returns {"status": "grace_period", "days_remaining": 5}, the manager writes:
{
"facts": {
"subscription_status": "grace_period"
},
"constraints": {
"grace_period_days": 5
}
}The manager also handles conflicts. If a later tool call returns {"subscription_status": "active"}, the ledger overwrites the stale fact. Constraints can have expiration logic: a grace period that counts down or a rate limit that resets hourly.
Policy Checker
Before executing an environment-changing tool call (cancel, refund, downgrade), the policy checker evaluates domain rules against the current ledger state. Policies are expressed as predicates:
def can_cancel_subscription(ledger):
if ledger.constraints.get("grace_period_days", 0) > 0:
return False, "Cannot cancel during grace period"
if not ledger.conditions.get("identity_verified"):
return False, "Identity verification required"
return True, NoneIf the check fails, the tool call is blocked and the agent receives an error message explaining the violation. The agent can then call a different tool (like verify_identity) or ask the user for more information.
Prompt Rendering
At each turn, the ledger is serialized into a structured block and injected into the prompt:
Current Task State:
Facts:
- subscription_status: grace_period
- account_balance: $42.00
Identifiers:
- user_id: usr_12345
Constraints:
- grace_period_days: 5
Conditions:
- identity_verified: falseThis gives the model a clean, scannable view of what it knows and what rules apply. The agent can reference ledger fields by name instead of searching through conversation history.
Failure Modes and Trade-Offs
| Risk | Mitigation | Cost |
|---|---|---|
| Ledger extraction errors | Schema validation on tool outputs, fallback to prompt-only mode | Extra parsing latency per tool call |
| Policy rule conflicts | Explicit priority ordering, audit log of blocked calls | Manual rule curation |
| Ledger bloat in long conversations | TTL on facts, constraint expiration logic | State may be pruned too aggressively |
| Model ignores ledger block | Few-shot examples showing ledger usage, prompt engineering | Token overhead per turn |
The biggest operational risk is ledger drift. If a tool returns unstructured text instead of JSON, the ledger manager may fail to extract state. If the agent hallucinates a fact that never appeared in a tool output, the ledger won’t catch it unless you add a verification step.
Observability and Compliance
LedgerAgent produces three audit artifacts per conversation:
- Ledger snapshots: versioned state at each turn.
- Policy check logs: which rules were evaluated, which passed, which blocked a call.
- Tool call trace: mapping from ledger state to tool invocation.
For regulated workflows (finance, healthcare, insurance), these artifacts satisfy explainability requirements. You can answer “Why did the agent deny this refund?” by pointing to the ledger snapshot and the policy rule that failed.
Serialization format matters. JSON is easy to parse but verbose. Protocol Buffers are compact but require a schema registry. The paper does not specify a format, but production systems will need versioned schemas and backward compatibility guarantees.
Deployment Shape
LedgerAgent adds two network hops per tool call: one to update the ledger, one to check policies. In a synchronous request-response flow, this increases latency by 20-50ms depending on where the ledger and policy engine live.
Three deployment options:
- In-process: ledger and policy checker run in the same process as the agent. Lowest latency, no network calls, but state is lost if the process crashes.
- Sidecar: ledger manager runs as a sidecar container. State persists across agent restarts, adds 5-10ms per call.
- Remote service: ledger and policy engine are separate services. Highest latency (20-50ms), but you can share state across multiple agent instances and enforce policies centrally.
For customer-service agents handling 100+ concurrent conversations, the remote service model makes sense. You can cache ledger state in Redis and run policy checks in a stateless Lambda.
When to Use This
LedgerAgent is overkill for single-turn agents or agents that never call environment-changing tools. It makes sense when:
- Conversations span 5+ turns with multiple tool calls.
- Domain policies depend on facts observed earlier in the conversation.
- You need audit trails for compliance or debugging.
- Tool outputs introduce new constraints that affect future actions.
Avoid it when:
- Your agent only calls read-only tools (search, lookup, summarize).
- Policies are simple enough to encode in the tool schema itself.
- You can afford to re-parse the entire conversation history at every turn.
- Your domain has no regulatory requirements for explainability.
Technical Verdict
LedgerAgent solves a real production problem: agents that follow instructions but break business rules because they lost track of state. The ledger abstraction is clean and the policy checker prevents a class of errors that prompt engineering cannot fix.
The trade-off is operational complexity. You now have three moving parts (agent, ledger, policy engine) instead of one. You need schema validation, versioning, and conflict resolution logic. You need to decide where state lives and how long it persists.
For customer-service agents in regulated industries, the complexity is worth it. For experimental side projects or low-stakes automation, stick with prompt-only designs until you hit a policy violation in production.