mech.app
Dev Tools

Honcho: Why Stateful Agents Need a Memory Layer That Reasons in the Background

How Honcho separates memory storage from inference, enabling agents to build evolving user representations without blocking the main loop.

Source: github.com
Honcho: Why Stateful Agents Need a Memory Layer That Reasons in the Background

Most agent memory systems treat storage and reasoning as the same operation. You write a message, embed it, and retrieve similar chunks later. Honcho splits this into two phases: a write path that stores raw events and a background reasoning loop that builds queryable representations of users, sessions, and context without blocking your agent.

This matters when you need agents that remember evolving preferences, track multi-turn goals, or surface insights across sessions. Vector search alone cannot answer “What does this user care about right now?” or “How has this project’s scope changed over the last week?” Honcho runs inference asynchronously on stored events to produce those answers.

The project hit rank 11 on GitHub Trending for Python with 145 score and claims Pareto frontier performance on agent memory benchmarks. It offers both a managed API at api.honcho.dev and a self-hosted FastAPI server.

The Storage vs. Inference Split

Honcho’s architecture separates three concerns:

  1. Event storage: Messages, tool calls, and custom events land in a persistent store.
  2. Background reasoning: A separate process runs model inference on stored events to update user representations, session summaries, or project state.
  3. Query interface: Your agent reads the reasoned output (not raw events) to make decisions.

This design avoids two common failure modes. First, it prevents blocking the agent loop while you wait for embeddings or LLM calls to synthesize memory. Second, it lets you update memory representations without re-running the entire agent conversation.

How Background Reasoning Works

When you store a message or event, Honcho does not immediately run inference. Instead:

  • Events append to an immutable log keyed by user, session, or project.
  • A background worker polls for new events and triggers reasoning tasks.
  • Reasoning tasks call a model (configurable, defaults to OpenAI) to generate or update a “peer representation” or “session context.”
  • The updated representation lands in a queryable store (Postgres by default).

Your agent queries the representation store, not the raw event log. This means you can ask “What are the user’s current goals?” and get a natural-language summary, not a list of embeddings.

API Boundary

The SDK exposes two surfaces:

Write path:

honcho.apps.users.sessions.messages.create(
    app_id="my-app",
    user_id="user-123",
    session_id="session-456",
    content="I want to automate my invoicing workflow",
    is_user=True
)

Read path:

context = honcho.apps.users.sessions.get_context(
    app_id="my-app",
    user_id="user-123",
    session_id="session-456"
)
# Returns: "User is focused on invoice automation. Prefers Python. Mentioned Stripe integration."

The read path does not hit the event log. It reads the output of the background reasoning loop.

Schema Evolution and Versioning

Honcho stores events in an append-only log, so you never lose raw data. Representations (the output of reasoning) live in a separate table with a version field. When you change the reasoning prompt or model, Honcho can:

  • Reprocess old events to generate a new representation version.
  • Keep multiple representation versions active for A/B testing.
  • Roll back to a previous version if reasoning quality degrades.

This is critical for production agents. If you discover your summarization prompt is too verbose, you can reprocess the last 30 days of events without touching the agent code.

Deployment Shape

Honcho runs as a FastAPI server with three components:

ComponentRoleScaling Concern
API serverHandles SDK requests, writes eventsStateless, horizontal scale
Background workerPolls event log, runs reasoning tasksCPU/GPU bound, queue depth
PostgresStores events and representationsWrite throughput, index tuning

The managed API at api.honcho.dev handles all three. For self-hosting, you run the FastAPI server and a worker process (Celery or similar). The worker is where you tune concurrency and model choice.

Self-Hosting Configuration

Clone the repo and set environment variables:

export DATABASE_URL="postgresql://user:pass@localhost/honcho"
export OPENAI_API_KEY="sk-..."
export REASONING_MODEL="gpt-4o-mini"
export WORKER_CONCURRENCY=4

uvicorn honcho.main:app --host 0.0.0.0 --port 8000
celery -A honcho.worker worker --loglevel=info

The worker polls the event log every 10 seconds by default. You can tune WORKER_POLL_INTERVAL and REASONING_BATCH_SIZE to trade latency for throughput.

Observability and Failure Modes

Honcho exposes metrics at /metrics (Prometheus format):

  • honcho_events_written_total: Event ingestion rate.
  • honcho_reasoning_tasks_queued: Backlog of unprocessed events.
  • honcho_reasoning_duration_seconds: Model inference latency.

Common failure modes:

  1. Worker lag: If reasoning tasks pile up, your agent reads stale representations. Monitor reasoning_tasks_queued and scale workers.
  2. Model rate limits: The background worker can hit OpenAI rate limits. Use a local model or batch requests.
  3. Representation drift: If you change the reasoning prompt mid-flight, old and new representations coexist. Version them explicitly.

Integrations

Honcho integrates with Claude Code, OpenCode, and any agent framework that can call an HTTP API. The Python and TypeScript SDKs live in sdks/ and wrap the FastAPI endpoints.

For frameworks like LangChain or AutoGPT, you add Honcho as a memory backend:

from honcho import Honcho

honcho = Honcho(api_key="your-key")

# In your agent loop:
honcho.apps.users.sessions.messages.create(...)
context = honcho.apps.users.sessions.get_context(...)

The SDK handles retries and batching. You do not need to manage the event log or reasoning loop.

Benchmarks and Evals

Honcho claims Pareto frontier performance on agent memory benchmarks, comparing recall, latency, and cost against vector-only systems. The evals page shows:

  • Recall: Honcho surfaces relevant context 15-20% more often than naive RAG.
  • Latency: Background reasoning adds no latency to the agent loop (queries hit the representation store, not the model).
  • Cost: Amortizes model calls across many queries, reducing per-interaction cost by 3-5x.

The benchmark suite is open source. You can run it against your own memory system to compare.

When to Use Honcho

Use it when:

  • Your agent needs to remember evolving user preferences or project state.
  • You want to query “What does the user care about?” without re-embedding every message.
  • You need to update memory representations without re-running the agent.
  • You want to separate storage (cheap, durable) from reasoning (expensive, tunable).

Avoid it when:

  • Your agent is stateless or only needs single-turn context.
  • You already have a working memory system and cannot justify the migration cost.
  • You need sub-100ms query latency (background reasoning adds seconds to minutes of lag).
  • You want to avoid running a separate worker process.

Technical Verdict

Honcho solves a real problem: agents that need persistent, queryable memory without blocking on inference. The storage-vs-inference split is architecturally sound and maps cleanly to production concerns (scaling, versioning, observability). The managed API removes deployment complexity, and the self-hosted option gives you control over models and data residency.

The main trade-off is latency. Background reasoning means your agent reads slightly stale representations. If you need real-time memory updates, you will need to tune worker concurrency or run reasoning inline (which defeats the purpose). For most multi-turn agents, the latency is acceptable.

If you are building agents that need to remember users across sessions or track evolving goals, Honcho is worth evaluating. If your agent is stateless or only needs single-turn RAG, stick with a simpler vector store.