Agents that remember across sessions need a place to put that memory. The choice splits into three architectures: file-based curation, structured stores (vector plus graph), and reinforcement learning that trains memory behavior into the model weights. Each has different failure modes, retrieval latency profiles, and concurrency boundaries.
Pinglin’s evaluation compares all three in production workloads. The results expose when each pattern breaks and what you pay for durability.
File-Based Memory: Markdown as State
File-based memory treats persistence as a filesystem problem. The agent writes a MEMORY.md index and topic-specific markdown files. Retrieval is search plus read: grep the index, load the file, pass it into the next context window.
Architecture:
- Index file (typically
MEMORY.md) with one-line summaries - Topic files (
project_alpha.md,user_preferences.md) with structured markdown - Background consolidation pass that merges duplicates and prunes stale entries
- No database, no embeddings, no vector search
Concurrency model:
File locks or append-only logs. Most implementations (Claude Code, Cline, Cursor, Windsurf) do not handle concurrent writes. OpenClaw adds a consolidation daemon that runs between sessions, but it still assumes single-writer access during active use.
Failure modes:
- Concurrent agent writes corrupt the index
- Search misses relevant files if the index summary is stale
- Context window fills with irrelevant markdown if the agent over-retrieves
- No semantic ranking: the agent reads files in filesystem order or by keyword match
Latency:
Sub-millisecond for index search, 10-50ms for file reads depending on size. The bottleneck is the LLM deciding which files to load, not the filesystem.
Structured Stores: Vector + Graph
Structured memory mines every agent turn into atomic facts, embeds them into a vector index, and links them in a temporal graph. Retrieval is ranked similarity search followed by graph traversal for multi-hop reasoning.
Architecture:
- Fact extraction: LLM call per turn to mine structured entities and relations
- Vector index: embeddings for semantic similarity (Pinecone, Weaviate, Qdrant)
- Graph layer: temporal edges between facts, entity resolution across sessions
- Retrieval: top-k vector search, then graph walk to pull in connected context
Concurrency model:
Database-native. Writes are atomic at the fact level. Multiple agents can write concurrently without corruption, but you need transaction boundaries if you want read-your-writes consistency within a single agent turn.
Failure modes:
- Fact extraction hallucinations: the mining LLM invents entities that never existed
- Embedding drift: if you change the embedding model, old vectors become incomparable
- Graph traversal explosions: multi-hop walks pull in too much context and blow the window
- Stale entity resolution: the system merges two entities that should stay separate, or keeps them separate when they should merge
Latency:
Vector search: 20-100ms for top-k retrieval depending on index size and hosting (local vs. remote). Graph traversal adds 10-50ms per hop. Total retrieval can hit 200ms+ for complex queries with multiple hops.
RL-Based Memory: Trained Experience
Reinforcement learning moves memory behavior into the model. Episodes land in an experience bank, but the policy learns what to retrieve, when to trust it, and how to turn it into action. The memory is not a separate store; it is part of the trained weights.
Architecture:
- Episode bank: successful trajectories stored as (state, action, reward) tuples
- Policy training: RL algorithm (PPO, DPO, or similar) trains the model to use past episodes
- Retrieval: the model decides what to recall based on current state, no external search
- Forgetting: episodes that do not improve policy performance get pruned or down-weighted
Concurrency model:
Training is offline. The policy is frozen during inference, so there is no write concurrency problem. New episodes go into the bank and trigger a retraining cycle.
Failure modes:
- Catastrophic forgetting: the model unlearns old behaviors when trained on new episodes
- Pruned episodes: the RL algorithm discards an experience that turns out to be critical later
- Overfitting: the policy memorizes specific episodes instead of generalizing
- Training latency: retraining takes hours or days, so new memory is not available immediately
Latency:
Inference latency is the same as the base model (no external retrieval). Training latency is high: expect hours for small models, days for large ones. Memory updates are not real-time.
Comparative Trade-Offs
| Dimension | File-Based | Structured Store | RL-Trained |
|---|---|---|---|
| Retrieval Latency | < 50 ms (filesystem) | 50-200 ms (vector + graph) | 0 ms (no external lookup) |
| Write Concurrency | Single-writer (locks) | Multi-writer (database) | Offline (training) |
| Context Window Pressure | High (full files) | Medium (ranked facts) | Low (implicit in weights) |
| State Consistency | Eventual (consolidation) | Immediate (transactions) | Delayed (retraining) |
| Failure Recovery | Manual (edit markdown) | Automated (rollback) | Hard (retrain from scratch) |
| Semantic Ranking | None (keyword match) | Strong (embeddings) | Learned (policy) |
| Deployment Complexity | Minimal (filesystem) | High (vector DB + graph) | Very high (training infra) |
Implementation: File-Based Memory with Consolidation
OpenClaw’s memory-core plugin shows the file-based pattern with background consolidation. The agent writes to MEMORY.md during sessions, and a daemon merges duplicates between runs.
from pathlib import Path
from datetime import datetime, timedelta
# Consolidation runs between sessions to avoid write conflicts during active agent use.
# This keeps the index eventually consistent without requiring database-level locking.
def consolidate_memory(memory_path: Path, session_logs: list[Path]):
index = memory_path / "MEMORY.md"
entries = parse_index(index)
# Deduplicate by semantic similarity
embeddings = embed_entries(entries)
clusters = cluster_similar(embeddings, threshold=0.85)
# Merge clusters into single entries
merged = []
for cluster in clusters:
merged_text = llm_merge(cluster)
merged.append(merged_text)
# Prune stale entries (no access in 30 days)
cutoff = datetime.now() - timedelta(days=30)
access_log = load_access_log(session_logs)
pruned = [e for e in merged if access_log[e] > cutoff]
# Write back atomically
write_atomic(index, pruned)The consolidation pass runs between sessions, not during active use. This avoids write conflicts but means the index can be stale during a session.
Observability Gaps
Each pattern hides critical decisions from the operator:
File-based:
- No visibility into which files the agent actually read vs. which it should have read
- Consolidation logs show merges but not why the agent missed a relevant file
Structured stores:
- Fact extraction errors are silent unless you log the mining LLM calls
- Graph traversal paths are not exposed: you see the final context but not the walk
RL-trained:
- Episode selection is opaque: the policy decides what to recall, but you cannot inspect the decision
- Forgetting is invisible: pruned episodes disappear without a trace
You need custom instrumentation for all three. File-based systems need access logs per file. Structured stores need fact extraction diffs and graph traversal traces. RL systems need episode replay and policy attention maps.
Security Boundaries
File-based:
- Filesystem permissions control access
- No isolation between agents: all share the same memory directory
- Injection risk: an agent can write malicious markdown that another agent reads
Structured stores:
- Database ACLs control access per fact or entity
- Multi-tenancy is possible with row-level security
- Injection risk: fact extraction can mine malicious entities from user input
RL-trained:
- Model weights are global: no per-user or per-agent isolation
- Poisoning risk: adversarial episodes can corrupt the policy
- No runtime access control: the model decides what to recall
File-based systems are the easiest to audit (read the markdown). Structured stores are the easiest to isolate (database ACLs). RL systems are the hardest to secure (no runtime boundaries).
Technical Verdict
Use file-based memory when:
- You have a single agent with filesystem access
- Memory size is small (under 1MB of markdown)
- You can tolerate eventual consistency
- You want zero infrastructure dependencies
Use structured stores when:
- You need multi-agent concurrency
- Memory size is large (millions of facts)
- You need semantic ranking and multi-hop reasoning
- You can run a vector database and graph store
Use RL-trained memory when:
- You control the model training pipeline
- Memory behavior is part of the product (not a feature)
- You can tolerate hours or days of training latency
- You need the lowest possible inference latency
Most production agents start with file-based memory and migrate to structured stores when concurrency or scale becomes a problem. RL-trained memory requires dedicated training infrastructure and carries catastrophic forgetting risk, which limits its use to research environments and specialized products where memory behavior justifies the training cost.