mech.app
AI Agents

Flat-File Memory Sync: How Three AI Agents Share State Without a Vector Database

A reproducible pattern for cross-agent memory using markdown files and Syncthing instead of vector databases or runtime coordination layers.

Source: dev.to
Flat-File Memory Sync: How Three AI Agents Share State Without a Vector Database

Most multi-agent setups assume you will reach for a vector database, a runtime orchestrator, or a shared API service. Mehmet Aras built something simpler: three AI agents (Claude Code, Codex, Hermes) that share state through markdown files synchronized by Syncthing. No embedding pipeline, no central server, no query DSL. Just files, a sync daemon, and a strict write protocol.

This is not a toy. It is a working pattern for cross-agent memory that trades retrieval sophistication for debuggability and zero-latency reads.

The Problem: Agent Hot-Swap Amnesia

Aras runs three agents in rotation:

  • Claude Code handles daily coding tasks.
  • Codex CLI takes over when Claude hits quota limits or when a second model opinion is needed.
  • Hermes Agent manages personal logistics: travel, errands, language study.

Each agent reads a different instruction file (CLAUDE.md, AGENTS.md, SOUL.md), but those files are not shared memory. When you switch agents mid-task, the new agent starts cold. It does not know what you taught the previous agent, what changed yesterday, or what you already asked it to remember.

The naive fix is to re-explain context at the start of each session. After three rounds of “I am working on X, I prefer Y, please do not Z,” the cost of re-priming outgrows the value of the answer.

The goal: one shared file tree that all three agents read and write. Memory follows the user, not the agent.

Why Not the Usual Suspects?

Before building a flat-file solution, Aras evaluated the defaults:

ApproachWhy It Fails Here
ObsidianDesktop GUI that assumes a human in front of a window. Plugin runtime and sync service do not run headless on a VPS or home device.
Vector databaseAdds embedding pipeline, query latency, and operational complexity. Overkill when you need simple append and read, not semantic search.
Notion APIRate limits, network dependency, and opaque conflict resolution. Agents cannot inspect or debug state on disk.
Git-based syncRequires manual commit discipline. Merge conflicts are not resolved automatically, and agents cannot write without human intervention.

The filesystem is already a shared substrate. Syncthing already handles bidirectional sync without a central server. Markdown is already the lingua franca for LLM context. The plumbing is simpler than the alternatives.

Architecture: Syncthing + Markdown + Write Protocol

The memory layer has three components:

  1. Markdown files as the state substrate.
  2. Syncthing as the synchronization layer.
  3. A strict write protocol to avoid conflicts.

File Structure

memory/
├── INDEX.md          # Hub file with pointers to all memory fragments
├── projects/
│   ├── project-x.md
│   └── project-y.md
├── preferences/
│   └── coding-style.md
└── facts/
    └── personal-context.md

INDEX.md is the entry point. Every agent reads it first. It contains links to other memory files, organized by topic. Agents do not search the filesystem. They follow the index.

Syncthing Configuration

Syncthing runs on each device where an agent operates. It watches the memory directory and syncs changes bidirectionally. No central server. No cloud dependency. Conflicts are rare because agents follow a write protocol that minimizes simultaneous writes to the same file.

When a conflict does occur, Syncthing creates a .sync-conflict file. The agent is instructed to check for these files before writing and to alert the user if one appears.

Write Protocol

Agents follow three rules:

  1. Read INDEX.md first. Never assume you know where a fact lives.
  2. Append, do not overwrite. Add new entries to the end of a file. Do not rewrite existing lines unless explicitly instructed.
  3. Check for conflicts before writing. If a .sync-conflict file exists, stop and notify the user.

This protocol trades write flexibility for conflict avoidance. Agents cannot reorganize memory on their own. They can only add to it.

Agent Workflow Example: Hermes Write

Hermes Agent is asked to remember a new travel preference: “I prefer aisle seats on flights.”

  1. Hermes reads INDEX.md and finds the link to preferences/travel.md.
  2. Hermes opens preferences/travel.md and appends:
## Seating Preference
- Aisle seats preferred on all flights (added 2026-05-17)
  1. Hermes saves the file. Syncthing detects the change and syncs it to other devices.
  2. The next time Claude Code or Codex reads preferences/travel.md, the new preference is visible.

No API call. No embedding step. No query latency. The agent writes to disk, and Syncthing handles propagation.

Failure Modes and Observability

Stale Reads During Sync Window

Syncthing is eventually consistent. If Agent A writes to a file and Agent B reads the same file before the sync completes, Agent B sees stale data. The sync window is typically under one second on a local network, but it can stretch to minutes on a slow connection.

Mitigation: Agents are instructed to timestamp their writes. If a fact is critical, the agent can check the timestamp and warn the user if the data is older than expected.

Simultaneous Writes to the Same File

If two agents write to the same file at the same time, Syncthing creates a conflict file. The write protocol requires agents to check for .sync-conflict files before writing. If one exists, the agent stops and asks the user to resolve it manually.

This is rare in practice because the three agents operate in different domains (coding, CLI fallback, personal logistics). They rarely touch the same file within the same second.

Scale Limits

Flat-file memory does not scale like a vector database. When the memory tree grows beyond a few hundred KB, agents spend more tokens reading irrelevant context. The index helps, but it is not a substitute for semantic search.

Aras reports that the setup works well for a few dozen memory files totaling under 1 MB. Beyond that, you need either a more sophisticated index (e.g., topic tags) or a migration to embedding-based retrieval.

Debugging

The biggest advantage of flat-file memory is debuggability. You can open any memory file in a text editor and see exactly what the agent wrote. You can grep the directory. You can version it with Git if you want a history. You can inspect Syncthing logs to see when a file was synced.

Compare this to a vector database, where you need to query an API to see what the agent stored, and you cannot easily inspect the raw embeddings.

Privacy and Security Boundaries

Syncthing does not encrypt files at rest. If you store sensitive data in the memory tree, you need filesystem-level encryption (e.g., LUKS, FileVault, or VeraCrypt).

Syncthing does encrypt data in transit between devices. The sync protocol uses TLS, and devices authenticate with unique device IDs. An attacker on the network cannot read or modify memory files during sync.

Agents do not have network access to each other. They only read and write to the local filesystem. Syncthing is the only process that touches the network. This reduces the attack surface compared to a setup where agents call a shared API.

Language Regime

Aras enforces a language rule: all memory files are written in English, even though Hermes Agent handles Turkish-language tasks. This prevents language drift and ensures that all three agents can parse the same memory files without translation overhead.

Agents are instructed to translate user input into English before writing to memory. This adds a small latency cost but eliminates the need for multilingual parsing logic.

Code Snippet: Agent Memory Read

Here is the pseudocode an agent follows when reading memory:

def read_memory(topic: str) -> str:
    # Step 1: Read the index
    index_path = "memory/INDEX.md"
    index_content = read_file(index_path)
    
    # Step 2: Find the link to the topic
    link = extract_link(index_content, topic)
    if not link:
        return f"No memory file found for topic: {topic}"
    
    # Step 3: Read the target file
    target_path = f"memory/{link}"
    if not file_exists(target_path):
        return f"Memory file not found: {target_path}"
    
    # Step 4: Check for conflicts
    conflict_path = f"{target_path}.sync-conflict"
    if file_exists(conflict_path):
        return f"Conflict detected: {conflict_path}. Please resolve manually."
    
    # Step 5: Return the content
    return read_file(target_path)

The agent does not search the filesystem. It follows the index. If the index is wrong, the agent fails loudly instead of guessing.

Technical Verdict

Use this pattern when:

  • You run multiple agents that need to share mutable state.
  • You want zero-latency reads and do not need semantic search.
  • You value debuggability over query sophistication.
  • You operate on a small scale (dozens of files, under 1 MB total).
  • You can enforce a strict write protocol (append-only, check for conflicts).

Avoid this pattern when:

  • You need semantic search or similarity-based retrieval.
  • Your memory tree will grow beyond a few hundred KB.
  • You cannot enforce a write protocol (e.g., agents are untrusted or uncoordinated).
  • You need strong consistency guarantees (Syncthing is eventually consistent).
  • You need to query memory from a web service or external API (flat files are not network-accessible without additional plumbing).

Flat-file memory is not a replacement for vector databases. It is a different trade-off. You give up query power and scale in exchange for simplicity, debuggability, and zero-latency reads. For a small multi-agent setup where agents operate in different domains, that trade-off is often worth it.

Tags

agentic-ai orchestration infrastructure

Primary Source

dev.to