MiroFish: Swarm Prediction Through Emergent Agent Behavior

MiroFish constructs parallel digital worlds populated by thousands of autonomous agents, each with independent memory, personality, and behavioral logic. Instead of running statistical models, it predicts financial and social outcomes by letting agents interact freely and observing what emerges. You seed the simulation with real-world data (news, financial signals, policy drafts), inject variables mid-run, and watch the swarm evolve toward a forecast.

The project hit 62,644 GitHub stars in weeks and sits at #11 on Python trending. Backed by Shanda, it targets financial forecasting, public opinion analysis, and social prediction through swarm intelligence rather than single-agent reasoning or traditional ML pipelines.

Architecture: Seed, Spawn, Evolve, Observe

MiroFish follows a four-phase orchestration pattern:

1. Seed extraction: Ingest real-world data (news feeds, market signals, policy text) and parse it into structured knowledge fragments.
2. Agent spawning: Generate thousands of agents with distinct personalities, memory stores, and behavioral rulesets derived from the seed data.
3. Free interaction: Agents communicate, form opinions, trade, debate, and evolve without a central coordinator. The simulation runs in discrete time steps.
4. God-view injection: Users inject variables (policy changes, market shocks, new information) mid-simulation to test counterfactuals and observe trajectory shifts.

The engine maintains a knowledge graph to track agent relationships, memory states, and interaction history. Predictions emerge from aggregate agent behavior, not from a single model output.

Agent Initialization

Each agent receives:

• Personality vector: Traits like risk tolerance, social influence, information bias, and decision speed.
• Memory store: Long-term memory of past interactions, short-term working memory for current context.
• Behavioral logic: Rule-based or LLM-driven decision functions that govern how the agent reacts to new information and other agents.

Seed data determines the initial distribution of personalities and memory states. For example, a financial simulation seeded with bearish news might spawn more risk-averse agents, while a policy simulation seeded with public debate might create polarized opinion clusters.

Interaction Orchestration

MiroFish avoids a central event bus. Instead, agents operate in a shared environment with spatial or logical proximity rules. Interaction happens through:

• Message passing: Agents broadcast opinions or information to neighbors in the knowledge graph.
• State observation: Agents read shared environment variables (market prices, public sentiment scores) and update their internal state.
• Action execution: Agents take actions (buy, sell, vote, share information) that modify the environment and trigger reactions from other agents.

The simulation advances in discrete time steps. Each step, the engine:

1. Collects pending agent actions.
2. Resolves conflicts (e.g., multiple agents bidding for the same asset).
3. Updates the environment state.
4. Propagates changes to agent memory stores.

No global lock. Agents process in parallel within each time step, with conflict resolution handled by domain-specific rules (auction logic for financial markets, voting rules for social predictions).

State Management and Counterfactual Testing

MiroFish supports mid-simulation variable injection. Users can pause the simulation, inject a new event (policy announcement, market shock, viral news), and resume. The engine:

• Snapshots the current state (agent memory, environment variables, knowledge graph).
• Injects the new variable as a broadcast event or environment update.
• Resumes the simulation and tracks divergence from the baseline trajectory.

This enables A/B testing of scenarios. Run the same simulation twice, inject different variables, and compare outcomes.

Rollback and Branching

The engine supports state rollback through snapshot persistence. Each time step, the system can optionally write a checkpoint to disk (agent memory dumps, environment state, graph topology). To test a counterfactual:

1. Load a checkpoint from time T.
2. Inject the new variable.
3. Run forward to time T+N.
4. Compare with the original trajectory.

Branching creates parallel simulation instances from the same checkpoint, each with different injected variables. This is computationally expensive but allows exhaustive scenario exploration.

Feature	Implementation	Trade-off
Agent memory	In-memory key-value store per agent	Fast reads, high RAM cost at scale
Knowledge graph	NetworkX or graph database	Flexible queries, slower than flat structures
Snapshot persistence	Pickle or Parquet dumps	Simple rollback, large disk footprint
Parallel execution	Thread pool per time step	Scales to CPU cores, not distributed
Conflict resolution	Domain-specific rules (auction, voting)	Deterministic, requires manual tuning

Observability and Prediction Tracing

MiroFish exposes several observability primitives:

• Agent trace logs: Record every decision, interaction, and memory update for a subset of agents.
• Aggregate metrics: Track swarm-level statistics (average sentiment, market price, opinion distribution) over time.
• Interaction graphs: Visualize which agents influenced which decisions, forming a causal chain from seed data to final prediction.

To trace a prediction back to its source:

1. Identify the final aggregate metric (e.g., predicted stock price at time T).
2. Query the interaction graph for agents whose actions moved the metric.
3. Walk backward through agent memory logs to find the seed information or injected variable that triggered the behavior.

This is computationally expensive. Full tracing requires storing every agent state transition, which scales poorly beyond a few thousand agents. Most deployments log only critical agents (high-influence nodes in the knowledge graph) or sample randomly.

Example: Financial Forecast Trace

# Pseudo-code for tracing a price prediction
simulation = MiroFishEngine(seed_data="bearish_news.json")
simulation.run(steps=100)

final_price = simulation.environment["market_price"]
influencers = simulation.graph.top_k_influencers(metric="price", k=10)

for agent in influencers:
    trace = simulation.get_agent_trace(agent.id)
    print(f"Agent {agent.id} actions:")
    for step, action in trace:
        print(f"  Step {step}: {action.type} triggered by {action.cause}")

The trace reveals which agents sold aggressively, which news fragments they read, and which other agents influenced their decision. This exposes the causal path from seed data to emergent outcome.

Scaling and Computational Cost

MiroFish scales vertically within a single machine. The engine uses thread pools to parallelize agent execution within each time step, but does not distribute across nodes. Practical limits:

• 100-500 agents: Runs on a laptop with full tracing and rich memory.
• 1,000-5,000 agents: Requires a workstation with 32+ GB RAM, limited tracing.
• 10,000+ agents: Needs aggressive memory pruning, sparse interaction graphs, and no full tracing.

To scale beyond 10,000 agents, you need to:

• Shard the knowledge graph by agent clusters (geographic regions, social groups).
• Reduce memory fidelity (shorter history, coarser personality vectors).
• Disable full tracing and rely on sampled logs.

The engine does not currently support distributed execution across multiple machines. Adding that requires rethinking the interaction model (message queues instead of shared memory) and conflict resolution (distributed consensus instead of local rules).

Failure Modes and Edge Cases

Convergence stalls: If agents reach a stable equilibrium early, the simulation stops evolving. Predictions become static. Mitigation: inject random noise, introduce new agents mid-simulation, or add external shocks.

Runaway feedback loops: Agents amplify each other’s behavior (panic selling, viral misinformation). The simulation diverges from realistic outcomes. Mitigation: damping factors in behavioral logic, rate limits on action execution.

Memory bloat: Long simulations accumulate unbounded memory per agent. RAM exhaustion crashes the engine. Mitigation: sliding window memory (forget events older than N steps), periodic garbage collection.

Non-determinism: LLM-driven agent logic introduces randomness. Re-running the same simulation with the same seed produces different outcomes. Mitigation: fix random seeds for LLM sampling, or accept non-determinism and run multiple trials.

Graph topology drift: As agents interact, the knowledge graph topology changes. High-degree nodes become bottlenecks. Mitigation: prune low-weight edges periodically, cap node degree.

Deployment Shape

MiroFish runs as a Python process with optional Docker packaging. Typical deployment:

• Local workstation: For small simulations (hundreds of agents), rapid iteration.
• Cloud VM: For medium simulations (thousands of agents), batch processing.
• Scheduled jobs: Run nightly forecasts, store results in a database, expose via API.

No built-in web UI. Users interact via Python scripts or Jupyter notebooks. To productionize:

1. Wrap the engine in a FastAPI service.
2. Accept seed data and prediction queries via HTTP.
3. Run simulations asynchronously (Celery, RQ).
4. Store results in Postgres or S3.
5. Expose aggregate metrics and trace logs through a dashboard.

The engine does not include authentication, rate limiting, or multi-tenancy. You build that around it.

Technical Verdict

Use MiroFish when:

• You need to forecast outcomes driven by social dynamics, not just statistical trends (public opinion shifts, market panics, policy adoption).
• You want to test counterfactuals by injecting variables mid-simulation.
• You have the compute budget to run thousands of agents and the patience to tune behavioral logic.
• You value interpretability and want to trace predictions back to agent interactions.

Avoid MiroFish when:

• You need real-time predictions. Simulations take minutes to hours.
• You need deterministic, reproducible forecasts. LLM-driven agents introduce randomness.
• You need to scale beyond 10,000 agents without rearchitecting for distributed execution.
• You prefer black-box ML models over interpretable but complex agent-based simulations.

MiroFish is not a drop-in replacement for time-series forecasting or regression models. It is a tool for exploring emergent behavior in complex systems where agent interactions matter more than aggregate statistics. The plumbing is straightforward (spawn agents, run time steps, log state), but tuning agent logic and interpreting results requires domain expertise and experimentation.

Source Links

• MiroFish GitHub Repository
• Shanda (Backer)