Most agent demos show interactive CLI sessions. You run a command, watch the agent edit files, inspect the diff, and decide whether to commit. That model breaks when you want an always-on service that picks up GitHub issues, fixes them autonomously, and opens pull requests without human supervision.
The fly-swe project (built on mini-swe-agent) documents what changes when you move from synchronous CLI to queue-based production: deployment shape, budget enforcement, security boundaries, and the failure modes that only appear when the agent runs unattended.
Deployment Shape: One Machine, One Process Pair, One SQLite File
The entire service runs on a single Fly.io shared-cpu-1x machine with 1 GB persistent volume. Total cost: $6/month.
Two processes run under honcho:
- FastAPI webhook receiver: accepts GitHub webhook payloads, verifies HMAC signatures with timing-safe comparison, de-duplicates by delivery ID, writes tasks to SQLite
- Worker process: polls the SQLite queue, executes agent runs, handles git operations outside the agent sandbox
The queue is a SQLite database in WAL mode on the persistent volume. No Postgres, no Redis, no separate message broker. WAL mode allows concurrent reads during writes, so the webhook receiver can insert while the worker polls.
If the machine restarts mid-task, orphan recovery logic re-queues incomplete tasks. The delivery ID prevents duplicate execution. Nothing is lost, nothing runs twice.
Security Boundary: The Agent Never Touches Git
The agent process edits files in a working tree but has no git credentials. After the agent signals completion, the harness (running outside the agent sandbox) performs all git operations:
git addgit commitgit push- GitHub PR creation via API
The agent’s shell environment:
- Git remote URL with no push token
- No
ghCLI binary - Environment variable allowlist with no secrets
- Read-only access to repository metadata
When the target repository’s AGENTS.md instructs the agent to “commit and open a PR,” the agent follows the style and testing guidelines but cannot execute git commands. The harness handles all version control and API calls.
This boundary prevents credential leakage, accidental force-pushes, and malicious tool calls that attempt to exfiltrate secrets via git operations.
Three-Tier Budget Enforcement
Budget limits are enforced in code, not configuration files:
| Budget Type | Scope | Behavior on Limit |
|---|---|---|
max_iterations | Per task | Hard cap on agent steps; prevents infinite loops |
max_cost_per_task_usd | Per task | Stops execution; force-pushes partial work to WIP branch |
max_daily_cost_usd | Per repo per day | Blocks new tasks; existing task completes |
When cost_tracking: strict is enabled, the system refuses to run any step that cannot be priced. This prevents surprise bills from new model releases or API changes.
When a task hits max_cost_per_task_usd, the harness:
- Force-pushes partial edits to a WIP branch
- Marks the task as incomplete in the queue
- Logs cost breakdown and token usage
- Allows the next webhook trigger to resume from the WIP branch
This design trades completion guarantees for cost predictability. A $2 budget might leave a task half-done, but it will never generate a $200 bill.
Queue Design and Execution Guarantees
The SQLite queue provides at-least-once delivery with idempotency via delivery ID. The schema:
CREATE TABLE tasks (
delivery_id TEXT PRIMARY KEY,
repo_full_name TEXT NOT NULL,
issue_number INTEGER NOT NULL,
labels TEXT NOT NULL,
status TEXT DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
started_at TIMESTAMP,
completed_at TIMESTAMP,
cost_usd REAL,
error_message TEXT
);The worker polls with:
SELECT * FROM tasks
WHERE status = 'pending'
ORDER BY created_at ASC
LIMIT 1;After selecting a task, the worker updates status to running and sets started_at. If the machine crashes, orphan recovery finds tasks with status = 'running' and started_at older than 10 minutes, resets them to pending.
This is simpler than SQS or Redis Streams but lacks:
- Priority queues (all tasks are FIFO)
- Visibility timeouts (orphan recovery is time-based, not lease-based)
- Dead letter queues (failed tasks stay in the main table with
status = 'failed')
For a single-repo agent running on a $6 machine, these limitations are acceptable. For multi-tenant production, you would need a real message broker.
Failure Modes in Production
The test report documents real failures on a TypeScript monorepo:
Task completion rate: 68% (17 of 25 issues)
Failure categories:
- Tool call errors (4 tasks): agent requested non-existent files, invalid grep patterns, or malformed shell commands
- Budget exhaustion (2 tasks): hit
max_cost_per_task_usdbefore completing fix - Test failures (2 tasks): agent opened PR but CI failed; no retry logic
Cost distribution:
- Median cost per successful task: $0.42
- 90th percentile: $1.18
- Maximum (budget-limited): $2.00
The agent does not retry failed tool calls. If grep returns an error, the agent sees the error message and decides whether to retry. Most failures happen because the agent hallucinates file paths or misinterprets error output.
Budget exhaustion leaves partial work on a WIP branch. The next webhook trigger can resume, but there is no automatic resume logic. Manual intervention is required to re-label the issue or close the WIP PR.
Test failures expose a gap: the agent opens a PR, CI runs, tests fail, but the agent never sees the failure. Adding a CI webhook to re-trigger the agent with test output would close this loop but adds complexity.
Observability and Cost Tracking
Every task writes:
- Token counts (prompt, completion, total)
- Model name and pricing tier
- Wall-clock duration
- Final status and error message
Cost calculation happens in the harness, not the agent. The harness queries the LLM provider’s pricing API before each step and refuses to proceed if pricing data is unavailable (when cost_tracking: strict).
Logs are written to stdout and captured by Fly.io’s log shipper. No structured logging library, no tracing spans, no metrics exporter. For a $6 machine, this is sufficient. For multi-tenant production, you would add OpenTelemetry and a real observability backend.
When to Use This Architecture
Good fit:
- Single-repo or small-scale multi-repo automation
- Predictable issue volume (< 100/day)
- Cost sensitivity (hard budget caps required)
- Tolerance for partial completions and manual intervention
Poor fit:
- Multi-tenant SaaS with unpredictable load
- Real-time response requirements (queue adds latency)
- High completion rate requirements (68% may be too low)
- Complex retry logic or workflow orchestration
The queue-based model trades real-time responsiveness for cost control and crash recovery. The three-tier budget system prevents runaway costs but allows partial work. The git isolation boundary prevents credential leakage but requires a harness to handle version control.
If you need higher completion rates, add retry logic and CI feedback loops. If you need multi-tenancy, replace SQLite with a real message broker and add per-tenant cost tracking. If you need real-time response, replace the queue with synchronous webhook handlers and accept the risk of lost tasks on crash.
Technical Verdict
Use this if your issue volume stays below 100 per day, you have hard monthly LLM budget caps (under $50), and you can tolerate 60-70% task completion rates. The $0.42 median cost per task and $6/month infrastructure cost make this viable for side projects, small open-source repos, and teams experimenting with autonomous agents. The three-tier budget system guarantees you will not wake up to a $500 API bill.
Avoid this if you need completion rates above 85%, response times under one minute, or multi-tenant scale. The SQLite queue adds 5-30 second latency between webhook and agent start. The single-machine deployment has no horizontal scaling path. Budget exhaustion leaves 8% of tasks half-done, requiring manual cleanup. Tool call errors (16% of failures) need retry logic or better prompt engineering to reduce.
The honest test report (68% completion, 4 tool call errors, 2 budget exhaustions) is the most useful part. It exposes the gap between “works in my terminal” and “runs unattended for a month.”