Automatisch is an open-source workflow automation tool built as a self-hosted alternative to Zapier. It earned 317 points and 58 comments on Hacker News not because it does something novel, but because it exposes the orchestration primitives that both traditional automation platforms and modern agent frameworks need to solve: trigger handling, step execution boundaries, state persistence across failures, and retry logic when third-party APIs misbehave.
The project has been in development for 15 months by two co-founders who positioned it for teams that cannot send sensitive data to external cloud services. Healthcare, finance, and GDPR-bound European companies need workflow automation but cannot afford vendor lock-in or data exfiltration risk. That constraint forces architectural decisions that matter for anyone building agent orchestration infrastructure.
Trigger Handling: Polling vs. Webhooks
Workflow engines need two trigger modes:
- Polling: The engine queries an external API on a schedule (every 5 minutes, hourly) to check for new records. Simple to implement but wasteful and introduces latency.
- Webhooks: The external service calls your engine when an event occurs. Requires exposing an HTTP endpoint, managing authentication tokens, and handling replay attacks.
Automatisch supports both. Polling is the default because it works with any API that has a list endpoint. Webhooks require the external service to support them and the user to configure callback URLs, which adds friction during setup.
For agent orchestrators, this trade-off reappears when deciding whether an agent should poll a tool for status updates or whether the tool should push state changes to the agent. Polling is easier to reason about but burns cycles. Webhooks require infrastructure (a public endpoint, TLS, request validation) and introduce new failure modes (webhook delivery failures, retries, duplicate events).
Step Execution Model
Automatisch uses a job queue backed by PostgreSQL. Each workflow execution becomes a job, and each step in the workflow becomes a task within that job. The engine uses a worker pool to pull tasks off the queue and execute them.
Key design choices:
- Single database for state and queue: PostgreSQL stores both workflow definitions and execution state. This simplifies deployment but creates a single bottleneck.
- Worker pool concurrency: Multiple workers can execute different steps in parallel, but steps within a single workflow run sequentially by default unless explicitly marked as parallel branches.
- Idempotency: Each step execution is logged with a unique ID. If a worker crashes mid-execution, the retry logic can check whether the step already completed.
This model works for workflows with predictable step durations (seconds to minutes). It breaks down when steps take hours or when you need to coordinate thousands of concurrent executions. Agent orchestrators face the same problem when tool calls have unpredictable latency or when an agent spawns sub-agents that need their own execution contexts.
State Persistence Strategy
Automatisch persists execution state after every step. The database stores:
- Workflow definition (trigger, steps, connections between them)
- Execution history (which steps ran, when, with what inputs and outputs)
- Error logs (stack traces, retry counts, failure reasons)
This approach trades write amplification for recoverability. If a worker crashes, the next worker can pick up the job and resume from the last completed step. The engine does not need to replay the entire workflow from the beginning.
For agent systems, this maps to checkpointing: saving the agent’s state (memory, tool call history, intermediate outputs) after each action so that a crash does not lose hours of work. The trade-off is the same: more database writes in exchange for faster recovery and better observability.
Error Recovery and Retry Logic
Automatisch implements exponential backoff for failed steps. If a step fails, the engine waits 1 minute, then 2, then 4, up to a maximum of 60 minutes. After a configurable number of retries (default: 3), the workflow execution is marked as failed and the user receives a notification.
The retry logic is step-specific. If step 3 fails, the engine does not re-execute steps 1 and 2. This requires that each step is idempotent or that the engine tracks which side effects have already occurred.
Agent orchestrators need similar retry boundaries. If an agent calls a tool and the tool times out, should the agent retry the exact same call, retry with modified parameters, or escalate to a human? The answer depends on whether the tool call has side effects (sending an email, charging a credit card) and whether those side effects are idempotent.
Architecture Overview
┌─────────────┐
│ Web UI │
└──────┬──────┘
│
v
┌─────────────────────────────────────┐
│ API Server (Node.js) │
│ - Workflow CRUD │
│ - Trigger registration │
│ - Webhook endpoints │
└──────┬──────────────────────────────┘
│
v
┌─────────────────────────────────────┐
│ PostgreSQL Database │
│ - Workflow definitions │
│ - Execution state │
│ - Job queue │
└──────┬──────────────────────────────┘
│
v
┌─────────────────────────────────────┐
│ Worker Pool (Node.js) │
│ - Pull tasks from queue │
│ - Execute steps │
│ - Write results back to DB │
└─────────────────────────────────────┘The API server handles user requests and webhook deliveries. Workers poll the database for pending tasks. This separation allows horizontal scaling of workers without touching the API server, but both components share the same database, which becomes the scaling bottleneck.
Integration Layer
Automatisch uses a plugin architecture for integrations. Each integration (Slack, Google Sheets, Airtable) is a separate npm package that exports:
- Triggers: Functions that poll or register webhooks
- Actions: Functions that perform operations (send message, create row)
- Authentication: OAuth2 flows, API key management
This is the same pattern that agent frameworks use for tools. Each tool is a function with a schema (inputs, outputs, error types) and a runtime (the actual code that calls the external API). The orchestrator does not care what the tool does internally. It only cares about the contract: what inputs it needs and what outputs it returns.
Comparison: Self-Hosted vs. Cloud Workflow Engines
| Dimension | Self-Hosted (Automatisch) | Cloud (Zapier, Make) |
|---|---|---|
| Data residency | Full control, stays on your infrastructure | Data passes through vendor servers |
| Scaling | Manual (add workers, tune DB) | Automatic, vendor-managed |
| Cost | Infrastructure + maintenance time | Per-task pricing, can get expensive |
| Observability | Direct database access, custom logging | Vendor dashboard, limited export |
| Failure recovery | You own the retry logic and alerting | Vendor handles retries, may hit rate limits |
| Integration catalog | Smaller, community-driven | Hundreds of pre-built integrations |
Code Example: Defining a Workflow Step
Automatisch workflows are JSON objects. Here is a simplified step definition:
{
"id": "step_001",
"type": "action",
"appKey": "slack",
"key": "sendMessage",
"parameters": {
"channel": "{{trigger.channel_id}}",
"text": "New user signed up: {{trigger.user_email}}"
},
"nextSteps": ["step_002"]
}The parameters object uses template syntax to reference data from previous steps. The orchestrator resolves these references at runtime by looking up values in the execution context. This is the same pattern that agent frameworks use for tool call parameters: the agent fills in placeholders with data from its memory or from previous tool outputs.
Observability and Debugging
Automatisch logs every step execution to the database. The web UI shows:
- Execution timeline (when each step started and finished)
- Input and output data for each step
- Error messages and stack traces
This is table-stakes for agent orchestration. When an agent fails, you need to see the exact sequence of tool calls, the data that flowed between them, and where the failure occurred. Without this, debugging is impossible.
The challenge is log volume. A workflow with 10 steps generates 10 database rows per execution. An agent that makes 50 tool calls generates 50 log entries. If you run 1,000 executions per hour, that is 10,000 to 50,000 rows per hour. You need log rotation, archival, and efficient querying or your database will collapse.
Deployment Shape
Automatisch ships as a Docker Compose stack:
- One container for the API server
- One container for the worker pool
- One container for PostgreSQL
- One container for Redis (used for caching and rate limiting)
This is a reasonable starting point but has limitations:
- Single database: All state lives in PostgreSQL. If the database goes down, the entire system stops.
- No horizontal scaling for the API server: The Compose file runs one API container. To scale, you need a load balancer and session affinity.
- Worker pool is stateless: You can scale workers horizontally by increasing the replica count, but they all compete for the same job queue.
For production, you would replace Docker Compose with Kubernetes, add a managed PostgreSQL instance, and use a message queue (RabbitMQ, SQS) instead of polling the database for jobs.
Failure Modes
Self-hosted workflow engines fail in predictable ways:
- Database saturation: Too many concurrent executions writing state to the same PostgreSQL instance. Solution: partition by workflow ID, use read replicas, or switch to a distributed database.
- Worker starvation: Long-running steps block workers, preventing other jobs from executing. Solution: time limits per step, separate worker pools for fast and slow tasks.
- Webhook delivery failures: External services retry webhook deliveries, causing duplicate executions. Solution: idempotency keys, deduplication logic.
- API rate limits: Third-party APIs throttle requests, causing step failures. Solution: per-integration rate limiting, backoff logic, credential rotation.
Agent orchestrators hit the same failure modes. The difference is that agents often make more tool calls per execution and have less predictable execution times, which amplifies the impact of each failure.
Technical Verdict
Use Automatisch (or build something like it) when:
- You need workflow automation but cannot send data to external cloud services.
- You have the infrastructure team to run and maintain a PostgreSQL cluster, worker pool, and monitoring stack.
- Your workflows have predictable step durations (seconds to minutes) and moderate concurrency (hundreds of executions per hour, not thousands).
- You want full control over retry logic, observability, and integration code.
Avoid it when:
- You need to scale to thousands of concurrent executions without managing infrastructure. Use a cloud platform instead.
- Your workflows have unpredictable or very long step durations (hours). You need a more sophisticated execution model (durable execution, saga pattern).
- You do not have the team to handle database tuning, worker scaling, and incident response. The operational burden is real.
For agent builders, Automatisch is a useful reference architecture. It shows how to structure trigger handling, step execution, state persistence, and error recovery in a way that works for both traditional automation and agent orchestration. The trade-offs are the same: simplicity and control versus operational complexity.