Agents are no longer limited by their intelligence. They are limited by their ability to interact with systems. Most high-value workflows live on websites you do not control: airline booking, bank transfers, e-commerce checkouts. These sites change their DOM structure without warning, breaking your scrapers and leaving your agents blind.
Pomerado addresses this by converting any website into a Model Context Protocol (MCP) server with self-healing selectors. When the site’s HTML changes, the tool detects the breakage and automatically repairs its extraction logic. The goal is to make third-party websites as reliable as first-party APIs for agent tool calls.
The Launch HN post (58 points, 47 comments) frames the core problem: agents need deterministic tools, but web scraping is inherently brittle. Pomerado’s infrastructure sits between the agent and the website, absorbing the chaos of DOM mutations and exposing a stable interface.
The Self-Healing Mechanism
Traditional web scrapers use CSS selectors or XPath expressions to locate elements. When the site redesigns, those selectors break. You get null values, exceptions, or worse: silently wrong data.
Pomerado’s approach:
- Initial selector generation: An LLM analyzes the page structure and generates selectors for target data (flight prices, account balances, product availability).
- Execution and validation: The tool runs the selectors and validates the extracted data against expected types and patterns.
- Breakage detection: On subsequent runs, if selectors return null or malformed data, the system flags a potential DOM change.
- Automatic repair: The LLM re-analyzes the current page HTML, identifies the new location of the target data, and generates updated selectors.
- Verification: The new selectors are tested against recent page snapshots to confirm they extract the correct data.
This is not a one-time setup. The system continuously monitors extraction success rates and triggers repairs when confidence drops below a threshold.
Key infrastructure questions:
- How do you know the repair worked? You need ground truth. Pomerado likely uses a combination of schema validation (does the extracted data match expected types?) and historical comparison (does the new value align with recent trends?).
- What if the LLM hallucinates a selector? The repair loop must include a rollback mechanism. If the new selector performs worse than the old one, revert and escalate to human review.
- How do you version selectors? Each repair creates a new selector version. The system needs to track which version is active, when it was deployed, and what triggered the change.
Determinism Under Website Flux
A deterministic tool returns the same output for the same input. But websites are not deterministic. Prices change, inventory updates, and the DOM structure mutates.
Pomerado’s definition of determinism is narrower: given a stable website state, the tool extracts the same data every time. The self-healing layer absorbs structural changes (DOM mutations) but does not eliminate content changes (price fluctuations).
What this means for agents:
- Structural reliability: The agent can trust that calling
get_flight_price(origin, destination, date)will return a price or a clear error, not a null value or a CSS class name. - Content volatility: The price itself may change between calls. The agent must handle this at the orchestration layer (retry logic, price comparison, timeout policies).
Failure modes the tool cannot fix:
- Rate limiting: The website blocks your IP after too many requests.
- Authentication changes: The site adds CAPTCHA or changes its login flow.
- Data removal: The site stops displaying the data you are scraping (e.g., removes public pricing).
The self-healing layer is not magic. It repairs selector breakage, not business logic changes.
MCP Server Integration
The Model Context Protocol (MCP) defines how agents discover and call tools. An MCP server exposes a set of tools (functions) with typed schemas. The agent sends a tool call request, the server executes it, and returns structured data.
Pomerado’s MCP server architecture:
{
"tools": [
{
"name": "get_flight_price",
"description": "Fetch current price for a flight route",
"inputSchema": {
"type": "object",
"properties": {
"origin": {"type": "string"},
"destination": {"type": "string"},
"date": {"type": "string", "format": "date"}
},
"required": ["origin", "destination", "date"]
}
}
]
}When the agent calls get_flight_price, the MCP server:
- Navigates to the airline website (headless browser or HTTP client).
- Executes the current selector set to extract the price.
- Validates the extracted data against the schema.
- Returns the price or triggers a self-healing repair if extraction fails.
Why MCP matters here:
- Standardized interface: The agent does not care whether the tool is hitting an API, a database, or a website. It just calls a function.
- Schema enforcement: The MCP server guarantees the return type. If the website changes and the tool cannot extract valid data, it returns an error, not garbage.
- Composability: You can chain multiple Pomerado tools (check price, book flight, confirm reservation) in a single agent workflow.
Versioning and Testing Third-Party Website Tools
You do not control the website. You cannot run integration tests in a staging environment. The site can change at any time, and you will not get a changelog.
Versioning strategy:
| Component | Versioning Approach | Rollback Trigger |
|---|---|---|
| Selectors | Timestamp-based versions (v1.2024-11-12-14:30) | Extraction success rate drops below 95% |
| Tool schema | Semantic versioning (v1.0.0, v1.1.0) | Breaking change in website data structure |
| MCP server | Git commit SHA + deployment tag | Critical bug or performance regression |
Testing approach:
- Synthetic monitoring: Run the tool every N minutes against the live website. Log success/failure and extracted data.
- Snapshot comparison: Store HTML snapshots of key pages. When selectors change, diff the old and new HTML to understand what broke.
- Canary deployments: Deploy new selector versions to a subset of agent requests. If the canary performs worse, roll back.
What you cannot test:
- Future DOM changes: You do not know what the next redesign will look like.
- Edge cases: The website may behave differently for different users, regions, or times of day.
The best you can do is detect breakage quickly and repair automatically.
Mid-Flight Failure Modes
An agent is halfway through booking a flight. It has selected the outbound leg, entered passenger details, and is about to confirm payment. The airline site pushes a redesign. The payment confirmation button moves to a new DOM location. The agent’s tool call fails.
What happens next?
Option 1: Fail fast
- The tool returns an error:
SelectorNotFound: payment_confirm_button. - The agent logs the failure and escalates to a human operator.
- The booking is abandoned.
Option 2: Retry with self-healing
- The tool detects the selector failure and triggers a repair.
- The LLM analyzes the new page structure and generates a new selector.
- The tool retries the payment confirmation with the new selector.
- If successful, the booking completes. If not, escalate.
Option 3: State recovery
- The agent saves its progress (selected flight, passenger details) to persistent storage.
- After the tool repair completes, the agent resumes from the last successful step.
- This requires the agent orchestration layer to support checkpointing and replay.
The hard part is state management. Websites do not expose transaction IDs or session tokens in a structured way. If the agent loses its place in a multi-step workflow, it may have to start over.
Mitigation strategies:
- Idempotency checks: Before retrying, verify the previous step did not partially complete (e.g., check if the flight is already in the cart).
- Human-in-the-loop: For high-stakes workflows (payments, legal documents), require human confirmation before final submission.
- Graceful degradation: If the tool cannot self-heal within a timeout, return partial data and let the agent decide whether to proceed.
Infrastructure Trade-Offs
| Approach | Reliability | Latency | Cost | Control |
|---|---|---|---|---|
| Official API | High (versioned, documented) | Low (direct HTTP) | Variable (per-call pricing) | Low (vendor-controlled) |
| Static scraper | Low (breaks on DOM changes) | Low (no repair overhead) | Low (no LLM calls) | High (you own the code) |
| Self-healing scraper (Pomerado) | Medium (auto-repairs, but not instant) | Medium (repair adds latency) | Medium (LLM calls for repairs) | Medium (you configure, Pomerado manages) |
When self-healing makes sense:
- The website does not offer an API.
- The data changes frequently enough that manual scraper maintenance is expensive.
- The agent workflow can tolerate occasional repair latency (seconds to minutes).
When it does not:
- The website has an official API (use that instead).
- The data is static or changes infrequently (a static scraper is simpler).
- The agent workflow requires sub-second latency (self-healing repair is too slow).
Observability Requirements
You cannot debug what you cannot see. A self-healing scraper needs instrumentation at every layer.
Metrics to track:
- Selector success rate: Percentage of tool calls that extract valid data without triggering a repair.
- Repair frequency: How often selectors break and need automatic fixes.
- Repair latency: Time from breakage detection to successful repair deployment.
- False positive rate: How often the system flags a breakage that was not a DOM change (e.g., transient network error).
Logs to capture:
- Tool call trace: Agent request, selector execution, extracted data, validation result.
- Repair event: Old selector, new selector, diff of HTML snapshots, LLM reasoning.
- Failure escalation: When a repair fails and a human must intervene.
Alerts to configure:
- Selector success rate drops below threshold.
- Repair loop exceeds max iterations (possible infinite loop).
- Critical tool (e.g., payment confirmation) fails multiple times in a row.
Technical Verdict
Use Pomerado-style self-healing APIs when:
- You need agent access to websites without official APIs (airlines, banks, e-commerce).
- The target sites change their DOM structure frequently enough that manual scraper maintenance is a bottleneck.
- Your agent workflows can tolerate repair latency and occasional escalation to human operators.
- You have observability infrastructure to monitor selector health and repair success.
Avoid when:
- The website offers an official API (use that instead, even if it costs more).
- The data is static or the site rarely changes (a simple scraper is cheaper and faster).
- Your agent workflow requires guaranteed sub-second latency (self-healing repair adds unpredictable delays).
- You cannot afford the operational overhead of monitoring selector versions, repair loops, and failure escalations.
The core insight is that self-healing does not eliminate brittleness. It automates the repair cycle. You trade manual scraper maintenance for automated repair monitoring. The infrastructure complexity shifts from writing selectors to managing selector versions, repair triggers, and rollback policies.
Source Links
- Pomerado Launch on Hacker News (58 points, 47 comments)
- Practical AI Podcast: Models, Harnesses, and Multi-Agent Systems (background on agent terminology and orchestration)