Context.dev (YC S26) launched with a simple pitch: give us a URL and a JSON schema, get back structured data without writing a single CSS selector. For financial agents that need real-time pricing, SEC filings, or news sentiment without maintaining per-site parsers, this shifts the extraction boundary from code to schema.
The launch post on Hacker News drew 107 points and 73 comments, mostly from developers tired of maintaining scrapers that break every time a website redesigns. The core insight is that LLMs can map unstructured HTML to structured schemas more reliably than brittle XPath queries, as long as you handle the validation and retry plumbing correctly.
The Schema-Based Extraction Model
Context.dev’s approach centers on JSON schema definitions. Instead of writing selectors that target specific DOM elements, you describe the data structure you want. The API reads the page, uses an LLM to understand the content, and maps it to your schema.
The product site lists several capabilities: scraping and crawling (markdown, HTML, sitemap, search, full-site crawls), structured data extraction via JSON schema, brand data extraction (logos, colors, fonts), and website monitoring for changes. The HN discussion confirms that the API is designed for autonomous agent integration, not just human-driven workflows.
For a financial agent scraping pricing data, you define a schema with fields like price, currency, and plan_name. The API returns JSON that matches your schema or an error if extraction fails. No selectors, no DOM traversal, no regex parsing of text nodes.
Why Financial Agents Need This
Traditional web scraping breaks in predictable ways. A CSS class rename breaks your parser. A layout redesign breaks your XPath queries. A JavaScript framework migration breaks your headless browser automation. For financial agents that need to monitor dozens of data sources (competitor pricing, SEC filings, earnings transcripts, news sentiment), maintaining per-site parsers is not sustainable.
Schema-based extraction inverts the maintenance burden. Instead of updating selectors when a website changes, you update your schema when your data requirements change. The LLM handles the mapping between HTML structure and your desired output format.
The HN discussion includes comments from developers who switched from Firecrawl and other scraping tools. SiteGPT, an AI chatbot platform, migrated to Context.dev in under a day. Mintlify used it to transform GitHub repos into documentation sites with a 10-minute integration time. These migrations suggest that the API contract is straightforward and the schema approach reduces implementation complexity.
Schema Validation and Data Quality
The hard part is not extracting text. The hard part is guaranteeing that "$49/mo" becomes 49 as a number and "USD" as a currency code. Context.dev appears to handle type coercion and validation as part of the extraction process, though specific implementation details are not documented in the launch materials.
For financial agents, this matters because you cannot trade on ambiguous data. If a price field cannot be parsed reliably, you need to know before you act on it. The schema-based approach provides a contract: either you get data that matches your schema, or you get an error.
The product site mentions structured data extraction via JSON schema, which implies validation against the schema you provide. How confidence scoring works, whether partial results are returned, and what happens when extraction is ambiguous are implementation details not yet public.
Agent-First Infrastructure Requirements
Traditional scraping libraries assume a human is in the loop. Agent-first APIs assume thousands of autonomous calls with no supervision. Context.dev’s positioning as an API for “products and agents” suggests awareness of these requirements.
Key infrastructure concerns for financial agents include:
- Rate limiting and quotas: Agents need predictable billing and cannot rack up surprise costs
- Retry logic: Network failures and transient errors must be handled without manual intervention
- Caching: Repeated requests for the same URL should not burn quota unnecessarily
- Observability: Agents need to detect extraction failures and degraded accuracy before acting on data
The HN discussion does not detail how Context.dev handles these concerns, but the YC batch affiliation and agent-first positioning suggest that these are design priorities.
Multi-Page Aggregation and Crawling
Single-page extraction is straightforward. Multi-page aggregation is where most scrapers fail. The product site lists “full-site crawls” as a capability, which suggests support for pagination and multi-page data collection.
For financial agents scraping SEC filings or earnings transcripts, this means you can request data that spans multiple pages and receive a single aggregated result. The implementation details (how pagination is detected, how results are deduplicated, how depth limits work) are not documented in the launch materials.
Schema Versioning and Model Drift
The biggest operational risk with LLM-based extraction is model drift. When Context.dev upgrades its extraction model, your accuracy might change. The HN discussion does not address this directly, but it is a known problem with any LLM-powered infrastructure.
Best practices for agent-first APIs include schema version hashing, extraction results that include model version metadata, and test modes with cached results for deterministic CI runs. Whether Context.dev implements these patterns is not yet documented.
In practice, you version your schemas in Git and run integration tests against a staging environment before deploying schema changes to production agents.
Failure Modes and Soft Degradation
Schema-based extraction likely fails in predictable ways:
- Schema mismatch: Website structure changed, model cannot map fields
- Rate limiting: Target website blocks requests
- JavaScript rendering: Content loads after initial HTML
- Ambiguous content: Multiple valid interpretations exist
The architectural advantage over selector-based scraping is the potential for graceful degradation. Instead of throwing exceptions when a CSS class changes, a schema-based API could return partial results or confidence metadata. Whether Context.dev implements this pattern is not confirmed in the launch materials, but it is a logical design choice for agent-first infrastructure.
Selector-Based vs. Schema-Based Trade-offs
The fundamental trade-off is maintenance burden versus latency and cost. Selector-based scraping is fast (direct DOM access) but brittle (breaks on every redesign). Schema-based scraping is resilient (same schema works across layout changes) but adds LLM inference latency and per-request API costs.
For financial agents that scrape dozens of sites, the maintenance burden of selector-based scraping is prohibitive. You either maintain custom parsers for every site or accept that your data pipeline breaks frequently. Schema-based extraction shifts the burden to schema design, which changes less frequently than website layouts.
The cost model is per-request API pricing versus engineering time plus infrastructure. For low-volume use cases, maintaining a custom scraper is cheaper. For high-variety use cases (many sites, infrequent access to each), an API is more cost-effective.
Technical Verdict
Use Context.dev when:
- You need structured data from dozens of websites without maintaining per-site parsers
- Your agents make autonomous scraping decisions and you need predictable billing
- Schema changes are less frequent than website layout changes
- You can tolerate LLM inference latency (likely 1-5s per request)
Avoid it when:
- You scrape a single website at high volume (write a custom scraper)
- You need sub-second latency (LLM extraction adds overhead)
- Your schema is unstable and changes daily (you will burn through test quota)
- You need pixel-perfect screenshot data or non-text content extraction
For financial agents that need real-time pricing, filings, or news without maintaining scrapers, the trade-off is clear: pay per request and get reliability, or maintain scrapers and pay in engineering time. The shift from selector-based to schema-based extraction is a fundamental infrastructure change that makes web data accessible to autonomous agents without constant parser maintenance.
The launch materials do not document all implementation details (confidence scoring, retry logic, pagination handling), so production use will require testing and observability to understand failure modes and accuracy boundaries.