shot-scraper video: How Playwright's Screencast API Enables Agents to Record Their Own Demos

Simon Willison just shipped shot-scraper 1.10 with a new video command that lets agents record their own product demos. The entire feature was built by a coding agent, and it demonstrates a pattern worth studying: using CLI --help output as embedded agent documentation.

The implementation waited on Playwright 1.61.0, which finally removed the 800px width restriction from its screencast API. This is not the same as Playwright’s original video recording for test debugging. The screencast mechanism gives you frame-level control without unwanted chrome or white frames before the first URL loads.

Why Screencast vs. Video Recording

Playwright’s original video recording was designed for debugging test failures. It captured everything, including browser chrome and timing artifacts that make sense for post-mortem analysis but ruin product demos.

The screencast API introduced in Playwright 1.59 provides:

Frame-level start/stop control
No debugging chrome in output
Configurable viewport dimensions (fixed in 1.61.0)
Synchronization with page lifecycle events

This matters for agents because they need to generate clean, repeatable demos without manual editing. The old approach required post-processing to remove unwanted frames. The new API lets you start recording after the page loads and stop exactly when the interaction completes.

YAML Storyboard Format

The shot-scraper video command accepts a YAML storyboard that defines server startup, viewport configuration, and a sequence of browser interactions. Here’s the structure:

output: /tmp/demo.webm
server:
  - uv
  - run
  - datasette
  - -p
  - 6419
  - /tmp/demo.db
url: http://127.0.0.1:6419/demo/tasks
viewport:
  width: 1280
  height: 720
cursor: true
wait_for: 'button[data-table-action="insert-row"]'
javascript: |
  (() => {
    let clipboardText = "";
    Object.defineProperty(navigator, "clipboard", {
      configurable: true,
      get: () => ({
        writeText: async (text) => { clipboardText = String(text); },
        readText: async () => clipboardText,
      }),
    });
  })();
scenes:
  - name: Bulk insert existing table rows
    do:
      - pause: 0.8
      - click: 'button[data-table-action="insert-row"]'
      - wait_for: "#row-edit-dialog[open]"
      - fill:
          into: ".row-edit-bulk-textarea"
          text: |
            title,owner,status
            Task A,Ana,doing
            Task B,Ben,review
      - click: ".row-edit-save"
      - wait_for: "text=3 rows inserted."

The format is declarative, but the do blocks are imperative sequences. This hybrid approach reflects the reality of browser automation: you need to declare intent (viewport, cursor visibility) but specify exact interaction order (click, wait, fill).

Willison had the agent use Pydantic for validation, which serves two purposes. First, it catches malformed storyboards before Playwright starts. Second, the Pydantic schema documents the format in a way that’s easy to review and iterate on.

Here’s how the Pydantic schema defines the storyboard structure:

from pydantic import BaseModel, Field
from typing import List, Optional, Union

class ViewportConfig(BaseModel):
    width: int = 1280
    height: int = 720

class FillAction(BaseModel):
    into: str
    text: str

class SceneAction(BaseModel):
    pause: Optional[float] = None
    click: Optional[str] = None
    wait_for: Optional[str] = None
    wait_for_url: Optional[str] = None
    fill: Optional[FillAction] = None

class Scene(BaseModel):
    name: str
    open: Optional[str] = None
    wait_for: Optional[str] = None
    do: List[SceneAction]

class Storyboard(BaseModel):
    output: str
    server: Optional[List[str]] = None
    url: str
    viewport: Optional[ViewportConfig] = None
    cursor: bool = False
    wait_for: Optional[str] = None
    javascript: Optional[str] = None
    scenes: List[Scene]

This schema enforces type safety and provides clear error messages when agents generate invalid storyboards. The optional fields allow for minimal configurations while supporting complex multi-scene demos.

Clipboard Mocking Pattern

The javascript block in the storyboard injects a clipboard mock before any interactions run. This works around browser security restrictions that prevent headless automation from accessing the real clipboard API.

Object.defineProperty(navigator, "clipboard", {
  configurable: true,
  get: () => ({
    writeText: async (text) => { clipboardText = String(text); },
    readText: async () => clipboardText,
  }),
});

This pattern is necessary because:

Headless browsers block clipboard access by default
Real clipboard APIs require user gestures in most contexts
Agents need to test copy/paste flows without manual intervention

The mock stores clipboard text in a closure variable. It’s not a real clipboard, but it satisfies the API contract for demo purposes. This works for navigator.clipboard.writeText() and navigator.clipboard.readText() but not for keyboard events (Ctrl+C, Ctrl+V) or DataTransfer APIs used in drag-and-drop operations. This is a common gap between headless automation and real user sessions: security boundaries that make sense for production break automation workflows.

CLI Help as Agent Documentation

The feature was built by prompting a coding agent to run shot-scraper video --help and use that output as the specification. This pattern treats CLI help text as a bundled SKILL.md file.

The help output includes:

Command syntax and flags
YAML schema examples
Interaction primitives (click, fill, wait_for)
Server lifecycle management
Output format options (WebM, MP4)

This works because modern coding agents can parse structured help text and infer usage patterns. The alternative (separate documentation files, API specs, or example repositories) adds coordination overhead. Embedding the spec in --help means the agent always has current documentation.

Willison uses the same pattern in his showboat and rodney tools. The trade-off is that help text becomes longer and more structured, but the payoff is that agents can discover and use the tool without external context. Human users may find the verbose help text overwhelming, but agents benefit from having complete specifications in a single command output.

Architecture: Server Lifecycle and Recording Sync

The storyboard’s server block defines a command array that shot-scraper spawns as a subprocess. The tool waits for the server to respond at the specified url before starting Playwright.

Sequence:

Parse and validate YAML storyboard
Spawn server subprocess
Poll URL with exponential backoff
Launch Playwright with screencast enabled
Navigate to URL and wait for wait_for selector
Execute scene interactions in order
Stop recording and terminate server

The wait_for selector at the top level ensures the page is interactive before the first scene starts. Each scene can override the URL with an open directive, which triggers a new navigation and wait cycle.

This design assumes the server is stateless or can handle multiple requests without side effects. If your application requires database setup or authentication, you need to handle that in the server command or inject it via the javascript block.

Trade-offs and Failure Modes

Aspect	Benefit	Risk
Declarative YAML	Easy for agents to generate	Limited expressiveness for complex flows
Clipboard mock	Works in headless mode	Diverges from real browser behavior
Subprocess server	Isolated test environment	No control over shutdown timing
Pydantic validation	Catches errors early	Schema changes break existing storyboards
Help-as-docs	Always in sync with code	Verbose help text confuses human users

The biggest failure mode is timing. The pause directives are hardcoded delays, not adaptive waits. If the server is slow or the page takes longer to render, the recording will capture incomplete interactions. The wait_for selectors mitigate this, but only if you know which elements to wait for.

Another issue: the clipboard mock only works for JavaScript-based clipboard APIs. Native browser clipboard events (Ctrl+C, Ctrl+V) won’t trigger the mock. This limits the types of interactions you can test.

Technical Verdict

Use shot-scraper video when:

Agents need to generate version-controlled, repeatable demos without human intervention
Your interactions are deterministic sequences of clicks, fills, and waits
You want help-as-docs pattern to reduce agent context window overhead
Server startup and teardown can be scripted reliably
You can define success criteria as CSS selectors or text content
Output format (WebM or MP4) compatibility meets your distribution needs

Avoid this pattern when:

Your application requires complex authentication flows that can’t be mocked via JavaScript injection
Interactions depend on dynamic content that changes between runs (real-time data, randomized UIs)
You need to test native OS clipboard behavior or keyboard shortcuts
Recording overhead affects application performance in ways that invalidate the demo
You need adaptive waits based on network conditions or server load
Timing-sensitive async operations make hardcoded pause values unreliable

The timing fragility is the main constraint. If your demo requires precise synchronization with asynchronous operations, you’ll spend more time tuning pause values than writing the interactions. In those cases, imperative Playwright scripts with explicit wait conditions give you better control.

For multi-agent orchestration, storyboard versioning in git enables agents to discover and reuse demos across projects. One agent can commit a storyboard, and another can retrieve and modify it without needing to understand the underlying Playwright API.

Use shot-scraper video when you want agents to show their work without manual editing, and you can define interactions as a predictable sequence of clicks, fills, and waits.

Agentic AI, minus the noise.