Velorn's MCP Server: Exposing Timeline Operations to AI Agents

Velorn is a GPLv3 desktop video editor that runs a local MCP server to expose timeline operations to agents like Codex and Claude Code. Instead of embedding a chatbot UI next to the timeline, it treats the agent as an operator with structured access to project state, media inspection, edit proposals, asset organization, and export preparation.

The architecture choice matters. Most AI-assisted creative tools put a chat window in the sidebar and let the agent suggest edits through natural language. Velorn inverts this: the agent calls structured operations on a running editor instance, inspects the actual timeline state, and performs mutations through a defined schema. The agent becomes a client of the editor’s internal API, not a conversational layer on top of it.

Architecture: Loopback-Only MCP Server

Velorn runs an MCP server bound to 127.0.0.1 only. The server exposes operations as tools that agents can discover and invoke. The editor maintains project state in Zustand stores, and the MCP server reads and writes that state through the same internal boundaries the UI uses.

Security boundary:

Server listens on loopback only, no network exposure
No authentication layer (local trust model)
Agent must run on the same machine as the editor
File system access is scoped to the active project directory
No remote execution or cloud relay

This model assumes the agent is a local process you trust. If you run Codex or Claude Code locally, they connect to localhost:PORT and call tools. The MCP server does not validate intent or rate-limit operations. It trusts the caller.

State synchronization:

The editor and MCP server share the same Electron process. When an agent calls an edit operation, the MCP handler updates the Zustand store, which triggers React re-renders in the UI. The timeline view reflects agent edits in real time. The agent can call inspect_timeline to read the current state before proposing the next mutation.

Operation Schema: Inspect vs Mutate

The MCP server exposes two categories of tools:

Inspection tools:

inspect_project: Returns project metadata, timeline structure, track list, and asset inventory
inspect_timeline: Returns clip positions, durations, layer order, and effect stack
inspect_media: Returns frame data, resolution, codec, and duration for a specific asset
inspect_frame: Returns a base64-encoded JPEG of a specific frame at a given timestamp

Mutation tools:

propose_edit: Returns a structured edit plan without applying it (agent can review before committing)
perform_edit: Applies a sequence of timeline operations (trim, split, move, delete, add effect)
add_caption: Inserts a text overlay at a specific timecode with styling parameters
organize_assets: Moves files into bins or tags them for later use
prepare_export: Configures export settings (codec, resolution, bitrate) and queues the render

The agent decides between inspect and mutate by reasoning about the current state. In the demo, Codex calls inspect_timeline first, sees an empty project, then calls organize_assets to create a structure, then perform_edit to add generated clips.

Edit operation structure:

{
  "tool": "perform_edit",
  "arguments": {
    "operations": [
      {
        "type": "add_clip",
        "track": 1,
        "start_time": 0.0,
        "asset_id": "gen_12345",
        "duration": 5.0
      },
      {
        "type": "add_effect",
        "clip_id": "clip_001",
        "effect": "fade_in",
        "params": { "duration": 1.0 }
      },
      {
        "type": "trim",
        "clip_id": "clip_001",
        "new_duration": 4.5
      }
    ]
  }
}

Each operation is atomic. If one fails (invalid asset ID, timeline conflict), the entire batch rolls back. The MCP server returns an error response with the failing operation index.

Rollback and Undo

Velorn maintains an undo stack in the Zustand store. Every mutation tool call creates a snapshot of the timeline state before applying changes. If an agent calls perform_edit with a batch of operations and one fails, the server restores the snapshot and returns an error.

Undo behavior:

Manual undo (Ctrl+Z in the UI) steps back through the history stack
Agent-initiated undo is not exposed as an MCP tool (design choice to prevent infinite loops)
The agent can call inspect_timeline to verify the result of an edit, then call another perform_edit to correct mistakes

The undo stack is bounded (default 50 steps). If the agent performs 100 edits in a session, the earliest 50 are pruned. This prevents memory growth in long-running sessions.

Failure modes:

Failure	Behavior	Recovery
Invalid asset ID	Rollback entire batch, return error	Agent re-inspects and retries
Timeline conflict (overlapping clips)	Rollback, return conflict details	Agent adjusts timing and retries
FFmpeg render failure	Export marked failed, project unchanged	Agent can adjust export settings
ComfyUI generation timeout	Generation job marked stale, no timeline change	Agent can retry or skip
MCP server crash	Editor remains running, agent loses connection	Restart MCP server, agent reconnects

Integration with ComfyUI and FFmpeg

Velorn uses FFmpeg for all media processing (decode, encode, thumbnail extraction) and ComfyUI for generation. The MCP server does not directly call these tools. Instead, it queues jobs in the editor’s task system, which manages subprocess lifecycle.

Generation workflow:

Agent calls prepare_generation with a ComfyUI workflow JSON and parameters
Editor queues the job and returns a job ID
Agent polls inspect_job_status until completion
Generated frames are imported as a new asset in the project
Agent calls perform_edit to add the asset to the timeline

ComfyUI runs as a separate process on localhost:8188. Velorn sends HTTP requests to the ComfyUI API. If ComfyUI is not running, generation tools return an error, but editing and MCP operations continue to work.

Export workflow:

Agent calls prepare_export with codec, resolution, and output path
Editor validates settings and queues the render
FFmpeg subprocess starts, writing to the output file
Agent can call inspect_export_progress to get frame count and ETA
On completion, the output file is available in the project directory

FFmpeg runs as a child process with stdout/stderr piped to the editor. The MCP server does not expose raw FFmpeg control. The agent configures exports through structured parameters, and the editor translates them to FFmpeg command-line arguments.

Demo: Agent-Driven Video Creation

In the Show HN demo, Codex was given minimal direction: “You’re live on YouTube, use Velorn to create a really cool motion graphics video.” The agent produced a 3.5-minute video by:

Inspecting the empty project
Creating asset bins for organization
Generating motion graphics clips via ComfyUI
Adding clips to the timeline with precise timing
Applying fade and transition effects
Adding captions with keyframed animations
Mixing audio levels across tracks
Exporting the final render

The agent made 47 MCP tool calls over 12 minutes of wall-clock time. The uncut recordings show the full reasoning trace, including backtracking when the agent realized a clip duration conflicted with the audio beat.

Key observations:

Agent used inspect_timeline after every mutation to verify state
Agent called propose_edit three times before committing to a final structure
Agent retried one export after realizing the codec settings were wrong
Agent did not use undo, it always inspected and corrected forward

Trade-Offs: Agent-as-Operator vs Agent-as-Copilot

Dimension	Agent-as-Operator (Velorn)	Agent-as-Copilot (Typical)
Control surface	Structured API with inspect/mutate tools	Natural language suggestions in chat UI
State visibility	Agent reads actual timeline state	Agent infers state from conversation history
Rollback	Automatic on batch failure, manual undo in UI	User manually reverts suggested changes
Latency	One tool call per operation	Multiple chat turns to clarify intent
Failure recovery	Agent retries with corrected parameters	User rephrases prompt and tries again
Observability	Tool call logs show exact operations	Chat transcript shows intent, not actions

The operator model gives the agent precise control but requires a well-defined operation schema. The copilot model is easier to implement (wrap existing UI actions in LLM calls) but harder to debug when the agent misunderstands intent.

Deployment Shape

Velorn ships as an Electron app with the MCP server embedded. Users download a single binary for Windows, macOS, or Linux. The MCP server starts automatically when the editor launches and stops when the editor quits.

Process topology:

┌─────────────────────────────────────┐
│ Electron Main Process               │
│  ├─ Zustand Store (project state)   │
│  ├─ MCP Server (localhost:PORT)     │
│  └─ Task Queue (FFmpeg, ComfyUI)    │
└─────────────────────────────────────┘
         │
         ├─ FFmpeg subprocess (render jobs)
         ├─ Whisper subprocess (transcription)
         └─ HTTP client → ComfyUI (generation)

┌─────────────────────────────────────┐
│ Agent (Codex, Claude Code)          │
│  └─ MCP Client → localhost:PORT     │
└─────────────────────────────────────┘

The agent runs as a separate process (typically in VS Code or a standalone CLI). It discovers the MCP server via a config file that specifies the port. The agent does not need to know about Electron, React, or Zustand. It only sees the MCP tool schema.

Observability

Velorn logs all MCP tool calls to a JSON file in the project directory. Each log entry includes:

Timestamp
Tool name
Arguments (sanitized to remove file paths)
Result or error
Execution time

The editor UI shows a live feed of MCP activity in a debug panel. Users can see which tools the agent is calling and inspect the arguments. This is critical for debugging when the agent produces unexpected results.

Metrics collected:

Tool call count by type
Average execution time per tool
Error rate by tool
Timeline state size (clip count, track count, effect count)
Export queue depth

These metrics are not sent anywhere. They are stored locally and can be exported as CSV for analysis.

Technical Verdict

Use Velorn’s MCP pattern when:

You have a stateful desktop app with complex operations (CAD, DAW, game engine)
You want agents to operate the app, not just suggest changes
You need precise control over what the agent can inspect and mutate
You can define a stable operation schema that maps to internal state
You trust the agent to run locally on the same machine

Avoid this pattern when:

Your app is primarily CRUD (MCP overhead is not worth it)
You need multi-user collaboration (loopback-only does not scale)
Your operation schema changes frequently (agent retraining cost is high)
You want the agent to run in the cloud (security boundary is wrong)
You prefer a conversational UX over structured tool calls

The loopback-only security model is the key constraint. If you need remote agents or multi-user access, you will need authentication, rate limiting, and audit logs. Velorn skips all of that by assuming local trust.

Agentic AI, minus the noise.