Porting PyTorch to ONNX + WebGPU: Browser Inference Without a Server

Simon Willison just published a walkthrough of porting a 0.2B parameter PyTorch inpainting model to run entirely in-browser using ONNX Runtime Web and WebGPU. The project exposes the full pipeline from model export to client-side inference, including how CacheStorage handles 1.3GB model files and what breaks when Hugging Face redirects interfere with browser caching.

This is not a demo. It is a working deployment at simonw.github.io/moebius-web that loads 1.24GB of ONNX weights from Hugging Face and runs inference in Chrome, Firefox, and Safari without touching a server.

The Export Pipeline

Converting PyTorch to ONNX requires explicit handling of dynamic axes and opset versioning. The torch.onnx.export call looks like this:

torch.onnx.export(
    dec,
    (lat,),
    dec_path,
    opset_version=18,
    input_names=["latent"],
    output_names=["image"],
    dynamic_axes={
        "latent": {0: "B"},
        "image": {0: "B"}
    }
)

The dynamic_axes parameter tells ONNX that the batch dimension can vary at runtime. Without this, the exported model only accepts fixed-size inputs. The opset_version=18 pins the operator semantics: ONNX defines operators like Conv, MatMul, and Einsum in versioned sets, and opset 18 determines which variants are available and how they behave.

An ONNX file bundles two things:

A directed computation graph where nodes are operators (Conv, MatMul, Add, Einsum, Softmax, Gather, Resize) wired together by named tensors
The learned parameter tensors (convolution kernels, embedding tables) stored as initializers in the graph

The graph describes what to compute, not how or on what hardware. That abstraction is what lets ONNX Runtime Web execute the same model on WebGPU, WebGL, or WASM backends.

CacheStorage for 1.3GB Model Files

The initial deployment downloaded 1.3GB of weights on every page load. Hugging Face serves model files through redirects, which can bypass standard browser HTTP caching depending on cache headers and redirect chains.

The fix uses the CacheStorage API, the same mechanism Transformers.js uses for Whisper Web:

const cache = await caches.open("transformers-cache");
let response = await cache.match(url);

if (!response) {
    response = await fetch(url);
    await cache.put(url, response.clone());
}

CacheStorage is separate from the HTTP cache. It persists across sessions and works reliably with large files. The browser manages eviction based on storage quotas, but in practice 1.3GB stays cached until the user clears site data.

This works in Chrome, Firefox, and Safari. The storage quota varies by browser and available disk space, but all three handle multi-gigabyte caches without special configuration.

ONNX Runtime Web vs. PyTorch Execution

Server-side PyTorch runs on CUDA with full operator support and dynamic memory allocation. ONNX Runtime Web on WebGPU has constraints:

Aspect	PyTorch + CUDA	ONNX Runtime Web + WebGPU
Operator coverage	Full PyTorch API	Subset defined by opset version
Memory management	Dynamic allocation, unified memory	Fixed buffers, explicit transfers
Precision	FP32, FP16, mixed precision	FP32, FP16 (backend-dependent)
Debugging	Stack traces, tensor inspection	Limited: check browser console, WebGPU errors
Deployment	Requires GPU drivers, CUDA toolkit	Runs in any WebGPU-capable browser

The opset version matters because newer operators may not have WebGPU implementations yet. Opset 18 is a safe choice for broad compatibility, but cutting-edge PyTorch features may require opset 19+ and may not run in-browser.

WebGPU memory is separate from JavaScript heap. Transferring tensors between CPU and GPU is explicit and slow. ONNX Runtime Web hides this, but large models still pay the cost on first inference.

Agent Orchestration Flow

Willison ran two parallel Claude Code sessions: one working on a Datasette feature, another porting Moebius. The porting session had 5-10 minute iteration cycles while waiting for the main project to finish refactors.

The agent workflow:

Research phase: Claude.ai (with GitHub cloning) analyzed the Moebius repo and assessed feasibility. Output saved to research.md.
Preparation: Cloned Moebius source, model weights (via GIT_LFS_SKIP_SMUDGE=0), Transformers.js, and ONNX Runtime repos into /tmp/Moebius.
Execution: Claude Code worked in /tmp/Moebius/moebius-web, committing early and maintaining notes.md and plan.md as it progressed.
Deployment: Agent used hf CLI to publish ONNX weights to Hugging Face, then deployed the frontend to GitHub Pages.
Debugging: Human tested in Chrome, pasted errors and screenshots back to Claude Code, which iterated fixes.

The agent used a subagent to analyze the obfuscated Whisper Web demo and extract the CacheStorage pattern. This avoided burning top-level context on minified JavaScript.

Failure Modes and Observability Gaps

Browser compatibility: WebGPU is not universally available. Older browsers fall back to WebGL or WASM, which are slower and may not support all operators.

Model size limits: CacheStorage works for 1.3GB, but 10GB+ models may hit quota limits or cause out-of-memory errors during inference.

Opset mismatches: If the PyTorch model uses operators not in opset 18, export fails or produces a graph that ONNX Runtime Web cannot execute. The error messages are cryptic: “Unsupported operator X in opset Y.”

Redirect caching: Hugging Face redirects can break HTTP caching. CacheStorage fixes this, but only if you know to use it. The initial deployment reloaded 1.3GB every time.

Debugging: WebGPU errors appear in the browser console but do not map cleanly to ONNX graph nodes. Tensor inspection requires manual logging or external tools.

Deployment Shape

The final architecture:

Model weights: 1.24GB ONNX file hosted on Hugging Face at huggingface.co/simonw/Moebius-ONNX
Frontend: Static HTML/JS on GitHub Pages at simonw.github.io/moebius-web
Inference: ONNX Runtime Web with WebGPU backend, running entirely client-side
Caching: CacheStorage API for persistent model storage across sessions

No server. No API keys. No usage tracking. The user downloads 1.3GB once, then runs inference locally.

Technical Verdict

Use this approach when:

Your model is under 2GB and fits in browser storage quotas
You want zero server costs and zero latency after initial download
Your users have modern browsers with WebGPU support
Privacy matters: no data leaves the client

Avoid when:

Your model exceeds 5GB or requires operators not in opset 18
You need to support older browsers without WebGPU
Inference speed is critical: server GPUs are still faster
You want usage analytics or model versioning control

The CacheStorage pattern is the key insight. Without it, client-side ML is impractical for anything larger than toy models. With it, you can ship production-grade inference to the browser.