mech.app
Dev Tools

SkillOpt: Training Agent Skills Like Neural Networks Without Touching Weights

Text-space optimizer applies epochs, batches, and validation gates to frozen LLM agents through trajectory-driven skill edits and deployable artifacts.

Source: github.com
SkillOpt: Training Agent Skills Like Neural Networks Without Touching Weights

Microsoft’s SkillOpt (11,620 stars, trending #5 in Python) brings neural-network training concepts to frozen LLM agents. Instead of gradient descent on model weights, it optimizes natural-language skill descriptions through trajectory-driven edits, validation gates, and deployable best_skill.md artifacts. The v0.2.0 release ships SkillOpt-Sleep, a nightly self-evolution engine that harvests production traces, mines failure patterns, replays dream rollouts, and consolidates updates behind a held-out validation gate.

This is not fine-tuning. The base model stays frozen. SkillOpt treats the skill document (a Markdown file) as the trainable parameter space and applies epochs, mini-batches, and learning rates to text edits.

How Text-Space Optimization Works

SkillOpt runs a training loop over agent trajectories:

  1. Collect trajectories: Execute the agent on a batch of tasks, recording tool calls, intermediate states, and final outcomes.
  2. Compute reward: Multi-objective scoring (task success, token efficiency, tool-use correctness).
  3. Generate skill edits: A meta-agent (often GPT-4 or Claude) proposes natural-language changes to the skill document based on failure patterns.
  4. Validate: Run the updated skill on a held-out validation set. Accept the edit only if validation performance improves.
  5. Repeat: Iterate for N epochs, adjusting the edit proposal temperature (learning rate analog) and batch size.

The skill document follows the agentskills.io SKILL.md standard, so no format conversion is needed. A typical skill includes:

  • Task description and success criteria
  • Tool-use patterns and examples
  • Error-handling strategies
  • State-management conventions

Architecture: Training Harness and Validation Pipeline

┌─────────────────────────────────────────────────────────────┐
│  Training Harness                                           │
│  ┌─────────────┐   ┌──────────────┐   ┌─────────────────┐ │
│  │ Task Batch  │──▶│ Agent Exec   │──▶│ Trajectory Log  │ │
│  │ (mini-batch)│   │ (frozen LLM) │   │ (tool calls +   │ │
│  └─────────────┘   └──────────────┘   │  states)        │ │
│                                        └────────┬────────┘ │
│                                                 │          │
│                    ┌────────────────────────────▼────────┐ │
│                    │ Reward Scorer                       │ │
│                    │ (multi-objective: success, tokens,  │ │
│                    │  tool correctness)                  │ │
│                    └────────────────┬────────────────────┘ │
│                                     │                      │
│  ┌──────────────────────────────────▼──────────────────────┐  │
│  │ Meta-Agent (GPT-4 / Claude)                         │  │
│  │ Proposes skill edits based on failure patterns      │  │
│  └──────────────────────┬──────────────────────────────────┘  │
│                         │                                  │
│  ┌──────────────────────▼──────────────────────────────────┐  │
│  │ Validation Gate                                     │  │
│  │ • Run updated skill on held-out tasks               │  │
│  │ • Accept edit if val_score > prev_val_score         │  │
│  │ • Reject and revert if degradation detected         │  │
│  └──────────────────────┬──────────────────────────────────┘  │
│                         │                                  │
│  ┌──────────────────────▼──────────────────────────────────┐  │
│  │ best_skill.md artifact (deployable)                 │  │
│  └─────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

The validation gate is the critical safety boundary. Without it, the meta-agent can propose edits that improve training performance but degrade generalization. The held-out validation set acts like a test set in supervised learning, preventing overfitting to the training batch.

SkillOpt-Sleep: Nightly Self-Evolution Engine

The v0.2.0 release introduces SkillOpt-Sleep, a background process that runs the full training loop on production traces:

  1. Harvest: Collect agent trajectories from the past 24 hours (or configurable window).
  2. Mine: Extract failure patterns, tool-use errors, and state-management bugs.
  3. Replay: Re-execute failed tasks with candidate skill edits.
  4. Consolidate: Merge accepted edits into the skill document, maintaining a version history.

SkillOpt-Sleep runs as a CLI (skillopt-sleep) and supports multi-objective reward functions:

# Example reward configuration (YAML)
reward:
  success_weight: 1.0
  token_efficiency_weight: 0.3
  tool_correctness_weight: 0.5
  latency_penalty: 0.1

The engine also supports experience replay (re-training on historical trajectories) and dream rollouts (synthetic task generation to explore edge cases). Both techniques borrow from reinforcement learning but operate in text space rather than parameter space.

Cross-Tool Backend Support

SkillOpt v0.2.0 ships plugin shells for:

  • Claude (Anthropic API)
  • Codex (OpenAI Codex API)
  • GitHub Copilot (via VSCode extension protocol)
  • Devin (Cognition Labs agent runtime)
  • OpenClaw (open-source agent framework)

Each backend implements a common interface:

class AgentBackend:
    def execute_task(self, skill: str, task: dict) -> Trajectory:
        """Run agent with given skill on task, return trajectory."""
        pass
    
    def validate(self, skill: str, val_tasks: list) -> float:
        """Compute validation score on held-out tasks."""
        pass

This abstraction lets you train skills on one backend (e.g., Claude) and deploy on another (e.g., Devin), as long as both support the SKILL.md format.

Deployment Shape and Artifacts

SkillOpt produces two primary artifacts:

  1. best_skill.md: The highest-scoring skill document from the training run. This is a plain Markdown file that can be versioned in Git, reviewed by humans, and deployed to any agent runtime that supports the SKILL.md standard.
  2. training_log.jsonl: Line-delimited JSON with per-epoch metrics (training reward, validation reward, edit acceptance rate, token usage).

Deployment options:

Deployment ModeUse CaseTrade-offs
Static skill fileDeterministic agent behavior, human review requiredNo runtime adaptation, manual update cycle
SkillOpt-Sleep daemonContinuous self-evolution, nightly updatesRisk of silent degradation if validation gate fails
Hybrid (staged rollout)Train offline, deploy after human approvalBest of both worlds, higher operational overhead

The validation gate is your primary defense against skill degradation. If the gate fails (e.g., validation set is too small or unrepresentative), SkillOpt-Sleep can silently degrade agent performance.

Observability and Failure Modes

SkillOpt exposes metrics through the training log:

  • Edit acceptance rate: Percentage of proposed edits that pass validation. Low acceptance (< 20%) suggests the meta-agent is proposing poor edits or the validation set is too strict.
  • Validation score delta: Change in validation performance per epoch. Negative delta triggers an automatic rollback.
  • Token usage per epoch: Tracks meta-agent cost. High token usage (> 100k tokens/epoch) indicates the meta-agent is generating verbose or redundant edits.

Common failure modes:

  1. Validation set contamination: If validation tasks leak into the training batch, the validation gate becomes ineffective. Watch for validation score delta near zero despite low edit acceptance rate. Solution: Enforce strict train/val splits and rotate validation tasks periodically.
  2. Meta-agent hallucination: The meta-agent proposes edits that sound plausible but introduce subtle bugs (e.g., incorrect tool parameter names). Manifests as high edit acceptance rate but declining production success rate. Solution: Add a syntax checker and tool schema validator before the validation gate.
  3. Overfitting to training batch: The skill becomes hyper-specialized to the training tasks and fails on novel inputs. Shows as increasing training reward but flat or declining validation score. Solution: Increase validation set size and diversity, or use dream rollouts to explore edge cases.

Code Example: Running a Training Loop

from skillopt import SkillOptimizer, AgentBackend
from skillopt.backends import ClaudeBackend

# Initialize backend and optimizer
backend = ClaudeBackend(api_key="your-key")
optimizer = SkillOptimizer(
    backend=backend,
    initial_skill_path="skills/initial_skill.md",
    validation_split=0.2,
    epochs=10,
    batch_size=32,
    learning_rate=0.1  # controls edit proposal temperature
)

# Load training tasks
tasks = load_tasks("data/training_tasks.jsonl")

# Run training loop
best_skill, metrics = optimizer.train(tasks)

# Save artifacts
best_skill.save("skills/best_skill.md")
metrics.to_jsonl("logs/training_log.jsonl")

The learning_rate parameter controls the meta-agent’s edit proposal temperature. Higher values (0.3-0.5) encourage more aggressive edits, lower values (0.05-0.1) favor conservative refinements.

Security Boundaries

SkillOpt’s security surface includes:

  • Meta-agent prompt injection: If training tasks contain adversarial inputs, the meta-agent might propose malicious skill edits. Mitigation: Sanitize training data and add a human-in-the-loop review step before deploying updated skills.
  • Validation gate bypass: If an attacker controls the validation set, they can force acceptance of degraded skills. Mitigation: Store validation tasks in a separate, access-controlled repository.
  • Artifact tampering: The best_skill.md file is plain text and can be modified outside the training loop. Mitigation: Sign skill artifacts with a cryptographic signature and verify before deployment.

Technical Verdict

Use SkillOpt when:

  • You have a frozen LLM agent (no access to model weights or fine-tuning budget) and want to improve task-specific performance through skill refinement.
  • You can collect agent trajectories at scale (minimum 500 task executions per training run, ideally 2,000+).
  • Your validation set is at least 10% of training set size and at least 50 tasks. Smaller validation sets produce unreliable validation gates.
  • Your production task distribution is stable. If task types shift by more than 30% (measured by embedding distance or manual clustering), retrain from scratch rather than continuing self-evolution.
  • You need a deployable artifact (Markdown file) that can be versioned, reviewed, and rolled back.
  • Your baseline agent success rate is above 40%. Below that threshold, SkillOpt struggles to extract useful patterns from mostly-failed trajectories.

Avoid SkillOpt when:

  • Your agent already has access to fine-tuning or retrieval-augmented generation (RAG) pipelines that can adapt to new tasks without skill document edits.
  • You lack a representative validation set. Without a strong validation gate, SkillOpt-Sleep can silently degrade agent performance.
  • Your tasks are too diverse for a single skill document. SkillOpt works best when training tasks share common patterns (e.g., all tasks involve API calls to the same service). If task clustering reveals more than five distinct task types, consider training separate skills per cluster.
  • You need real-time adaptation. SkillOpt’s training loop runs offline (minutes to hours per epoch), so it cannot respond to live production failures.
  • Your validation set is smaller than 50 tasks or less than 5% of training set size. The validation gate will produce false positives (accepting bad edits) or false negatives (rejecting good edits).

The validation gate is the linchpin. If you cannot construct a held-out validation set that accurately predicts production performance, SkillOpt becomes a liability.