GitHub shipped a production post-mortem that contradicts the core assumption driving most agent frameworks: more powerful tools do not always make agents better. When they upgraded Copilot’s code review capabilities with richer API calls, performance degraded. The fix required reshaping the entire workflow around Unix-style composable utilities that expose pull request evidence incrementally.
This is not a theoretical debate. GitHub runs code review agents at scale across millions of repositories. The lessons here apply to anyone building multi-step agent workflows where tool boundaries matter more than tool power.
The Tool Upgrade That Broke Everything
GitHub’s original code review agent used monolithic API calls. A single tool invocation would fetch file diffs, commit metadata, and review context in one shot. When they upgraded to more granular, feature-rich tools (better error handling, more filtering options, richer return types), the agent started making worse decisions.
The problem was not the tools themselves. The problem was that the agent now had to orchestrate more complex decision trees. Each tool call introduced branching logic: should I filter by file type first? Should I fetch metadata before or after reading diffs? The LLM spent tokens reasoning about tool sequencing instead of reasoning about code quality.
Key failure modes:
- Agents over-fetched data because they could not predict which subset would be relevant
- Tool call sequences became non-deterministic (different runs used different orders)
- Context windows filled with orchestration logic instead of code evidence
- Cost per review increased despite better tooling
Unix Pipes for Agent Workflows
GitHub’s fix was architectural. They rebuilt the tool layer around composable utilities that mimic Unix pipes. Each tool does one thing, accepts structured input, and emits structured output that the next tool can consume.
Before (monolithic):
# Single tool call with many parameters
review_context = fetch_pr_data(
pr_id=123,
include_diffs=True,
include_commits=True,
include_comments=True,
filter_by=["*.py", "*.js"],
max_files=50
)After (composable):
# Chain of single-purpose tools
files = list_changed_files(pr_id=123)
filtered = filter_by_extension(files, [".py", ".js"])
diffs = fetch_diffs(filtered[:10])
context = build_review_context(diffs)The difference is not just syntactic. The Unix-style approach forces the agent to build evidence incrementally. Each tool call produces a small, inspectable artifact. The agent can decide whether to continue the chain or pivot based on what it has learned so far.
Incremental Evidence Gathering
The core insight is that code review is not a single retrieval problem. It is a search problem where the agent refines its understanding of what matters as it explores the pull request.
GitHub’s new workflow treats the pull request as a graph of evidence nodes. The agent starts with high-level metadata (which files changed, how many lines, commit messages). Based on that, it decides which files to inspect. Based on file diffs, it decides which functions to analyze in detail.
Evidence flow:
- Metadata scan: PR title, description, file list, line counts
- Triage: Identify high-risk files (large diffs, critical paths, test coverage gaps)
- Targeted diff fetch: Pull only the diffs for triaged files
- Contextual analysis: Fetch surrounding code for flagged functions
- Review synthesis: Generate comments based on accumulated evidence
This is not a waterfall. The agent can loop back. If a function call in one file references another file, the agent fetches that file’s context on demand. The workflow is a directed acyclic graph (DAG) of tool calls, not a linear script.
State Management and Observability
The Unix-style approach simplifies state management. Each tool call is a pure function: same input always produces same output. The agent’s state is just the accumulated list of tool outputs. You can serialize the entire workflow as a JSON array of tool calls and results.
Observability wins:
- Replay: Re-run any workflow by replaying the tool call sequence
- Debugging: Inspect the exact evidence the agent saw at each decision point
- Cost attribution: Measure token usage per tool, not per entire review
- Failure isolation: If one tool call fails, the rest of the chain is unaffected
GitHub logs every tool call with input, output, latency, and token count. They can reconstruct why an agent made a specific comment by walking the evidence chain backward from the comment to the tool calls that produced the relevant diffs.
Cost Reduction Mechanics
The cost savings came from three sources:
- Fewer tokens in prompts: Agents no longer need to reason about complex tool parameters. The tool interface is simple: take this input, produce that output.
- Less over-fetching: Agents fetch only the diffs they need, not the entire PR upfront.
- Better caching: Because tool outputs are deterministic, GitHub can cache results across reviews. If two PRs touch the same file, the diff fetch is cached.
GitHub reports a measurable reduction in review cost, though they do not publish exact numbers. The key metric is cost per useful comment. Agents now generate fewer low-value comments (style nits, obvious observations) and more high-value comments (logic bugs, security issues, performance regressions).
Tool Design Trade-offs
| Dimension | Monolithic Tools | Unix-Style Tools |
|---|---|---|
| Orchestration complexity | Low (single call) | Higher (multi-step chains) |
| Agent reasoning load | High (complex parameters) | Low (simple interfaces) |
| Over-fetching risk | High (fetch everything) | Low (fetch on demand) |
| Observability | Opaque (black box call) | Transparent (inspectable chain) |
| Caching effectiveness | Low (unique calls) | High (reusable outputs) |
| Failure blast radius | Large (entire call fails) | Small (one step fails) |
The trade-off is real. Unix-style tools require more orchestration logic. You need a workflow engine that can execute DAGs, handle retries, and manage partial failures. But the payoff is that the agent’s job becomes simpler. It reasons about evidence, not about API design.
When This Approach Fails
Unix-style composition works when the problem is exploratory and the agent needs to refine its understanding incrementally. It does not work when:
- Latency matters more than cost: If you need sub-second responses, multiple tool calls add up. A single monolithic call is faster.
- The problem is well-defined: If you always need the same data, fetching it in one shot is more efficient.
- The agent is not capable of planning: Weak models struggle with multi-step workflows. They need simple, high-level tools.
GitHub’s code review agent uses GPT-4 class models. Smaller models (GPT-3.5, open-source alternatives) may not handle the orchestration overhead. You would need to bake more of the workflow into the tool layer itself, which brings you back to monolithic tools.
Implementation Sketch
Here is a minimal workflow engine that executes Unix-style tool chains:
from typing import Any, Callable, List, Dict
class ToolChain:
def __init__(self):
self.tools: Dict[str, Callable] = {}
self.trace: List[Dict[str, Any]] = []
def register(self, name: str, func: Callable):
self.tools[name] = func
def execute(self, plan: List[Dict[str, Any]]) -> Any:
context = {}
for step in plan:
tool_name = step["tool"]
args = step["args"]
# Resolve references to previous outputs
resolved_args = self._resolve(args, context)
result = self.tools[tool_name](**resolved_args)
context[step["output_key"]] = result
self.trace.append({
"tool": tool_name,
"args": resolved_args,
"result": result
})
return context
def _resolve(self, args: Dict, context: Dict) -> Dict:
resolved = {}
for key, value in args.items():
if isinstance(value, str) and value.startswith("$"):
resolved[key] = context[value[1:]]
else:
resolved[key] = value
return resolved
# Usage
chain = ToolChain()
chain.register("list_files", lambda pr_id: ["main.py", "test.py"])
chain.register("filter_files", lambda files, ext: [f for f in files if f.endswith(ext)])
chain.register("fetch_diff", lambda files: {f: f"diff for {f}" for f in files})
plan = [
{"tool": "list_files", "args": {"pr_id": 123}, "output_key": "files"},
{"tool": "filter_files", "args": {"files": "$files", "ext": ".py"}, "output_key": "filtered"},
{"tool": "fetch_diff", "args": {"files": "$filtered"}, "output_key": "diffs"}
]
result = chain.execute(plan)
print(result["diffs"])The agent generates the plan (the list of tool calls). The workflow engine executes it and handles variable resolution. You can extend this with retries, caching, and parallel execution.
Security Boundaries
Unix-style tools make security boundaries explicit. Each tool is a separate function with its own permission scope. You can enforce that the agent cannot call delete_file without first calling request_approval. The workflow engine checks permissions at each step.
With monolithic tools, permissions are implicit. A single tool might have broad access (read files, write comments, trigger CI). If the agent misuses it, the blast radius is large. With composable tools, you can limit each tool to the minimum required permission.
Technical Verdict
Use Unix-style composable tools when:
- Your agent needs to explore a problem space incrementally (code review, debugging, research)
- You need fine-grained observability and cost attribution
- You are using capable models (GPT-4 class or better) that can handle multi-step planning
- Latency is not your primary constraint (you can tolerate multiple round trips)
Avoid this approach when:
- You need sub-second response times
- The problem is well-defined and always requires the same data
- You are using weaker models that struggle with orchestration
- Your tools have high per-call overhead (network latency, authentication)
GitHub’s lesson is that tool power is not the same as tool usability. Agents perform better when tools are simple, composable, and aligned with the agent’s reasoning process. The Unix philosophy (do one thing well, compose via pipes) applies to agent workflows just as it applies to shell scripts.