A16Y launched an endpoint detection and response (EDR) product for AI agents. The pitch is familiar to anyone who has managed traditional endpoint security: discover shadow IT, assess risk, enforce runtime guardrails, and provide a kill switch. The difference is that the endpoints are not laptops running Chrome. They are autonomous agents with API credentials, decision loops, and the ability to execute trades, approve payments, or modify production infrastructure.
The problem is real. Agents accumulate permissions, spin up unvetted MCP servers, and execute tool calls without human approval. A single prompt injection or runaway loop can escalate into a financial loss or data breach before anyone notices. Traditional EDR tools see process execution and network traffic. They do not see agent reasoning traces, tool invocations, or the difference between legitimate exploration and malicious drift.
The Runtime Enforcement Problem
Agents are non-deterministic. You cannot write a static policy that says “block all trades over $10,000” because the agent might need to execute a legitimate $15,000 hedge. You cannot block all file writes because the agent might need to log state for recovery. The enforcement layer must understand context: what the agent is trying to do, why, and whether that behavior matches historical baselines.
A16Y’s approach is a three-layer control stack:
- Discovery layer: Inventory all agents, MCP servers, skills, and plugins across workstations. Identify hardcoded secrets, exposed configs, and unmanaged credentials.
- Governance layer: Define policies for tool usage, authentication methods, and data access. Centralize identity and audit trails.
- Runtime enforcement: Inline controls that can halt agent execution, block unauthorized tool calls, and enforce budget limits.
The hard part is the third layer. You need to instrument the agent runtime without breaking the decision loop. If you inject a policy check before every tool call, you add latency. If you check only at the start of a session, you miss mid-execution drift. If you kill the agent during execution, you need to preserve state for forensic analysis and rollback.
Behavioral Baselines for Non-Deterministic Agents
Traditional EDR uses behavioral baselines: this process normally reads 10 MB of data per hour, so a sudden 1 GB read is anomalous. For agents, the equivalent is tool call patterns. An agent that normally invokes the trading API twice per session and suddenly makes 50 calls in 10 minutes is either exploring a new strategy or stuck in a loop.
The challenge is distinguishing legitimate exploration from malicious drift. Agents are designed to try new approaches. A spike in API calls might be a valid response to market volatility. A new tool invocation might be the agent adapting to a changed environment.
A16Y’s product likely uses a combination of:
- Static thresholds: Hard limits on API call counts, token budgets, and transaction values.
- Contextual baselines: Compare current behavior to historical patterns for this agent, this user, and this task type.
- Anomaly scoring: Flag deviations that exceed a statistical threshold, but do not auto-block. Route to human review.
The following strategies represent trade-offs between latency, accuracy, and coverage:
| Strategy | Latency | False Positives | Coverage | Rollback Complexity |
|---|---|---|---|---|
| Pre-call policy check | High (per-call overhead) | Low (explicit rules) | Narrow (known tools only) | Low (block before execution) |
| Post-call audit | Low (async logging) | High (detect after damage) | Wide (all tool calls) | High (undo transactions) |
| Inline anomaly scoring | Medium (scoring per call) | Medium (tuning required) | Wide (all behavior) | Medium (halt mid-session) |
| Budget enforcement | Low (counter increment) | Low (explicit limits) | Narrow (cost/count only) | Low (stop before breach) |
Most production systems will use a combination. Budget enforcement catches runaway loops. Pre-call checks block known-bad tools. Post-call audits provide forensic trails. Inline anomaly scoring catches novel attacks.
Kill-Switch Architecture
The kill switch is the most critical component. When an agent goes rogue, you need to halt execution immediately, preserve state for analysis, and prevent further damage. The architecture depends on where the agent runs:
Local agents (desktop MCP servers, IDE plugins):
- The EDR client runs as a privileged process on the workstation.
- It monitors agent API calls via OS-level hooks or SDK instrumentation.
- On kill switch activation, it terminates the agent process, captures memory dumps, and locks credentials.
Cloud agents (hosted orchestrators, serverless functions):
- The enforcement layer runs as a sidecar or API gateway.
- It proxies all tool calls and maintains a session state machine.
- On kill switch activation, it drops in-flight requests, snapshots state to durable storage, and revokes temporary credentials.
Hybrid agents (local reasoning, cloud tools):
- The enforcement layer must coordinate between local and cloud components.
- A kill switch activation must propagate to both sides and ensure no orphaned transactions.
The state preservation requirement is non-trivial. You need to capture:
- The agent’s reasoning trace (what it was trying to do).
- The tool call sequence (what it actually did).
- The input prompt and any injected instructions.
- The credential set and permission scope.
- The transaction state (committed, pending, or rolled back).
This is forensic data. You need it to answer: was this a prompt injection, a logic bug, or a legitimate action that violated policy?
Code Example: Budget Enforcement Hook
Here is a simplified budget enforcement hook for a Python agent runtime. It tracks token usage and API call counts, and raises an exception when limits are exceeded.
class BudgetExceededError(Exception):
pass
class BudgetEnforcer:
def __init__(self, max_tokens=100_000, max_calls=50):
self.max_tokens = max_tokens
self.max_calls = max_calls
self.tokens_used = 0
self.calls_made = 0
self.call_log = []
def before_tool_call(self, tool_name, args):
if self.calls_made >= self.max_calls:
raise BudgetExceededError(
f"Call limit exceeded: {self.calls_made}/{self.max_calls}"
)
self.calls_made += 1
self.call_log.append({
"tool": tool_name,
"args": args,
"timestamp": time.time()
})
def after_llm_call(self, prompt_tokens, completion_tokens):
total = prompt_tokens + completion_tokens
if self.tokens_used + total > self.max_tokens:
raise BudgetExceededError(
f"Token limit exceeded: {self.tokens_used + total}/{self.max_tokens}"
)
self.tokens_used += total
def snapshot(self):
return {
"tokens_used": self.tokens_used,
"calls_made": self.calls_made,
"call_log": self.call_log
}
# Usage in synchronous agent loop
enforcer = BudgetEnforcer(max_tokens=50_000, max_calls=20)
try:
for step in agent.run():
enforcer.before_tool_call(step.tool, step.args)
result = step.execute()
enforcer.after_llm_call(step.prompt_tokens, step.completion_tokens)
except BudgetExceededError as e:
forensic_data = enforcer.snapshot()
log_incident(forensic_data, error=str(e))
halt_agent()Production systems must handle async tool calls via queue-based state snapshots, distributed enforcement across multiple runtime instances, and credential revocation coordinated with identity providers. The pattern remains the same: instrument the agent loop, track resource usage, and halt on breach.
Shadow AI Discovery
The discovery layer is less glamorous but equally important. Employees install agents, MCP servers, and plugins without IT approval. These tools accumulate credentials, access sensitive data, and create attack surface. The EDR product must:
- Scan workstations for installed agents (process names, file paths, registry keys).
- Identify MCP servers by port scanning and protocol fingerprinting.
- Extract configuration files and look for hardcoded secrets (API keys, database passwords).
- Map credential usage to understand what each agent can access.
This is traditional endpoint inventory work, adapted for agent-specific artifacts. The output is a risk report: which agents are running, what they can do, and where the exposure points are.
Operational Constraints
Latency overhead: Inline enforcement adds latency to every tool call. If the agent makes 100 API calls per session, and each policy check adds 50ms, you have added 5 seconds to the session. This is tolerable for long-running workflows but breaks real-time agents.
False positives: Anomaly detection will flag legitimate behavior as suspicious. A trading agent that suddenly makes 50 calls might be responding to a flash crash. A code agent that writes 1,000 files might be scaffolding a new project. Human review is required, which defeats the purpose of autonomous execution.
State consistency: Halting an agent during execution is hard. If the agent has already executed a trade but not logged the result, the kill switch must either roll back the trade or ensure the log is written. This requires coordination with external systems (trading APIs, databases) that may not support transactional semantics.
Credential sprawl: Agents accumulate credentials over time. The EDR product must track which credentials are in use, revoke them on kill switch activation, and prevent reuse. This requires integration with identity providers, secret managers, and API gateways.
Technical Verdict
Use this approach when:
- You are deploying agents with access to financial APIs, payment rails, or production infrastructure.
- You need audit trails and forensic data for compliance or incident response.
- You have a mix of local and cloud agents and need centralized policy enforcement.
- You are willing to accept some latency overhead in exchange for runtime safety.
Avoid this approach when:
- Your agents are read-only or operate in sandboxed environments with no external access.
- You need real-time performance and cannot tolerate per-call policy checks.
- Your agents are fully deterministic and can be validated with static analysis.
- You have a small number of agents and can rely on manual review and approval workflows.
The EDR model makes sense for agents because the risk profile is similar to traditional endpoints. Both accumulate permissions, both can be compromised, and both require continuous monitoring. The enforcement layer must be smarter about context and intent than traditional endpoint security. Budget limits and kill switches are table stakes. The real engineering challenge is building behavioral baselines that distinguish malicious drift from legitimate adaptation without breaking the agent’s ability to explore and learn.