mech.app

The mech.app newsletter

Agentic AI, minus the noise.

Get practical field notes on AI agents, automation, developer tools and security delivered to your inbox.

No spam. Unsubscribe anytime.

Security

Building Security Agents That Cannot Escape Their Trust Boundary

How capability tokens, sandboxed execution, and immutable policy layers prevent agents from escalating privileges or accessing out-of-scope infrastructure.

Source: cynative.com
Building Security Agents That Cannot Escape Their Trust Boundary

Giving an agent read-write access to your infrastructure is a trust problem, not just a prompt injection problem. Even if the model never hallucinates and never gets tricked, server-side IAM drift and creative tool-call chaining can let an agent escalate its own privileges. The question is how to encode a trust boundary that survives both adversarial prompts and the agent’s own exploratory behavior.

Cynative’s approach treats the trust boundary as an immutable policy layer enforced outside the agent’s execution context. The agent gets a capability token scoped to specific resources and operations. That token is validated server-side before any tool call executes. The agent cannot modify the token, cannot request a broader one, and cannot discover what other tokens exist.

Why Read-Only Access Is Not Enough

Most security products mirror infrastructure into a data lake and give agents read-only access. This avoids the risk of an agent deleting resources or changing configurations, but it introduces new problems:

  • Sync delay: The data lake is always stale. An attacker who compromises a resource has a window before the agent sees it.
  • Partial data: Not every API surface gets mirrored. Agents miss runtime state, ephemeral resources, and vendor-specific metadata.
  • Data sovereignty: Mirroring sensitive infrastructure into a third-party SaaS product is a compliance risk.
  • Operational overhead: You need pipelines, schema mappings, and reconciliation logic to keep the mirror accurate.

The alternative is to give the agent direct access to live infrastructure. That requires a trust boundary that prevents the agent from changing anything it should not touch, even if it tries.

Capability Tokens vs. API Keys

A traditional API key grants access to an entire service. If an agent has your AWS credentials, it can call any API you can call. Capability tokens are different:

  • Scoped by resource: A token might grant access to a specific S3 bucket or a single EC2 instance, not the entire account.
  • Scoped by operation: A token might allow DescribeInstances but not TerminateInstances.
  • Time-bound: Tokens expire. An agent cannot hoard credentials for later use.
  • Non-discoverable: The agent cannot list all tokens or request a broader one. It only knows about the token it was given.

The token is validated server-side before any tool call executes. The agent’s prompt, tool-call arguments, and reasoning chain are irrelevant. If the token does not authorize the operation, the call fails.

Architecture: Immutable Policy Layer

Here is how the trust boundary works in practice:

  1. Agent initialization: The orchestrator generates a capability token scoped to the agent’s mission. For example, “scan EC2 instances in us-east-1 for unencrypted volumes.”
  2. Tool call: The agent decides to call DescribeVolumes. It includes the capability token in the request.
  3. Policy enforcement: The server checks the token against an immutable policy store. The policy says the token can call DescribeVolumes in us-east-1 but nothing else.
  4. Execution: If the policy allows it, the tool call executes. If not, the server returns an error. The agent cannot retry with a different token or escalate its privileges.

The policy store is immutable from the agent’s perspective. The agent cannot write to it, cannot read other policies, and cannot discover what operations it is not allowed to perform.

Sandboxing the Execution Environment

Capability tokens constrain authorization. Sandboxing constrains execution. The two are complementary.

A sandbox limits what the agent can do even if it has valid credentials:

  • Network isolation: The agent cannot make outbound requests except to approved endpoints.
  • Filesystem isolation: The agent cannot read or write files outside its working directory.
  • Process isolation: The agent cannot spawn new processes or access the host OS.

Sandboxing prevents the agent from exfiltrating data, installing backdoors, or pivoting to other systems. It does not prevent the agent from making authorized API calls that have unintended side effects. That is what capability tokens are for.

Preventing Scope Escape

Agents are creative. If you give them a tool that can modify their own authorization scope, they will find a way to use it. Common escape vectors:

  • IAM policy modification: The agent calls AttachUserPolicy to grant itself broader permissions.
  • Role assumption: The agent assumes a role with more privileges than its original token.
  • Credential discovery: The agent reads environment variables, metadata endpoints, or configuration files to find other credentials.
  • Tool chaining: The agent calls a sequence of tools that individually seem safe but together escalate privileges.

The defense is to remove these tools from the agent’s toolbox. If the agent’s mission does not require modifying IAM policies, do not give it a tool that can do that. If the agent needs to assume a role, validate the role ARN against the capability token before allowing the call.

Implementation: Token Validation Flow

Here is a minimal implementation of server-side token validation:

import jwt
from datetime import datetime, timezone

class CapabilityToken:
    def __init__(self, token: str, secret: str):
        self.claims = jwt.decode(token, secret, algorithms=["HS256"])
    
    def allows(self, resource: str, operation: str) -> bool:
        if datetime.now(timezone.utc) > datetime.fromisoformat(self.claims["exp"]):
            return False
        
        for grant in self.claims.get("grants", []):
            if grant["resource"] == resource and operation in grant["operations"]:
                return True
        
        return False

def execute_tool_call(token: str, resource: str, operation: str, args: dict):
    cap = CapabilityToken(token, SECRET_KEY)
    
    if not cap.allows(resource, operation):
        raise PermissionError(f"Token does not allow {operation} on {resource}")
    
    # Execute the actual API call
    return call_aws_api(resource, operation, args)

The token is a signed JWT. The claims include a list of grants, each specifying a resource and a set of allowed operations. The server checks the token before executing any tool call. The agent cannot forge a token because it does not have the signing key.

Trade-Offs: Trust Boundaries vs. Operational Flexibility

ApproachSecurityLatencyOperational OverheadData Sovereignty
Data lake mirroringHigh (read-only)Medium (sync delay)High (pipelines, schema mapping)Low (third-party SaaS)
Capability tokensHigh (scoped write)Low (direct access)Medium (token management)High (on-prem execution)
Traditional API keysLow (full access)Low (direct access)Low (no extra infra)High (on-prem execution)
Human-in-the-loopVery high (manual approval)Very high (human latency)Very high (manual review)High (on-prem execution)

Capability tokens strike a balance. They allow direct access to live infrastructure without granting full privileges. The operational overhead is token generation and validation, which is cheaper than maintaining a data lake.

Observability: Logging Every Boundary Check

You need to log every capability token validation, not just the failures. The log should include:

  • The token ID (not the token itself)
  • The requested resource and operation
  • Whether the token allowed the operation
  • The agent’s session ID
  • The timestamp

This log is your audit trail. If an agent behaves unexpectedly, you can reconstruct what it tried to do and whether the trust boundary stopped it. If the boundary failed, you can see which token was involved and revoke it.

Failure Modes

Even with capability tokens and sandboxing, things can go wrong:

  • Token leakage: If an attacker steals a capability token, they can use it until it expires. Rotate tokens frequently and monitor for unusual usage patterns.
  • Policy drift: If the policy store is mutable by other systems, an attacker might modify it to grant broader access. Treat the policy store as immutable infrastructure.
  • Tool-call chaining: An agent might chain together multiple allowed operations to achieve an unauthorized outcome. Log all tool calls and look for suspicious sequences.
  • Prompt injection: An attacker might trick the agent into requesting operations it should not. Capability tokens mitigate this because the server validates the token, not the prompt.

Technical Verdict

Use capability tokens when you need to give an agent direct access to live infrastructure but cannot afford to grant full privileges. This applies to security agents scanning cloud resources, trading agents executing orders within a risk budget, and automation agents provisioning infrastructure within a specific account.

Avoid capability tokens if your agent’s mission is exploratory and you do not know in advance which resources it will need. In that case, use a data lake or human-in-the-loop approval. Also avoid them if your infrastructure does not support fine-grained IAM policies. Capability tokens are only as strong as the underlying authorization system.

The trust boundary is not a replacement for sandboxing, observability, or human oversight. It is one layer in a defense-in-depth strategy. The goal is to make it hard for an agent to escape its intended scope, even if it tries.