The agent platform market is selling three different products under one label. Teams are confusing frameworks (LangGraph, CrewAI) with managed runtimes (Claude Managed Agents, Bedrock AgentCore) and infrastructure control planes. Each solves a different problem. Each fails in different ways when you push it into the wrong layer.
This creates a procurement problem. Vendors pitch “agent platforms” without clarifying which layer they actually operate at. You end up evaluating a framework’s API ergonomics when you need answers about state durability, or asking a managed runtime about multi-tenant credential isolation when it was designed for single-runtime deployments.
Here’s how to ask the infrastructure questions that expose what actually ships.
The Three Layers Teams Confuse
Layer 1: Agent Logic (frameworks)
Reasoning loops, tool-calling, state transitions, planning. LangGraph and CrewAI live here. They give you primitives to define agent behavior. They do not run your agents in production.
Layer 2: Agent Runtime (managed platforms)
Execution sandbox, session isolation, interrupt/resume, built-in memory. Claude Managed Agents, Bedrock AgentCore, Gemini Enterprise Agent Platform provide this. They run agents. They do not coordinate multiple runtimes or centralize credentials across teams.
Layer 3: Agent Control Plane (infrastructure)
Multi-runtime coordination, credential centralization, session durability across runtimes, per-agent identity, observability, scheduling, cost attribution. Almost nobody ships this for agents yet. Most teams build it themselves around month three.
The silent assumption in agent platform marketing is that managing agents is the same as managing LLMs. It’s not. LLMs are stateless request-response. Agents hold state, call tools, span sessions, and fail mid-workflow. The plumbing is different.
State Persistence: The First Failure Mode
Ask: How does the platform handle state when an agent crashes mid-workflow?
Most demos show happy-path execution. Production agents crash. The network drops. The LLM times out. The tool returns a 500. You need to know where state lives and how recovery works.
| State Strategy | Recovery Model | Failure Mode |
|---|---|---|
| In-memory | Restart from beginning | Lost context, duplicate tool calls |
| Database-backed | Resume from last checkpoint | Checkpoint lag, partial writes |
| Event-sourced | Replay from event log | Log size growth, replay cost |
| Vendor-managed | Opaque (trust the runtime) | No visibility, no control |
If the platform stores state in-memory, every crash is a full restart. If it checkpoints to a database, you need to know the checkpoint frequency and what happens to in-flight tool calls. If it’s event-sourced, you need to know log retention and replay semantics.
Red flag: The vendor can’t tell you where state lives or shows you a dashboard instead of explaining the persistence model.
Deployment Isolation: Can You Run Agents Separately?
Ask: Can agents run in isolated containers, serverless functions, or do they require a monolithic runtime?
Some platforms couple all agents into a single process. This means one agent’s memory leak crashes the entire deployment. One agent’s dependency conflict blocks updates for every other agent.
What to look for:
- Per-agent container images or function packages
- Independent scaling (one agent scales to 100 instances, another stays at 1)
- Isolated failure domains (agent A crashes, agent B keeps running)
- Separate deployment pipelines (agent A ships without touching agent B)
If the platform requires a monolithic runtime, you’re back to coordinating releases across teams. If it supports isolated deployments, ask how agents discover each other and how shared state (like a conversation context) moves between isolated runtimes.
Red flag: The platform’s “multi-agent” examples show all agents defined in one codebase with shared imports.
Observability: Can You See What Happened Without Vendor Dashboards?
Ask: What observability hooks exist for tracing agent decisions, tool calls, and token usage without vendor-specific dashboards?
Vendor dashboards are useful. They are not sufficient. You need raw telemetry you can route to your existing observability stack. OpenTelemetry traces, structured logs, metrics in Prometheus format.
Minimum viable observability:
- Trace IDs that span agent reasoning, tool calls, and LLM requests
- Structured logs with agent state snapshots at decision points
- Metrics for token usage, tool latency, error rates per agent
- Export to standard formats (OTLP, JSON logs, Prometheus)
If the platform only surfaces telemetry in its own dashboard, you can’t correlate agent behavior with the rest of your system. You can’t alert on agent-specific SLOs in your existing monitoring. You can’t debug cross-service failures.
Red flag: The vendor says “we have great dashboards” but can’t show you the raw telemetry schema or export endpoints.
Tool Authorization: Who Controls the Credentials?
Ask: How does the platform handle tool authorization and rate limiting when multiple agents share the same external API credentials?
Agents call tools. Tools need credentials. If ten agents share one Stripe API key, you need per-agent rate limiting, credential rotation, and audit logs.
Credential management patterns:
- Centralized secret store: Platform manages credentials, agents request tokens at runtime
- Per-agent identity: Each agent gets its own credentials, platform handles rotation
- Passthrough: Agents use developer-provided credentials, platform doesn’t touch them
- Vendor-managed: Platform provides built-in integrations, you can’t bring your own
If the platform uses a centralized secret store, ask how credential rotation works and whether agents can outlive a rotated secret. If it supports per-agent identity, ask how you map agent identities to external service accounts. If it’s passthrough, you’re managing credentials yourself.
Red flag: The platform’s tool integration examples hardcode API keys in agent definitions.
Escape Hatches: Can You Leave?
Ask: What’s the migration path if you need to leave the platform? Can you export agent definitions, state snapshots, and execution logs in standard formats?
Vendor lock-in is real. The best platforms make it easy to leave. The worst platforms make it impossible.
What you need to export:
- Agent definitions in a portable format (not vendor-specific DSL)
- State snapshots (conversation history, working memory, tool call results)
- Execution logs (what the agent did, when, and why)
- Telemetry history (for post-migration debugging)
If the platform stores agent logic in a proprietary format, you’re rewriting agents to leave. If it doesn’t expose state snapshots, you lose conversation context on migration. If execution logs are only in the vendor dashboard, you can’t debug regressions after switching.
Red flag: The vendor says “you own your data” but can’t show you the export API or file formats.
Architecture: What a Real Control Plane Looks Like
A production agent control plane sits above frameworks and runtimes. It coordinates multiple agents, centralizes credentials, and provides observability across all execution environments.
┌─────────────────────────────────────────────────┐
│ Agent Control Plane │
│ - Session routing │
│ - Credential vault │
│ - Observability aggregation │
│ - Cost attribution │
└─────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Runtime A│ │ Runtime B│ │ Runtime C│
│ (Claude) │ │ (Bedrock)│ │(LangGraph│
│ │ │ │ │ on K8s) │
└──────────┘ └──────────┘ └──────────┘The control plane doesn’t execute agents. It routes sessions to the right runtime, injects credentials, collects telemetry, and handles cross-runtime state transfer when an agent needs to move (for cost, latency, or capability reasons).
Key responsibilities:
- Session routing: Map incoming requests to the correct agent runtime based on agent ID, user context, or load
- Credential injection: Provide time-limited tokens to agents at runtime, rotate secrets without redeploying agents
- State transfer: Move conversation context between runtimes when switching from a fast/cheap agent to a slow/smart one
- Observability aggregation: Collect traces and logs from all runtimes, provide unified query interface
- Cost attribution: Tag every LLM call and tool invocation with agent ID, user ID, and session ID for billing
Most teams build this themselves. A few platforms are starting to ship it. When evaluating a platform, ask whether it provides control plane capabilities or expects you to build them.
The Questions Vendors Avoid
Here’s the checklist. If the vendor can’t answer these, they’re selling a demo, not infrastructure.
- State persistence: Where does state live? What happens on crash? Can I inspect state snapshots?
- Deployment isolation: Can I run agents in separate containers? Can I scale them independently?
- Observability: Can I export traces and logs to my existing stack? What’s the telemetry schema?
- Credential management: How do agents get credentials? Can I rotate secrets without downtime?
- Escape hatches: Can I export agent definitions and state? What format?
- Failure recovery: What happens when a tool times out? When the LLM rate-limits? When the network drops?
- Multi-runtime coordination: Can I run the same agent on different runtimes? How does state move?
- Cost attribution: Can I tag every LLM call with agent ID and user ID? Can I set per-agent budgets?
Technical Verdict
Use an agent platform when:
- You need managed execution and don’t want to run your own infrastructure
- The platform’s observability hooks integrate with your existing stack
- You can export agent definitions and state in standard formats
- The platform supports isolated deployments and independent scaling
- The vendor can explain state persistence and failure recovery in detail
Avoid an agent platform when:
- The vendor can’t explain where state lives or how recovery works
- Observability is locked to vendor dashboards with no export
- All agents must run in a monolithic runtime
- Credential management is opaque or hardcoded
- There’s no clear migration path off the platform
The best platforms expose their plumbing. The worst platforms hide it behind demos and dashboards. Ask the infrastructure questions early. If the vendor deflects, you’re buying a prototype, not production infrastructure.