Multi-tenant agent systems are moving from prototype to production. The question is no longer whether you can run multiple customers on shared infrastructure, but how you enforce tenant boundaries when agents share execution environments, memory stores, and tool credentials.
AWS just published production patterns for pool-model multi-tenancy using Bedrock AgentCore. The example is healthcare AI agents serving multiple clinics and hospitals on shared compute. The architecture matters because it shows how to isolate state, credentials, and observability without duplicating infrastructure for every tenant.
Pool Model vs. Silo Model
Two deployment patterns dominate multi-tenant agent systems:
Silo model: Each tenant gets dedicated infrastructure. Separate Lambda functions, separate agent runtimes, separate memory stores. Complete isolation, but expensive at scale.
Pool model: Tenants share infrastructure. Single set of Lambda functions, shared agent runtime, shared memory backend. Isolation happens at the session and data layer, not the compute layer.
| Dimension | Pool Model | Silo Model |
|---|---|---|
| Cost | Shared compute, lower baseline | Dedicated resources per tenant |
| Isolation | Logical (session, credentials, data) | Physical (separate infrastructure) |
| Scaling | Horizontal, tenant-agnostic | Per-tenant capacity planning |
| Noisy neighbor risk | Requires rate limiting, quotas | Naturally isolated |
| Operational complexity | Single deployment, shared observability | Multiple deployments, per-tenant monitoring |
| Compliance | Depends on data isolation guarantees | Easier to audit, harder to manage |
AgentCore implements pool-model multi-tenancy by enforcing tenant boundaries at the runtime level. You run one agent, but every session, memory strand, and tool invocation is scoped to a tenant identifier.
How AgentCore Enforces Tenant Isolation
The architecture uses a three-level hierarchy: Tier → Tenant → User.
Tier defines service level (premium, standard, basic). Different tiers get different models, memory retention, and tool access.
Tenant represents the customer organization (clinic, hospital, business unit). Each tenant gets isolated credentials, memory, and audit logs.
User is the individual making requests. Users belong to tenants, and their sessions inherit tenant-level isolation.
Request Flow
- API Gateway receives request with tenant context in headers or JWT claims.
- Lambda authorizer validates tenant, extracts tier and tenant ID.
- Request context includes
tenantId,tier,userId. - AgentCore session is created with tenant-scoped memory strand.
- Tool invocations use tenant-specific credentials from Secrets Manager.
- Logs and metrics are tagged with tenant ID for cost attribution.
Memory Isolation
AgentCore uses memory strands to store conversation history and agent state. Each strand is scoped to a tenant. When you create a session, you pass a strand ID that includes the tenant identifier:
strand_id = f"tenant-{tenant_id}-user-{user_id}-{session_type}"
session = agentcore_client.create_session(
agentId=agent_id,
memoryStrandId=strand_id,
sessionConfiguration={
"maxTurns": tier_config["max_turns"],
"memoryRetention": tier_config["retention_days"]
}
)The strand ID is opaque to the agent runtime. AgentCore ensures that memory queries only return data from the specified strand. Cross-tenant memory leakage is prevented at the API level, not by application logic.
Credential Isolation
Tools need credentials (database passwords, API keys, third-party tokens). In a multi-tenant system, each tenant must use their own credentials, even when sharing the same tool implementation.
AgentCore supports dynamic credential injection. When defining a tool, you specify a credential resolver that fetches tenant-specific secrets:
def get_tenant_credentials(tenant_id, tool_name):
secret_name = f"{tenant_id}/{tool_name}/credentials"
response = secrets_manager.get_secret_value(SecretId=secret_name)
return json.loads(response['SecretString'])
tool_definition = {
"name": "query_patient_records",
"credentialResolver": lambda context: get_tenant_credentials(
context["tenantId"],
"patient_db"
)
}The agent runtime calls the resolver before invoking the tool. The resolver fetches credentials from Secrets Manager using a path that includes the tenant ID. Even if two tenants use the same tool, they authenticate with different credentials.
Service Tier Differentiation
Different tenants pay for different capabilities. Premium tenants get Claude Opus, 90-day memory retention, and access to all tools. Basic tenants get Claude Haiku, 7-day retention, and limited tools.
AgentCore does not have built-in tier management. You implement tiers by varying session configuration and tool availability based on tenant metadata.
Model selection happens at session creation. You pass a different modelId based on tier:
tier_models = {
"premium": "anthropic.claude-3-opus-20240229-v1:0",
"standard": "anthropic.claude-3-sonnet-20240229-v1:0",
"basic": "anthropic.claude-3-haiku-20240307-v1:0"
}
session = agentcore_client.create_session(
agentId=agent_id,
modelId=tier_models[tenant_tier],
memoryStrandId=strand_id
)Tool filtering happens at agent configuration. You define multiple agent versions, each with a different tool set. When creating a session, you select the agent version that matches the tenant tier.
Rate limiting is enforced at the API Gateway level. Each tenant gets a usage plan with request quotas. API Gateway throttles requests when a tenant exceeds their quota. This prevents one tenant from consuming all available compute.
Cost Attribution and Observability
In a pool model, you need granular cost tracking. You cannot rely on infrastructure-level billing because multiple tenants share the same Lambda functions and Bedrock endpoints.
CloudWatch Logs are tagged with tenant ID. Every log entry includes tenantId in structured JSON. You can filter logs by tenant, aggregate error rates, and track usage patterns.
CloudWatch Metrics use custom dimensions. When logging metrics, include TenantId and Tier as dimensions:
cloudwatch.put_metric_data(
Namespace='AgentCore/MultiTenant',
MetricData=[{
'MetricName': 'TokensConsumed',
'Value': token_count,
'Unit': 'Count',
'Dimensions': [
{'Name': 'TenantId', 'Value': tenant_id},
{'Name': 'Tier', 'Value': tier}
]
}]
)You can query metrics per tenant, calculate cost per tenant, and identify high-usage customers.
Bedrock model invocation logs include request metadata. You can tag each invocation with tenant context and use CloudWatch Logs Insights to aggregate token usage:
fields @timestamp, tenantId, inputTokens, outputTokens
| filter tenantId = "clinic-123"
| stats sum(inputTokens + outputTokens) as totalTokens by bin(5m)This gives you per-tenant billing data without running separate infrastructure.
Failure Modes and Blast Radius
Pool-model multi-tenancy shares failure modes across tenants. If the agent runtime crashes, all tenants are affected. If a tool has a bug, all tenants using that tool see errors.
Noisy neighbor: One tenant sends a flood of requests, saturating Lambda concurrency. Other tenants see throttling errors. Mitigation: API Gateway usage plans with per-tenant quotas, Lambda reserved concurrency for critical tenants.
Credential leakage: A bug in the credential resolver returns the wrong tenant’s secrets. Tenant A authenticates to Tenant B’s database. Mitigation: Include tenant ID in secret paths, validate tenant context before every credential fetch, audit secret access logs.
Memory cross-contamination: A bug in strand ID generation causes two tenants to share a memory strand. Tenant A sees Tenant B’s conversation history. Mitigation: Use deterministic strand ID generation with tenant ID as a required component, validate strand ownership before session creation.
Tool execution errors: A tool crashes when processing Tenant A’s request. The Lambda function restarts, affecting in-flight requests from other tenants. Mitigation: Isolate tool execution in separate Lambda functions, use async invocation with SQS for long-running tools.
When to Use Pool Model
Pool model makes sense when:
- You have many tenants with similar usage patterns.
- Cost efficiency is more important than physical isolation.
- You can enforce logical isolation with session scoping and credential management.
- You need to scale horizontally without per-tenant capacity planning.
Avoid pool model when:
- Regulatory requirements mandate physical infrastructure separation.
- Tenants have wildly different usage patterns (one tenant uses 100x more than others).
- You cannot tolerate any risk of cross-tenant data leakage.
- You need per-tenant performance guarantees that shared infrastructure cannot provide.
Technical Verdict
AgentCore’s pool-model multi-tenancy works when you need cost-efficient scaling for many tenants with similar workloads. The session-scoped memory, dynamic credential injection, and tenant-tagged observability give you logical isolation without duplicating infrastructure.
Use it for SaaS platforms where tenants are roughly equal in size and usage. Avoid it for regulated industries that require physical separation or for platforms with extreme variance in tenant workloads.
The architecture is production-ready if you implement rate limiting, credential validation, and blast radius controls. Without those, you risk noisy neighbors, credential leakage, and cascading failures across tenants.