AWS shipped serverless agent execution and built-in evaluation primitives for Bedrock AgentCore in September 2026. This is the first major cloud provider to bundle evals directly into the agent runtime rather than treating them as separate observability add-ons. The update also includes automated knowledge base syncing with native enterprise connectors for SharePoint, Confluence, and S3.
The infrastructure decisions behind these features reveal how AWS expects production agent systems to handle state persistence, quality gates, and incremental data sync without custom polling loops.
Serverless Agent Execution: State Persistence Without Step Functions
AgentCore’s serverless execution model runs agents without provisioning containers or managing orchestration queues. The runtime handles:
- Cold start optimization: Pre-warmed execution contexts for sub-second invocation latency
- State persistence: Automatic checkpointing between tool calls without forcing developers into Step Functions or external queues
- Execution isolation: Each agent invocation runs in a separate security boundary with ephemeral credentials
The state persistence mechanism is the critical piece. Traditional serverless functions lose context between invocations. AgentCore maintains conversation state, tool call history, and intermediate results across multi-step reasoning chains without requiring developers to serialize state to DynamoDB or S3.
How State Checkpointing Works
AgentCore writes state snapshots after each tool invocation. The runtime stores:
- Conversation history (user prompts, agent responses)
- Tool call results (API responses, database queries)
- Reasoning traces (chain-of-thought steps, decision points)
- Execution metadata (timestamps, token counts, model versions)
State snapshots live in a managed store with automatic TTL. Developers configure retention windows (1 hour to 30 days) based on compliance requirements. The runtime garbage-collects expired snapshots without manual intervention.
Built-In Evaluation: Blocking Gates or Async Side-Effects?
AWS positioned built-in evaluation as a quality control mechanism baked into the agent runtime. The implementation reveals three distinct evaluation modes:
| Evaluation Mode | Execution Path | Latency Impact | Use Case |
|---|---|---|---|
| Blocking | Synchronous gate before response | +200-500ms per eval | High-stakes decisions (financial transactions, medical advice) |
| Async sampling | Background process, 10% of invocations | None | Continuous quality monitoring |
| Post-invocation | Triggered after response delivery | None | Compliance audits, training data collection |
Blocking evals run as inline checks before the agent returns a response. The runtime executes evaluation prompts against the same model or a separate judge model. If the eval fails (hallucination detected, policy violation, low confidence score), the agent can retry with different reasoning or escalate to human review.
Async sampling runs evals on a subset of invocations. The runtime samples 10% of traffic by default (configurable from 1% to 100%). Evaluation results flow to CloudWatch Logs and EventBridge for downstream analysis.
Post-invocation evals run after the response reaches the user. These are useful for compliance logging and training data collection where latency is not a concern.
Evaluation Primitives
AgentCore ships with four built-in evaluation types:
- Hallucination detection: Checks if the response contradicts retrieved knowledge base content
- Policy compliance: Validates responses against custom guardrails (PII leakage, prohibited topics)
- Confidence scoring: Estimates response quality based on reasoning trace consistency
- Tool call validation: Verifies tool invocations match expected schemas and parameter ranges
Developers can chain multiple evals in sequence or run them in parallel. The runtime aggregates results and exposes pass/fail status through the response metadata.
# Example: Configuring blocking evals for financial advice agent
agent_config = {
"evaluations": [
{
"type": "hallucination_detection",
"mode": "blocking",
"threshold": 0.95, # Fail if confidence < 95%
},
{
"type": "policy_compliance",
"mode": "blocking",
"guardrails": ["no_investment_advice", "require_disclaimers"],
},
{
"type": "confidence_scoring",
"mode": "async_sampling",
"sample_rate": 0.25, # Eval 25% of invocations
}
]
}Automated Knowledge Base Syncing: Incremental Updates Without Polling
Native enterprise connectors for SharePoint, Confluence, and S3 handle incremental sync without developers writing custom change-detection logic. The connectors use:
- Webhook subscriptions: SharePoint and Confluence notify AgentCore when documents change
- S3 event notifications: Trigger sync jobs when objects are created, modified, or deleted
- Fallback polling: Hourly scans for connectors that don’t support webhooks
The sync engine maintains a change log with document versions, modification timestamps, and content hashes. When a document updates, the connector:
- Fetches the changed document
- Extracts text and metadata
- Generates embeddings using Bedrock’s Titan or Cohere models
- Updates the vector index with new embeddings
- Marks old embeddings as stale (soft delete with TTL)
Stale embeddings remain queryable for 24 hours to avoid breaking active agent sessions. After TTL expiration, the runtime purges them from the index.
Connector Security Boundaries
Each connector runs with scoped credentials. SharePoint connectors authenticate with OAuth 2.0 and request read-only permissions for specific site collections. S3 connectors use IAM roles with bucket-level access policies.
AgentCore does not store connector credentials. The runtime exchanges short-lived tokens on each sync operation. Tokens expire after 1 hour and cannot be reused across sync jobs.
Observability: What Gets Logged and Where
AgentCore writes structured logs to CloudWatch Logs with the following dimensions:
- Invocation logs: Request ID, agent ID, user ID, timestamp, latency, token count
- Tool call logs: Tool name, input parameters, output, execution time, error codes
- Evaluation logs: Eval type, mode, result, confidence score, failure reason
- Sync logs: Connector type, document count, embedding count, sync duration, errors
Logs use JSON format with consistent schema across all log types. Developers can query logs with CloudWatch Insights or stream them to S3 for long-term analysis.
EventBridge receives evaluation results and sync completion events. Developers can trigger Lambda functions, Step Functions workflows, or SNS notifications based on specific conditions (eval failures, sync errors, high latency).
Failure Modes and Recovery
The serverless execution model introduces new failure scenarios:
- Cold start timeouts: If pre-warming fails, invocations may exceed the 30-second timeout
- State corruption: Concurrent invocations can overwrite state snapshots if not properly isolated
- Eval deadlocks: Blocking evals that call external APIs can hang if those APIs are slow or unavailable
- Sync backpressure: High-volume document updates can overwhelm the embedding generation pipeline
AgentCore handles these with:
- Automatic retries: Failed invocations retry up to 3 times with exponential backoff
- Circuit breakers: Disable blocking evals if latency exceeds 1 second for 5 consecutive invocations
- Rate limiting: Sync jobs throttle at 100 documents per minute to prevent embedding API saturation
- Dead letter queues: Failed invocations land in SQS for manual inspection
Technical Verdict
Use AgentCore serverless execution when:
- You need sub-second cold starts without managing containers
- Your agents require multi-step reasoning with automatic state persistence
- You want quality gates (evals) enforced at the runtime layer, not bolted on later
- Your knowledge bases sync from SharePoint, Confluence, or S3 and you want to avoid custom polling logic
Avoid it when:
- Your agents need execution times longer than 15 minutes (current serverless limit)
- You require custom state backends (Redis, Postgres) for complex workflow orchestration
- Your evaluation logic depends on external services with unpredictable latency
- You need fine-grained control over embedding generation (model selection, chunking strategies, metadata filtering)
The built-in eval primitives are the most interesting piece. Treating evals as first-class runtime constructs rather than observability afterthoughts changes how you design agent quality controls. Blocking evals add latency but prevent bad outputs from reaching users. Async evals give you continuous monitoring without impacting response times. The trade-off table above should guide your mode selection.
The automated sync connectors save engineering time but lock you into AWS-supported data sources. If you need custom connectors (Notion, Google Drive, internal APIs), you still write polling loops or webhook handlers.