LinkedRecords is an open-source backend-as-a-service that lets users bring their own storage: S3 buckets, Dropbox folders, self-hosted instances, or any other backend they control. It started as a workaround for Google Docs limitations in 2018 and evolved into a Firebase alternative with a triplestore-inspired data model. The interesting part is not the BaaS itself but the authorization and state management problems it exposes when agents need to operate across user-controlled storage boundaries.
For fintech and trading infrastructure, this architecture matters. Regulatory requirements often mandate that financial data stays in specific jurisdictions or under customer control. Agent-driven trading systems, compliance monitoring tools, and portfolio analytics platforms need to respect data sovereignty while maintaining the performance and security characteristics of centralized systems.
The Authorization Problem
Traditional SaaS assumes a single storage layer under your control. You authenticate users, they get a session, and your backend queries a database you own. Agent-driven applications break this model. An agent might need to:
- Read trading data from Alice’s S3 bucket in eu-west-1
- Write compliance reports to Bob’s self-hosted instance
- Coordinate state across both without leaking credentials
- Maintain audit logs when the storage layer is outside your infrastructure
LinkedRecords forces you to solve this at the architecture level. The agent cannot hold long-lived credentials for every user’s storage backend. The user cannot trust the agent with unrestricted access. You need a delegation model that works across arbitrary storage providers.
Financial Use Cases
Agent-driven trading systems: A trading agent needs to read market data from multiple user-controlled sources, execute trades through user-authorized brokers, and write results back to user-specified storage. The agent cannot hold API keys for every brokerage account. Users need to revoke access without changing their brokerage credentials.
Compliance-sensitive financial data: Banks and investment firms often require that customer data never leaves their infrastructure. An AI agent analyzing portfolios or generating reports must operate on data stored in the customer’s environment, not in your cloud account.
Multi-party financial workflows: Loan origination, trade settlement, and syndicated lending involve multiple parties who do not trust each other with full data access. Each party controls their own storage. Agents coordinate the workflow without centralizing sensitive data.
Architecture: Compute-Storage Separation
LinkedRecords decouples the application layer from the storage layer. The application runs in your infrastructure (or serverless functions). The data lives wherever the user wants it.
Core components:
- Client SDK: Handles authentication and storage provider abstraction
- Authorization layer: Manages scoped tokens and permission boundaries
- Real-time sync engine: Coordinates state across distributed storage backends
- Schema registry: Tracks data models independently of storage location
The authorization flow works like this:
- User authenticates with your app
- User grants scoped access to their storage backend (OAuth for cloud providers, API keys for self-hosted)
- Your app receives a time-limited delegation token
- Agent operations use the delegation token to read/write data
- Storage provider enforces access controls at the file or object level
This is similar to OAuth delegation but extended to arbitrary storage backends. The agent never sees the user’s root credentials. The user can revoke access without changing their storage credentials.
Multi-Tenant Query Performance
When data lives in 50 different S3 buckets, caching becomes critical. LinkedRecords uses a hybrid approach:
- Local cache: Each agent maintains a working set of recently accessed records
- Invalidation signals: Storage backends send webhooks or poll-based change notifications
- Lazy loading: Only fetch data when an agent operation requires it
- Prefetch hints: Application code can declare likely access patterns
The tradeoff table:
| Strategy | Latency | Consistency | Storage Cost | Failure Mode |
|---|---|---|---|---|
| No cache | High (network RTT per query) | Strong | Zero | Slow but correct |
| Local cache | Low | Eventual | Memory per agent | Stale reads possible |
| Distributed cache (Redis) | Medium | Tunable | Shared infrastructure | Cache stampede on invalidation |
| Prefetch | Low (when correct) | Eventual | Wasted bandwidth if wrong | Over-fetching |
For agent workflows, eventual consistency is usually acceptable. The agent can retry on conflict or use optimistic locking. The bigger risk is cache invalidation lag causing the agent to operate on stale data and produce incorrect results.
State Synchronization Across Backends
When an agent needs to coordinate writes across multiple user-controlled backends, you need a coordination layer. LinkedRecords uses a triplestore-inspired model where each record is a set of subject-predicate-object triples. This makes partial replication easier.
Synchronization strategies:
- Optimistic locking: Each record has a version number. Writes include the expected version. Storage backend rejects stale writes.
- Event sourcing: Append-only log of changes. Agent replays events to reconstruct state. Works well with S3 (immutable objects).
- CRDT-based merge: For collaborative editing scenarios. Requires storage backend to support atomic compare-and-swap.
The failure mode that matters: what happens when a user’s storage backend is offline or slow? The agent needs to decide whether to block, skip that user’s data, or queue the operation for retry. LinkedRecords provides a task queue abstraction that persists pending operations in the application layer (which you control) and retries when the backend becomes available.
Schema Migration Without Control
Traditional databases let you run migrations during deployment. When users control the storage, you cannot force a schema upgrade. LinkedRecords handles this with versioned schemas and backward-compatible readers.
Migration flow:
- Application declares schema version N+1
- Agent writes new data in N+1 format
- Agent reads data in any version from N-2 to N+1
- User’s storage backend holds mixed-version data indefinitely
- Application provides a migration tool users can run voluntarily
This is similar to API versioning but applied to the data layer. The agent must handle multiple schema versions in the same query result. You need explicit deserialization logic for each supported version.
def read_record(storage_backend, record_id):
raw_data = storage_backend.get(record_id)
schema_version = raw_data.get("_schema_version", 1)
if schema_version == 1:
return migrate_v1_to_v3(raw_data)
elif schema_version == 2:
return migrate_v2_to_v3(raw_data)
elif schema_version == 3:
return raw_data
else:
raise UnsupportedSchemaVersion(schema_version)The cost is code complexity. You carry migration logic for every schema version you need to support. The benefit is users never experience forced downtime or data loss from schema changes.
RBAC and Audit Logs
Role-based access control is straightforward when you control the database. You check permissions before every query. When the storage layer is external, you have two options:
- Application-layer RBAC: Your app checks permissions before issuing storage operations. The storage backend sees all requests as coming from the application service account.
- Storage-layer RBAC: The storage backend enforces permissions. Your app translates user roles into storage-specific access policies (S3 bucket policies, Dropbox sharing rules, etc.).
LinkedRecords uses application-layer RBAC because storage backends have inconsistent permission models. The tradeoff is that a compromised application can bypass access controls. You mitigate this with:
- Scoped delegation tokens (time-limited, operation-limited)
- Audit logs written to a separate append-only store you control
- Cryptographic signatures on sensitive operations
Audit logs are critical. Since you do not control the storage layer, you cannot rely on storage-level access logs. LinkedRecords writes audit events to a centralized log (your infrastructure) before executing the operation. If the operation fails, the audit log still records the attempt.
Deployment Shape
LinkedRecords runs as a stateless service. Each request:
- Authenticates the user
- Retrieves scoped storage credentials from a secure vault (HashiCorp Vault, AWS Secrets Manager)
- Executes the operation against the user’s storage backend
- Writes audit log
- Returns result
You can deploy this as:
- Serverless functions: AWS Lambda, Google Cloud Functions. Cold start latency matters if you are doing real-time collaboration.
- Container service: ECS, Kubernetes. Better for long-running agent workflows that need to maintain state across multiple operations.
- Edge workers: Cloudflare Workers, Fastly Compute. Lowest latency for read-heavy workloads but limited execution time.
The stateless design means you can scale horizontally without coordination. The bottleneck is usually the user’s storage backend, not your compute layer.
Failure Modes
Storage backend unavailable: Agent operations fail. You need retry logic with exponential backoff. For critical workflows, queue the operation and notify the user.
Credential expiration: Scoped tokens expire. The agent must detect 401/403 responses and request a new token. If the user has revoked access, the operation fails permanently.
Concurrent writes: Two agents modify the same record. Optimistic locking catches this. The losing agent must retry with the updated version.
Schema version skew: Agent expects schema version 3, user’s data is version 1. Migration logic handles this, but if you drop support for old versions, reads fail.
Audit log write failure: The operation succeeds but the audit log write fails. You have a gap in your audit trail. Mitigate with at-least-once delivery (write audit log to a queue, process asynchronously).
Technical Verdict
Use LinkedRecords or a similar architecture when:
- Data sovereignty is a hard requirement (healthcare, finance, government)
- Users already have data in external systems you need to integrate
- You are building a platform where users bring their own infrastructure
- Compliance requires data to stay in specific jurisdictions or storage tiers
- You are building agent-driven trading or compliance systems where users control brokerage credentials and financial data
Avoid this approach when:
- You need strong consistency across all operations
- Query performance is critical and you cannot tolerate network latency
- Your agents need complex joins or aggregations across user data
- You are building a consumer app where users expect zero configuration
- Real-time trading decisions depend on sub-100ms data access
The authorization model is the hardest part. If you get it wrong, you either leak credentials or lock users out of their own data. The performance characteristics are also unpredictable because you do not control the storage layer. But for use cases where data sovereignty matters more than performance, decoupling compute from storage is the only viable architecture.