Retail brokers increasingly restrict API access while continuing to send detailed email confirmations. This creates a practical agent pattern: parse notifications into structured trade records, reconcile state across multiple messages, and maintain accurate position tracking without direct programmatic access.
The Sensible Trader implementation shows how this works in production. The challenge is not extracting a ticker and price. It is building a state machine that handles partial fills, rolls, assignments, duplicate notifications, and ambiguous events that cannot be safely interpreted without human review.
The State Problem
A spreadsheet can log every transaction and still lose position state. A buy and a sell appear as unrelated rows. A rolled covered call looks like a realized loss plus a new naked call. An assignment closes an option while leaving underlying shares unchanged.
The rows are present. The answer to “what remains open?” is wrong.
Broker emails already contain the fills. The engineering challenge is turning those emails into trustworthy position state without duplicating trades or guessing at ambiguous events.
Email as Event Stream
Broker confirmations arrive as semi-structured text:
- Order placed notifications
- Execution confirmations (full or partial fills)
- Settlement notices
- Corporate action alerts (splits, dividends, assignments)
- Cancellation confirmations
Each email represents a state transition. The agent must:
- Parse unstructured text into structured fields
- Classify the event type
- Reconcile against existing position state
- Detect duplicates when the same trade generates multiple emails
- Handle out-of-order delivery
Parsing Boundaries
The system uses a two-stage extraction pipeline:
Stage 1: Regex extraction for high-confidence fields that follow consistent patterns across broker formats:
- Timestamps (ISO 8601 or broker-specific formats)
- Account numbers
- Order IDs
- Symbols (ticker extraction with options notation)
- Quantities (signed integers indicating direction)
- Prices (decimal extraction with currency handling)
Stage 2: LLM-based field mapping for ambiguous or context-dependent fields:
- Event classification (execution vs. cancellation vs. assignment)
- Fee breakdowns (commission, regulatory, exchange)
- Option action types (open, close, roll, assignment, exercise)
- Multi-leg strategy identification
The boundary exists because regex fails on natural language variations (“filled,” “executed,” “completed”) while LLMs hallucinate on numeric precision. Hybrid parsing uses regex for numbers and LLMs for semantics.
def parse_broker_email(email_body, broker_format):
# Stage 1: Extract structured fields
extracted = {
'timestamp': extract_timestamp(email_body),
'symbol': extract_symbol(email_body, broker_format),
'quantity': extract_signed_quantity(email_body),
'price': extract_decimal_price(email_body),
'order_id': extract_order_id(email_body, broker_format)
}
# Stage 2: Classify event semantics
event_classification = llm_classify_event(
email_body=email_body,
extracted_fields=extracted,
context=get_recent_position_state(extracted['symbol'])
)
# Confidence gating
if event_classification['confidence'] < 0.85:
return queue_for_human_review(extracted, event_classification)
return {**extracted, **event_classification}State Reconciliation Strategy
The agent maintains a position ledger with these invariants:
- Every open position has a net quantity (long, short, or flat)
- Every closed position has a realized P&L
- Every partial fill updates quantity without creating a new position
- Every roll closes one position and opens another atomically
Reconciliation logic:
- Duplicate detection: Hash (order_id, timestamp, symbol, quantity, price). If hash exists, skip processing.
- Partial fill aggregation: Group by order_id. Sum quantities until order is fully filled.
- Close matching: When a sell arrives, match against open lots using FIFO, LIFO, or specific lot identification based on broker rules.
- Roll detection: Two events within 60 seconds with same underlying, opposite directions, different strikes/expirations = roll.
Failure Modes and Human Review Queue
Certain events cannot be safely automated:
| Event Type | Automation Risk | Queue Trigger |
|---|---|---|
| Partial fill on multi-leg spread | Mismatched leg execution creates undefined state | Leg count mismatch |
| Assignment with no prior short position | Possible untracked short or data loss | Position not found |
| Corporate action (split, merger) | Ratio calculation errors compound | Non-standard ratio |
| Duplicate with different prices | Possible amendment or correction | Hash collision with price delta |
| Out-of-order settlement | State depends on processing sequence | Timestamp inversion |
The human review queue is not a failure state. It is a design decision. Guessing at ambiguous events creates silent position drift. Queueing them preserves data integrity.
Orchestration Flow
Email Ingestion (IMAP/Gmail API)
↓
Broker Format Detection (rule-based classifier)
↓
Hybrid Parsing (regex + LLM)
↓
Confidence Gating (threshold: 0.85)
↓
├─ High Confidence → State Reconciliation
│ ↓
│ Duplicate Check
│ ↓
│ Position Update
│ ↓
│ Journal Entry
│
└─ Low Confidence → Human Review Queue
↓
Manual Classification
↓
Feedback Loop (retrain classifier)Observability
Key metrics for email-based trade capture:
- Parse success rate: Percentage of emails successfully classified
- Duplicate detection rate: Emails skipped due to hash match
- Queue rate: Percentage of emails requiring human review
- State drift: Difference between agent-calculated positions and broker-reported positions (reconciled daily)
- Latency: Time from email receipt to position update
State drift is the critical metric. If agent positions diverge from broker positions, the entire system loses trust.
Security Boundaries
Email access creates attack surface:
- Credential storage: OAuth tokens for Gmail API or IMAP credentials in encrypted vault
- Email filtering: Only process emails from verified broker domains (SPF/DKIM validation)
- Injection prevention: Sanitize all extracted fields before database insertion
- Audit trail: Log every email processed, every position change, every human review decision
Phishing risk is real. A fake broker email with malicious data could corrupt position state. Domain validation and SPF checks are not optional.
Deployment Shape
The production system runs as:
- Email poller: Cron job every 5 minutes (IMAP) or webhook (Gmail API push notifications)
- Parser service: Stateless workers processing email queue
- State reconciler: Single-threaded service with exclusive lock on position database
- Review UI: Web interface for human classification of queued events
Single-threaded reconciliation prevents race conditions when multiple emails arrive simultaneously. The parser can scale horizontally. The reconciler cannot.
Technical Verdict
Use email-based trade capture when:
- Broker APIs are restricted, rate-limited, or unavailable
- Email confirmations contain sufficient detail (timestamps, prices, fees)
- You can tolerate 5-minute latency between trade execution and journal update
- You have a human review process for ambiguous events
Avoid when:
- Real-time position tracking is required (use WebSocket APIs instead)
- Broker emails lack critical fields (some brokers omit fees or lot identification)
- You cannot implement duplicate detection (risk of double-counting trades)
- Email access credentials cannot be secured properly
The pattern works because broker emails are already structured enough for reliable extraction, but unstructured enough that full automation is unsafe. The human review queue is the design that makes it trustworthy.