Built Technologies processes over $500 billion in real estate projects. Their documents arrive as mixed PDFs: a single file might contain a loan agreement, three inspection reports, insurance certificates, and invoices. Traditional OCR and template-based extraction fail when document boundaries shift, page counts vary, and domain-specific clauses appear in unpredictable locations.
The company built a five-stage document intelligence pipeline on AWS Bedrock and the Intelligent Document Processing (IDP) Accelerator. The pipeline reduces workflows from days to minutes and supports hundreds of document types. It serves as the foundation for agentic products that review construction draws, validate insurance coverage, and analyze loan agreements.
This is not a chatbot over PDFs. It is a production pipeline that classifies, splits, extracts, evaluates, and reasons over complex financial documents with human-in-the-loop validation and audit trails.
Pipeline Architecture
The system breaks document processing into five discrete stages. Each stage produces structured output that the next stage consumes. This separation allows technical teams and domain experts to improve individual processors without rewriting the entire pipeline.
Stage 1: Classification
The classifier receives a PDF and assigns a document type. Real estate finance has hundreds of types: draw requests, lien waivers, pay applications, inspection reports, insurance certificates, loan agreements, offering memoranda. Each type has different extraction requirements and validation rules.
The classifier uses a fine-tuned model on Bedrock. It returns a document type label and a confidence score. Low-confidence classifications route to a human review queue before proceeding.
Stage 2: Splitting
A single uploaded PDF often contains multiple documents. A construction draw package might include the draw request, five invoices, three lien waivers, and two inspection reports. The splitter identifies document boundaries and separates the file into individual documents.
This stage uses layout analysis and semantic boundary detection. It looks for title pages, signature blocks, and context shifts. Each split document carries metadata about its position in the original file for audit purposes.
Stage 3: Extraction
The extractor pulls structured data from each document. For an invoice, it extracts vendor name, invoice number, line items, amounts, and payment terms. For a lien waiver, it extracts contractor name, project address, waiver type, and covered period.
Extraction uses a combination of Bedrock models and the IDP Accelerator’s entity recognition. The system maintains extraction schemas for each document type. Domain experts define required fields, optional fields, and validation rules without writing code.
Stage 4: Evaluation
The evaluator checks extraction quality. It compares extracted values against expected formats, cross-references related documents, and flags inconsistencies. For example, if an invoice amount exceeds the remaining budget in the draw request, the evaluator raises a flag.
Evaluation rules are configurable per document type. Some checks are hard constraints (a date must be in the future). Others are soft warnings (an amount is unusually high). Each flagged item includes a confidence score and a reason.
Stage 5: Reasoning
The reasoning layer answers questions about the processed documents. An agent might ask, “Is this draw request compliant?” or “What is the total amount of outstanding lien waivers?” The reasoning layer queries the structured data from extraction and applies domain logic.
This stage uses retrieval-augmented generation (RAG) over the extracted entities and document text. It maintains a vector store of document chunks with metadata about document type, extraction confidence, and validation results. When an agent asks a question, the reasoning layer retrieves relevant chunks, checks validation flags, and generates an answer with citations.
State Management and Collaboration
The pipeline uses a queue-based architecture. Each stage publishes results to an SQS queue that the next stage consumes. This decoupling allows stages to scale independently and supports retry logic when extraction or evaluation fails.
State is stored in DynamoDB. Each document has a record that tracks its progress through the pipeline, extraction results, validation flags, and human review decisions. Agents query this state to determine whether a document is ready for automated processing or needs escalation.
The system provides a shared environment where technical teams and domain experts collaborate. Domain experts define extraction schemas and validation rules through a UI. Technical teams deploy model updates and tune confidence thresholds. This separation of concerns allows rapid iteration without requiring domain experts to write code or engineers to become real estate finance specialists.
Human-in-the-Loop Validation
Confidence scores determine when human review is required. Each stage produces a confidence score for its output. If classification confidence is below 0.85, the document routes to a review queue. If extraction confidence for a critical field is below 0.90, a human validates the extracted value before the document proceeds.
Human reviewers see the original document, the extracted data, and the confidence scores. They can approve, correct, or reject the extraction. Corrections feed back into the training data for model fine-tuning.
The system tracks which documents required human intervention and why. This observability helps teams identify document types that need better training data or extraction logic improvements.
Observability and Audit Trails
Every extraction decision is traceable. The system logs which model version processed each document, which extraction schema was used, what confidence scores were produced, and whether a human reviewed the output.
When an agent makes a decision based on processed documents (for example, approving a construction draw), the system records which document sections contributed to that decision. If a loan approval depends on 50 parsed files, the audit trail shows which extracted entities from which documents influenced the final decision.
This lineage tracking is critical for financial compliance. Regulators and auditors can trace an agent’s decision back to the source documents and see exactly what data the agent used.
Deployment Shape
The pipeline runs on AWS Lambda for compute and uses Bedrock for model inference. Each stage is a separate Lambda function. This serverless architecture scales automatically when document volume spikes (for example, at month-end when draw requests surge).
The IDP Accelerator provides pre-built components for layout analysis, entity recognition, and table extraction. Built customized these components for real estate finance document types.
Vector embeddings for the reasoning layer are stored in Amazon OpenSearch. Document metadata and extraction results live in DynamoDB. Original PDFs and split documents are stored in S3 with lifecycle policies that archive older documents to Glacier.
Failure Modes and Mitigation
| Failure Mode | Impact | Mitigation |
|---|---|---|
| Misclassification | Wrong extraction schema applied | Confidence thresholds route low-confidence classifications to human review |
| Incorrect splitting | Multiple documents treated as one or single document split incorrectly | Boundary detection uses multiple signals (layout, semantics, page breaks); human review for low confidence |
| Extraction errors | Missing or incorrect field values | Field-level confidence scores; critical fields require high confidence or human validation |
| Evaluation false positives | Valid documents flagged as problematic | Configurable validation rules; soft warnings vs. hard constraints; human override capability |
| Reasoning hallucination | Agent answers question with incorrect information | RAG retrieves only from extracted entities with citations; confidence scores attached to answers |
The system monitors extraction accuracy by document type. When accuracy for a specific document type drops below a threshold, the system alerts the team and increases the human review rate for that type until the issue is resolved.
Trade-offs
Latency vs. accuracy: The five-stage pipeline adds latency compared to a single-pass extraction. A simple document might take 30 seconds to process instead of 5 seconds. Built accepted this trade-off because accuracy and auditability matter more than speed for financial document processing.
Flexibility vs. complexity: Allowing domain experts to define extraction schemas and validation rules without code increases flexibility but adds complexity to the schema management system. The team built a UI for schema editing and version control to manage this complexity.
Automation vs. human oversight: Higher confidence thresholds reduce errors but increase human review volume. Built tuned thresholds per document type based on risk. Critical documents (loan agreements) have higher thresholds than routine documents (invoices).
Technical Verdict
Use this architecture when you need to process complex, variable-length documents with high accuracy requirements and audit trails. The five-stage pipeline works well for financial services, legal document processing, healthcare records, and other domains where extraction errors have significant consequences.
Avoid this approach if you need real-time document processing (under 1 second) or if your documents are highly standardized. The multi-stage pipeline adds latency and operational complexity. For simple, consistent documents, a single-pass extraction with template matching will be faster and cheaper.
The human-in-the-loop validation and collaborative schema management are the differentiators. If your domain experts cannot define extraction rules without engineering help, or if you cannot afford human review for low-confidence extractions, this architecture will not deliver value.
The system shines when document types number in the hundreds, when extraction accuracy directly impacts business decisions, and when you need to explain to auditors exactly how an agent reached a conclusion.