ClaimFlow

The Problem

Medical insurance claims processing is high-stakes and document-heavy, yet most AI tooling either automates final decisions (introducing unaccountable risk) or offers no structured auditability. The gap is a system that uses AI for extraction, classification, retrieval, and summarization while enforcing human approval at every transition and logging every model call and decision in a tamper-evident chain.

My Role

Sole engineer: designed and built the full-stack architecture, trained and fine-tuned the EfficientNet-B0 classifier, implemented the provider-pluggable LLM layer and ChromaDB RAG pipeline, wrote the hash-chained audit log, and authored the 253-test validation suite.

The Approach

• Built a three-stage workflow (imaging specialist review, medical specialist review, insurance agent adjudication) in FastAPI with SQLAlchemy, each stage gated by human approval before the claim can advance, enforcing a state-machine that prevents out-of-order transitions.
• Fine-tuned an EfficientNet-B0 classifier on 15,000 ROCOv2 radiology images to 94.2% test accuracy (ECE 0.016) and fused it with deterministic tampering forensics (DICOM/metadata consistency, ELA, frequency analysis, copy-move detection) so the imaging layer is not wholly dependent on any single generative model.
• Built a provider-pluggable LLM client with Claude as primary and Gemini as fallback, structured outputs, stop-reason guardrails, per-call cost audit, and prompt-injection defences that treat uploaded documents as untrusted data rather than instructions.
• Implemented per-claimant ChromaDB RAG retrieval with namespace isolation and an anonymizer for cross-claimant precedent lookup, then enforced a hash-chained, actor-aware audit log across every workflow transition, model call, retrieval, and email action, validated by a 253-test backend suite covering state-machine, audit-tamper, PII, prompt-injection, real-CNN serving, and full lifecycle E2E.

Key Decisions

• Chose a fine-tuned EfficientNet-B0 over a vision LLM for modality classification because the classifier is cheap, fast, repeatable, and independently testable, while the LLM handles language-heavy summarization where deterministic alternatives are weaker, mixing the two gives auditability without sacrificing capability.
• Made the provider-pluggable LLM layer swappable behind one client interface so the demo runs Claude and Gemini while the documented production path points to Bedrock-style controlled deployment, avoiding vendor lock-in at the workflow level.
• Treated uploaded claim documents as untrusted data throughout: structured outputs, role separation, retrieval namespace isolation, and injection tests in the test suite, because the failure mode in a high-trust environment is not wrong answers, it is uncontrolled instruction execution from adversarial document content.

Impact & Results

• 94.2% test accuracy (ECE 0.016) on the EfficientNet-B0 imaging classifier trained on 15,000 ROCOv2 radiology images, with deterministic tampering forensics fused in for robustness beyond the CNN alone.
• 253-test backend suite covering state-machine transitions, audit-tamper detection, PII handling, prompt-injection cases, real-CNN serving, and full lifecycle E2E, giving confidence that the governance layer holds under adversarial inputs.
• Hash-chained, actor-aware audit log records every model call, retrieval, workflow transition, and email action, providing a tamper-evident trace that satisfies the auditability bar required in regulated decision-support contexts.

The Stack

What I Learned

• Building the test suite around adversarial cases (prompt injection, audit tamper, PII leakage) rather than only happy-path flows surfaced design gaps I would not have caught through manual testing, the discipline of writing injection tests forced me to harden the document-ingestion boundary before it was obviously broken.
• Fusing deterministic forensic signals with the CNN classifier taught me that model accuracy metrics alone are insufficient for high-trust pipelines: the ECE and accuracy numbers matter, but what the system does when the model is wrong or the input is manipulated matters more, which is why the fallback and audit layers exist independently of the classifier.
• I would add PHI redaction at the ingestion boundary and production-grade RBAC earlier in the design rather than as post-MVP concerns, both are architectural constraints that affect schema, API surface, and retrieval isolation decisions, and retrofitting them cleanly is more work than building them in.