Accuracy benchmark

Last run: August 2026 · Reproducible from our eval harness

Most "AI document extraction" accuracy claims are unverifiable. Ours isn't: we test against a 100-document public benchmark — rent rolls, tenant schedules, and operating statements filed with the U.S. Securities and Exchange Commission, spanning multifamily, office, retail, industrial, hospitality, and net-lease assets. Public documents anyone can check — and we publish exactly what we measured.

Methodology

• Corpus: 60 real rent rolls, tenant rosters, tenant-level net-lease/industrial schedules, and unit-mix tables from SEC EDGAR filings (8-K loan exhibits and CMBS prospectus supplements) — office, multifamily, student housing, manufactured housing, self-storage, industrial, retail, and mixed portfolios.
• Goldens: expected values were hand-transcribed from each source document by a human, independently of the extractor.
• Scoring: every golden field (unit numbers, tenants, square footage, lease dates, rents, deposits, stated totals) must match the extraction exactly — no partial credit, no fuzzy matching on numbers.
• Verification layer: beyond extraction, each document passes our deterministic checks — recomputed totals reconciled against stated totals, unit counts cross-checked.

Per-document results

Scroll for all 60 documents.

Scanned documents

The benchmarks above use native digital documents. Real deals run on scans — so we re-ran both corpora as simulated scans: every document rendered to a grayscale 300 DPI image with page skew, sensor noise, and JPEG compression, then wrapped as an image-only PDF (no text layer — extraction must read pixels).

Same goldens, same exact-match scoring: 789/789 fields across the 60 scanned rent rolls and 620/620 across the 40 scanned operating statements, with all verification checks passing. Result was stable across three consecutive full runs of each corpus. The scanned corpora ship alongside the originals in our eval harness.

What this does and doesn't claim

This benchmark shows exact-match performance on real, messy, public rent-roll documents — including OCR-degraded source text (one document literally spells "LIFESTYLES" as "LJFESTYLES"; we extract what's printed, faithfully). It does not claim every possible document will extract perfectly. Documents we handle less well today: handwritten rent rolls, photos of paper taken at steep angles, and non-English documents. That's why every extraction ships with a verification report: recomputed totals reconciled against stated totals, and any value the model can't read with confidence is flagged, never guessed. You always know which fields to double-check.

The corpus grows over time. Want your (anonymized) document format represented? Send it over.

T12 / operating statements

The API also extracts trailing-twelve-month operating statements with the same philosophy: every line item categorized, and a deterministic verification pass that recomputes row totals from monthly values and reconciles Effective Gross Income, Total Operating Expenses, and NOI against the underlying rows. Same benchmark discipline, separate corpus: 40 real cash-flow analysis tables from CMBS prospectus supplements filed with the SEC — multifamily, retail, office, industrial, student housing, mixed-use, and hospitality.

Scored fields per document: NOI, Effective Gross Income, Total Operating Expenses, and individual income/expense line items (5–8 per statement), matched exactly against hand-transcribed values. All deterministic verification checks passed on every document.

Scroll for all 40 statements.

Try it on your own rent roll — 3 free

Related: AI rent roll parsing · Rent roll & T12 reconciliation · Convert a rent roll PDF to Excel · T12 to Excel · Rent roll extraction API