Parapet
Pre-bid software for estimators at mid-market general contractors. I stopped it in Sep '26.
Why I built itEstimators spend 16 to 24 hours per project reading drawings to write a scope of work, so I tested whether AI could do that reading.
- The range comes from my interviews with construction estimators. It depends on project scope.
- Only the estimator knows how the scope gets built, so the work is slow and hard to hand over.
- Before the rebuild I set a quality line of 7.5 out of 10. Scores above 6 came rarely and never held.
- Status
- stopped
- When
- Apr '26 to Sep '26
- Type
- B2B SaaS, web
- My role
- Solo, product and engineering
I stopped in Sep '26. The extraction pipelines missed their evaluation thresholds. I had replaced the score with a proposed test of estimator time saved, but stopped before that test ran.
The product question
Copying the estimation software estimators already used was easy. Without reliable drawing extraction, was that enough to compete with it? An AI feature needs an eval to back it or to contradict it, so I set the threshold before building it.
Scope
- Apr '26 to Sep '26
- first to last commit in the code repo
- 5 scoring rounds
- the five documented scoring rounds of the first pipeline, May '26 to Jun '26
Riverside Retail - Division 09 finishes
| Subcontractor | AI fit score | Quote | Scope gaps |
|---|---|---|---|
| Northline Drywall | 91 | Received | 0 gaps |
| Keystone Interiors | 84 | Received | 3 gaps |
| Harbor Ceilings | 78 | RFQ sent | - |
| Summit Flooring | 69 | Not invited | - |
Scope item 09 51 13 acoustical ceilings: allowance missing. Scope item 09 65 00 resilient flooring: excluded in quote notes.
Illustrative mockup with generic content, drawn in HTML and CSS. Company names are invented.
What I tried
I built the pre-bid workflow around AI extraction from drawings.
- Parapet covered the pre-bid workflow: project setup, a record of drawings, a subcontractor database with AI scoring, geo-based bid lists, RFQs, quote parsing with gap analysis against the scope, and project rooms.
- The rebuilt extraction became the only extraction path in production. Its first production run reported success with zero rows. The design partner never used extraction.
- I built it alone, product and engineering, with Claude Code agents.
- Each roadmap brief named the cheapest test that would kill the idea. A dated decision log was never rewritten.
- Extraction quality had a scored eval rubric (a fixed test set with a scoring sheet). In Sep '26 I added a kill line of 95% recall to the redesign.
Evals
What these measurements can tell us
The 7.5 score combined quality criteria. The later 95% line asked how much of an estimator's scope list the pipeline found. Time saved in a real estimation task stayed unmeasured.
The verdict also estimated that drawings held only part of an estimator's list. That estimate came from a separate information-coverage analysis. The public account does not establish a common denominator for those figures, so I do not use them as comparable percentages.
I proposed timing an estimator's review after that verdict. I stopped before running it; the amount of time the product could save remains untested.
An eval is a fixed test set with a scoring sheet. Mine compared the AI's draft of an estimator's scope list (one row per item, grouped by trade division) with the list a human estimator wrote for the same drawings. I wrote the thresholds down before each build, so a round could say no.
From Sep '26 the sheet scores 14 criteria from 0 to 10. The headline cannot exceed the correction-cost score plus 2, so a list that needs heavy correction cannot score well however much it found. The first pipeline used an earlier sheet of 13 criteria, and scores from the two sheets do not compare.
The thresholds
Five scoring rounds on the first pipeline
No round held above 7.5. The decision log says the April gain was "partly luck of the run". On the two criteria estimators asked for most, supplier attribution fell from 5.6 in April to 3.5 and then 2.0 in May, while prompt-only fixes were going in. The challenge review chose a structural fix, a deterministic reader beside the model. In Aug '26 I chose to rebuild extraction, and the first pipeline was deleted when the rebuilt one became the only path.
Rounds on the rebuilt pipeline
From Sep '26 the rebuilt pipeline ran on the 14-criteria sheet. Across six rounds in two weeks the mean rose from 4.3 to a plateau between 5.9 and 6.6. The correction-cost cap then held the headline at 4.0, then 3.0.
The verdict in Sep '26
The pipelines I tested did not hold above 7.5 across drawing sets.
A general-purpose vision model reading a handful of PDFs, steered by one client's past lists, produced round means between 5.9 and 6.6 in the evaluated runs. A second model red-teamed the verdict before I filed it.
- Five months, two rebuilds and three model families left the best valid rebuilt-round mean at 6.6.
- The last release shipped ten targeted fixes. One moved its score line: addenda, from 3.0 to 8.0, on the one set that has an addendum.
- Correction cost sat 10 to 70 times above the target. Across 583 rows on three sets, none was correct in every field.
- Quantities stayed at 2 to 3 out of 10 for four releases, because the pipeline reads quantities from tables and never measures.
- At the median the drawings hold about 60% of an estimator's list, so even a perfect reader leaves about 40% to write by hand.
Rows that come from printed schedules (doors, finishes, millwork) and addenda might still have worked. That was unproven, and the same round showed those divisions broken.
The whole-list score stopped being the success metric that day. The new test was the estimator's clock: time one estimator reviewing the AI list against starting from a past list, with a kill line of under 50% of review time saved. I stopped Parapet nine days after the verdict, before any estimator was timed.
Decision timeline
Started solo, with Claude Code agents. Each roadmap brief named the cheapest kill test, and a dated decision log was never rewritten. Extraction got a scored eval.
Five scoring rounds on the first pipeline. Only one set reached 7.5.
A challenge review found that three sprints of prompt-only fixes had given no durable lift on supplier names and revision tags. It chose a deterministic reader beside the model.
The first pipeline was deleted. The rebuilt pipeline became the only extraction path in production.
The first production run of the rebuilt pipeline reported success with zero rows. The design partner had not used extraction.
I wrote the kill line into the redesign before it was built: 95% recall per drawing set.
No measured set reached the 95% line.
Verdict: the tested pipelines did not hold above 7.5 across drawing sets. The score stops being the success metric. A timed review by an estimator replaces it, with a kill line of under 50% of review time saved.
Stopped, before the timed review ran. I removed the deployments and backed up the database. The code is kept and the restart steps are written down.
Questions I asked along the way
Do we name a price or is it "contact sales"?
- Option rejected
- Publishing a Team price on the site, which the product review recommended because a mid-market buyer needs a number to take internally.
- What happened
- The review's list of resolved decisions recorded "pricing tier: not a priority for now", so I deferred the decision.
Does the reconcile pass move the wrong-division rate at all?
- Option rejected
- Adding a set-level reconcile pass with the largest model on top of the first pipeline. The test was one comparison, with and without the pass, and I would drop it if the wrong-division rate fell by less than 20% relative.
- What happened
- The same day I weighed four options: swap the model only, patch five defects, send the whole PDF in one call, or buy an off-the-shelf takeoff tool. I chose a rebuild that reads each page first and assembles rows afterwards. The first pipeline was deleted in Aug '26, so the comparison never ran.
Can the whole score ever hold above 7.5?
- Option rejected
- One more scored round before showing the output to an estimator, which the Sep '26 handover asked for. I judged that an eleventh round scored by a model could not supply the number the product is judged on.
- What happened
- The answer was no. I replaced the score with an estimator's clock as the test, and I stopped the product before that test ran.
What happened
Stopped. The pilot ran with one design partner, a mid-market general contractor.
- The first pipeline never held above 7.5 across sets in five scoring rounds, from May '26 to Jun '26.
- I rebuilt extraction in August and wrote the 95% recall line into the redesign in early September. No measured set reached it.
- Then I wrote the verdict and replaced the score with a timed review by an estimator, with a kill line of under 50% of review time saved. I stopped before that review ran.
- On the hardest drawings the answer sat across several pages. A prompt could not add that context.
- Two limits remained: no body of data to train on and a build larger than one person could carry.
- I removed the deployments and backed up the database. The code is kept. The restart steps are written down.
What I took from it
- I wrote the 95% recall line into the redesign in Sep '26, before it was built, so every round after it had a number to measure against. That is step 4 of the nine steps.
- A passing test harness did not show what users would get. The production path needs an end-to-end run on real input, which is step 7, Check the work, of the nine steps and rule 3 of Claude Code setup.
- An AI feature needs an eval that can contradict it. I wrote the 7.5 line before the rebuild and the 95% recall line before the redesign was built, so the rounds could say no. See Evals.
- Three sprints of prompt-only fixes did not lift supplier names or revision tags, so in May '26 I chose a structural fix: a deterministic reader beside the model.