AI that checks fire risk assessments against the QA team's own rules

01The challenge

Tetra Consulting's QA team checks the fire risk assessments and health and safety reports its consultants write: more than 1,500 reports a month across the business. Feedback goes back as yellow highlights and comments in Word. Some reports went back and forth ten or more times as corrections were missed, and the knowledge of what to look for lived in individual reviewers' heads rather than in any system. Consultants sometimes fixed the Word file instead of the source report on their tablet, so corrections never reached Tetra's own records.

The rules did exist, in Tetra's QA guidance and in thousands of reports the team had already marked up. Tetra wanted consultants to be able to run a QA pass on their own report before it reached the team, without loosening its standards.

02What we did

1. 01September 2025 — proposed an agent that learns Tetra's standards from its QA guidance and years of marked-up reports
2. 02April–May 2026 — version 1 live for testing, with a rule set refined by Tetra's QA lead and a feedback loop on every rule
3. 03July 2026 — refocused on measurement: every QA decision becomes labelled data, and false positives became the target
4. 04August 2026 — reports split into section-level chunks, with each rule checked only against the sections it concerns
5. 05September 2026 — reviewer tooling completed: photos and tick boxes render, and a consultant gets a view-only copy of their QA'd report in one click
6. 06October 2026 — evaluation framework delivered seven weeks early, so every rule change is measured before the team sees it
7. 07Next — reports arrive from POPS when a reviewer is assigned, then a soft launch and rollout across around 800 reports a month

03What we built

The agent works in two stages: it decides which of Tetra's rules apply to a report, then writes its findings into the document as comments for a QA manager to approve, reject or edit. Sending a whole report with up to 200 rules in one prompt proved inaccurate, so each rule is now checked only against the sections it concerns, guided by notes on where to look, plus a short context pack covering the report type, cover page and hazard index.

Around 200 rules come from Tetra's QA guidance; another 2,000–3,000 mined from historical marked-up reports stay out of the live set until reviewed. Evaluation runs in two layers, whether the agent looked in the right section and whether it reached the right decision, scored against cases labelled by the QA team, with false positives weighted as worse than misses. A webhook from POPS, Tetra's workflow system, is being connected so that assigning a report to a reviewer starts its review.

• Rules drawn from guidance and marked-up reports
• Section-level checks with a shared context pack
• Review screen to approve, reject or edit
• Comments written back into the report
• Rule editing by Tetra's QA team
• Evaluation against labelled QA decisions
• View-only report links for consultants

04The result

On real Tetra reports, the first version of the section-level design cut false positives from around 80% to around 20%, before any second iteration. The evaluation framework arrived seven weeks ahead of plan, so accuracy is now scored rule by rule against labelled data rather than judged on review calls.

The agent is ready for live use with Tetra's QA team. Once reports flow in from POPS, every decision on around 800 reports a month adds to a record of how Tetra reviews: the evidence for where consultants can safely self-check, and for whether a fine-tuned model is worth training.