Skip to content
Log in

Free resources · For developers

Does a team of Claude reviewers find more bugs in code than one?

One auditor or three found 7 of 8 planted bugs in all 10 audits, and all 10 missed the same race condition.

Does splitting an audit across agents find more bugs?

Anthropic describes subagents as a way to "isolate context, run multiple analyses in parallel, and apply specialized instructions". We asked whether that structure improves a code audit.

What was tested?

We used a small Node app of 204 lines, with 8 planted bugs and 4 look-alike spots that are actually safe. The bugs include SQL injection, a delete route with no authentication, a fallback signing secret, and a stock update that is not atomic. We ran 5 audits with one general auditor, and 5 with three auditors focused on security, correctness and reliability. All used Claude Haiku. A fresh verifier then checked each finding.

What happened?

Every audit did report one real bug nobody planted: the order quantity is never validated, so a negative quantity raises stock.

What did the verifier do?

Across 97 other findings it confirmed 61 and refuted 36. On clear problems it agreed with itself: a login with no rate limit was confirmed 7 times and refuted never. On borderline issues it split. Tokens without an expiry claim were refuted 11 times and confirmed twice.

What does it mean?

More auditors gave more findings, not more of the planted bugs. The one bug that needed reasoning about two requests at once was missed by every setup. The verifier's verdict on a borderline issue changed between audits, so we treat it as a hint, and we think a person should decide.

How far can this be trusted?

What we would do

  1. Plant a few known bugs of different difficulty, including one that needs reasoning across requests, and see which audit finds them.
  2. Add focused auditors only if they find a planted bug the first one missed.
  3. We would keep a person in the loop, and read the verifier's reasons for refusals.

Check

Learn it step by step: Orchestrator and workers

Sources

Sign up for membership to see the sources.