Free resources · For businesses

Can a second chat catch mistakes in an email you wrote with AI?

A second Claude chat, given your sheet, caught each of 10 planted mistakes in every review. How it was told who wrote the draft made no difference.

Can a second chat catch mistakes in your drafts?

You wrote a customer email with AI and want to send it. A second chat can check it against your facts. Anthropic's guide describes a related check, where Claude finds a supporting quote for each claim. We tested a simpler check on customer emails.

What was tested?

A made-up bakery information sheet of 145 words, and 15 draft emails to customers. Ten drafts held one planted mistake each: a wrong closing time, a Sunday opening, a wrong deposit, a wrong total, and similar. Five drafts were correct. Claude Haiku 5.5 saw the sheet and one draft, then answered "Verdict: OK" or "Verdict: ERROR" with the reason. It was told one of three things. The first was that you wrote the draft. The second was that someone else wrote it. The third was that someone else wrote it, and that every number, day and condition must be checked one by one.

What happened?

Each framing caught all 10 planted mistakes in all 3 rounds: 30 of 30 reviews.
No framing flagged a correct draft as wrong: none of the 15 reviews of the 5 correct drafts.
The older Claude Haiku 4.5, in one round, caught all 10 planted mistakes in each framing, with no false alarms.

What does it mean?

A second chat that has the sheet in front of it caught every planted mistake in our test, in a short sheet. Researchers report that a model can score its own output higher. We cannot show that here. The reviewer never wrote the drafts, so "you wrote this" was only a line in the prompt.

How far can this be trusted?

Every mistake could be found by comparing the draft with a short sheet. A reviewer without the sheet was not tested.
Two small models and one kind of document. We planted the mistakes.

What we would do

Open a new chat. Our advice: do not reuse the one that wrote the draft. Our test could not show that reusing it is worse.
Paste your sheet, the draft, and this prompt:

Someone else wrote the draft email below for a customer, using the information sheet. Review their draft against the sheet: check every number, day and condition one by one.
Start your reply with exactly one line: either `Verdict: OK` or `Verdict: ERROR - <what is wrong>`. Then give one sentence of explanation.

Read the reason yourself. A person approves before anything reaches a customer.

Check

Keep the sheet in the review chat. A reviewer without the facts cannot check them.
This result covers short documents and two small models.

Learn it step by step: A separate reviewer