Engineering note

The validator that throws away most of what the model writes

Ask a language model to improve an email draft and it will do a decent job almost every time. The problem is the other times. It will round £4,850 to “around £5,000”, move Tuesday to Thursday because Thursday scans better, or close with a warm sign-off in a name that isn’t yours. Any of those, sent to a client, is worse than the clumsy draft you started with.

Why “the model is usually right” isn’t good enough

A summary that drifts slightly is a mild annoyance. A rewrite that drifts is a different category of problem, because the user is unlikely to catch it. They wrote the draft, they know what it says, and they are reading the improved version to see whether it reads better — not to audit it. The one number they would have checked is the one they no longer expect to have changed.

So MailTwin treats every model output that will be written back into a draft as untrusted input, and runs it through a local validator first. Not a model checking a model. Ordinary deterministic code, on the Mac, that can only ever say “this is safe to write” or “this is not”.

What it actually checks

The validator extracts a set of anchors from the original draft and requires every one to survive: URLs, email addresses, numbers, dates, times, currency amounts. If an anchor is missing from the rewrite, or a new one has appeared that wasn’t there before, the rewrite is rejected and the draft is left exactly as it was.

On top of that it rejects invented subjects, unfilled placeholders ([Name], [Company]), name blocks the user didn’t write, runaway expansion, wholesale collapse, and anything that reads as topic replacement rather than rewriting.

There are two validators, because the two jobs have different tolerances. Improve, inside the compose window, is a strict proofreader: it may fix grammar and tighten sentences and very little else. My Style may rephrase freely, because rewriting in the user’s own register is the entire point, but the anchor rules bind just as hard.

The sign-off bug we shipped anyway

Apple Mail appends your account signature when a reply window opens. The model, asked to write a reply, also produces a closing. For thirty-odd builds that meant users got two: the model’s “Best regards, Chris” followed by Mail’s real signature block.

The fix has two halves, and we needed both. The prompt now tells the model not to write a sign-off, and the output is post-processed to strip closer phrases anyway — in all fifteen shipped languages, because a German model happily writes Mit freundlichen Grüßen whatever the English instruction said. Prompt instructions are a preference, not a guarantee. If it matters, check it in code afterwards.

Where the signature actually goes

The related problem is that the user’s signature must survive a rewrite untouched. The wrong approach is to hope the model preserves it. What MailTwin does is split it off before the request is made, using a signature detector, send only the body, and re-attach the original afterwards — preserving the rich attributed run so a logo or a formatted block comes back identical.

The model never sees the signature, so it can’t improve it, translate it, or decide the job title needs updating.

What it costs

Rejections are not rare. A validator this strict throws away real output, and when it does, the user sees nothing happen — which is a bad experience compared to a plausible-looking rewrite they’d probably have accepted.

We took that trade deliberately, and it’s the one design decision here worth arguing about. The alternative is a tool that is right most of the time about your correspondence with your clients, and the failure isn’t discovered by you. It’s discovered by them.

Covered by Italiamac · iPhoneIslam · ifun.de · Faq-Mac · Michael Tsai

MailTwin is a menubar app for Apple Mail on macOS 13 or later. Eight AI providers with your own key; Apple Intelligence and Ollama need none and keep everything on the Mac. Built by Smilodon AS in Norway. mailtwin.ai · inside MailKit · finding Sent mail