Never Grade Your Own Work
Ask an agent to build a feature and then ask the same agent, in the same context, whether the feature works. It will say yes. Not because it is lazy or dishonest, but because of the most cross-validated finding in this whole research pass: models systematically overrate their own output, the bias is structural, and it does not yield to prompting.
The evidence, from three directions
Academic — the bias is causal and mechanistic. LLM evaluators score their own outputs higher than others' even when human annotators judge them equal (NeurIPS 2024). Fine-tuning a model to be better at recognizing its own text linearly increases its self-preference — the relationship is causal, not a confound. And the deeper mechanism is worse news than "models recognize themselves": judges favor low-perplexity, familiar-feeling text regardless of who wrote it (arXiv 2410.21819). That's why anonymizing authorship doesn't fix it — the model isn't recognizing its byline, it's rewarding its own style distribution. It also means a same-model actor/evaluator loop can amplify without bound: the evaluator rewards outputs that look like what it would have produced.
Academic — self-correction mostly doesn't. A survey of self-correction research (CACM) found reported successes typically involved weak baselines or tasks with obvious answers; that error detection, not revision, is the bottleneck — a model that can't see the mistake can't fix it; that intrinsic self-correction (the model relying on its own feedback) fails more often than it succeeds; and that self-bias intensifies with each additional round of self-refinement. External tools — a test run, a search — are what reliably helped.
Practice — Anthropic hit it head-on. Building harnesses for long-running coding agents, Anthropic reports that left to self-assess, Claude systematically marked features complete without end-to-end verification. Giving it concrete testing tools (browser automation to actually click through the feature) "dramatically improved performance" (Effective harnesses). At system scale, the MAST taxonomy of 1,600+ annotated multi-agent traces found ~24% of all failures were verification failures — premature termination, incomplete or incorrect checking.
Close the loop through the environment
The principle that falls out: a claim of "done" is worth exactly the external evidence behind it. The loop must close through something that is not the model's opinion —
In descending order of strength:
- Machine-checkable signals. Tests, compiler exit codes, screenshot diffs, a probe with an expected output. The Claude Code best-practices docs make the operational stakes explicit: a machine-checkable signal is what converts a supervised session into an unattended one — "without a check the agent can run, looks done is the only signal available, and you become the verification loop." Anthropic's Agent SDK guidance agrees on form: the best feedback is clearly defined rules plus which rules failed and why — rich, granular, external.
- A fresh-context reviewer. When no machine check exists (design quality, review judgment), use a builder/validator split: a different session with a clean context reviews the work. Fresh context attacks the familiarity gradient — the reviewer isn't steeped in the builder's reasoning, so the builder's mistakes aren't already normalized. A different model family attacks it harder. One warning from the same source: adversarial reviewers over-report — a reviewer prompted to find problems will find some even when none matter. Triage findings; don't chase all of them into over-engineering.
- The operator. For anything irreversible or judgment-laden, the human is the verification step. That's not a failure of autonomy — see Part 4.
What about Reflexion?
The apparent counterexample: Reflexion agents grade their own trajectories and improve — 91% pass@1 on HumanEval, beating single-shot GPT-4's 80% (Shinn et al.). Read the setup, though, and it confirms the rule: Reflexion's coding results lean on external unit-test feedback. The reflection verbalizes an environmental signal; it doesn't replace one. Where feedback is binary or self-simulated, the same pattern confabulates and freezes (Part 2). Reflection is a lever on ground truth, not a substitute for it.
Applying it
- Before any unattended task, name the check that will say it worked. If you can't name one, the task isn't unattended — route it to review.
- Structure work so checks exist: typed code (
tsc,cargo check), tests that name the failing step, probes with expected GREEN/RED output. - Review through a fresh session, and calibrate: findings are candidates, not verdicts.
- Never let "the agent says it's done" be the record. The record is the test run, the diff, the probe output.
XiaoSteve's operating doctrine has this as rule 6 — never grade my own work; a second session with clean context reviews anything important — and its practical bite is real: a recent fix shipped only after an adversarial fresh-context review flagged that a diagnostic's "MATCH" status couldn't distinguish "parameters match" from "nothing was sent" — a semantics bug the builder session had already rationalized away.
Next: if verification is the constraint, autonomy stops being a personality setting and becomes an engineering quantity. Part 4 →