Evaluator-optimizer
Draft, review, revise
People rarely get good writing or code right on the first try. They draft, get feedback, and revise. This pattern does the same with models.
A generator writes a draft. An evaluator checks it against criteria and returns specific feedback. The generator revises, and the loop repeats until the draft passes or a limit is reached.
- Generator draftsA first translation
- Evaluator reviewsChecks terms, tone, and missing parts
- Feedback returns“The term checkout is translated in two ways.”
- Pass or stopMeets the criteria, or stops after three rounds
1. Write criteria the evaluator can check
“Make it better” is not a criterion. “Every product term matches the glossary” is. Clear criteria lead to specific feedback, and specific feedback leads to useful revisions.
2. Use code checks where you can
Some criteria do not need a model: length limits, required sections, a glossary lookup, or tests that must pass. Combine them with a model review for the rest.
3. Limit the rounds
Each round costs time and money, and improvements usually shrink after two or three. Stop when the draft passes, when the feedback repeats, or when the limit is reached.
When is it useful?
When you have clear criteria and a first draft can be measurably improved, such as translation, code that must pass tests, or text with strict rules. Without clear criteria, the loop only adds cost.