How do we know it’s good?

How good is it, really?

For a digit classifier, “how good” was easy: accuracy on held-out digits. For a model that writes essays, code and proofs, it’s one of the hardest open problems in AI. A good evaluation has to be relevant to real use, resistant to cheating, and precise enough to tell two models apart.

• Benchmarks: fixed question sets with known answers (multiple choice, maths, code with tests).
• Human preference: people compare two answers side by side and pick the better one.
• Model graders: a strong model scores answers against a rubric. It’s cheap and scales, but has biases of its own.