Proof Evals Results
This proof-assistance evaluation is now in review and reporting. The study reviewed 60 model-student proof tutoring transcripts across 6 models, 10 proof tasks, and 6 rubric dimensions.
TLDR
The main difference between models was not mathematical correctness. Nearly all models understood the proof tasks. The strongest models preserved student agency, diagnosed mistakes, and resisted turning hints into submit-ready proofs.
google-frontier-preview was the top human-scored model. anthropic-balanced
was the strongest runner-up. The weakest dimension across the study was
student_agency, especially when models used near-complete fill-in templates.