AI NewsModels & agentsAnnouncement
A benchmark tests how six LLMs teach on a whiteboard
An open-source project tests six large language models as whiteboard tutors. The best one passed every blind review.
Image: GitHub
Why it mattersMost AI benchmarks ask a model to answer. Teaching a visible lesson on a shared board is a different task. The gap between models is large.
Most benchmarks ask a model to give the right answer. Teaching a lesson on a whiteboard is a different job. The model has to draw, write, narrate and stay on the board while the student watches. A new open-source project tests six models on that job. The repository is molly554/whiteboard-ai-tutor. It has 124 stars on GitHub, under the MIT licence.
The project calls itself a reference. It is not a hosted tool. The author says any language model can be plugged in.
What the test measures
The author ran the same prompts across six models in eight teaching scenarios. A blind review then checked whether each lesson was correct on the board. Two more numbers were measured per run: the time to the first stroke, and the cost of one lesson. All figures come from the project's README.
Gemini 4 Argon passed 11 of 11 scenarios. Its first stroke took 8.0 seconds. One lesson cost 0.0277 dollars. Claude Haiku 5.5 passed 9 of 11, in 12.9 seconds, at 0.0019 dollars per lesson. Gemini 3.8 Flash passed 8 of 11, in 3.4 seconds, at 0.0080 dollars. Claude Sonnet 5.5 passed 7 of 11, in 2.8 seconds, at 0.0158 dollars. DeepSeek V4 Pro passed 5 of 10, in 3.8 seconds, at 0.0064 dollars. DeepSeek V4.1 Flash passed 4 of 10, in 2.5 seconds, at 0.0017 dollars.
The full benchmark run costs about 4 dollars across all six models, the author writes.
What the project shows
The repository comes with 288 recorded lessons. Each one can be replayed in a browser gallery. Side-by-side pictures show how different models handled the same question. Failure cases are shown too: misplaced labels, wrong geometric positions, a model drawing outside the board.
A reader can run the same test with a different model. The project explains the review protocol and the scoring method. The scope is this one teaching task, with the eight scenarios and the blind-review rubric described in the README.
What to take from it
Price alone does not predict score here. Claude Haiku 5.5 is the cheapest high scorer in the table. The slowest first stroke, from the top scorer, is also the most expensive. A team picking a model for a visual task should test on the shape of its own task. The author gives the code and the scoring for anyone who wants to repeat the run.
Source
molly554/whiteboard-ai-tutor on GitHub.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
