AI NewsModels & agentsAnnouncement
Kombai puts GPT-6 Astra at 65.5 for design creativity, humans at 92.3
Kombai's Design Creativity Bench runs 34 frontier models through 80 UI design briefs across two product domains, scores each on originality, creative range and appropriateness, and reports the top model at 65.5 against a human reference of 92.3.
Image: Kombai
Why it mattersA team shipping an AI feature that generates UI now has a published number for how much creative range to expect from each model, with the method, the briefs and the paper all public, so the choice has a reference other than the vendor's own claim.
A reader who has watched every leading model draw the same shape of pelican when asked for one may have wondered whether the sameness is measurable. Kombai, a UI-generation company, published a benchmark on 8 October that measures it directly, in a domain that matters to a working developer: 80 open-ended UI design briefs, each adapted to two different product domains, with the model free to pick the layout, structure and visual direction.
The benchmark is called Design Creativity Bench. Thirty-four frontier models are evaluated at their highest reasoning setting, against expert human designers who completed the same 160 briefs. The paper, by Aman Rusia, Abhijit Bhole, Prashank Gupta and Dipanjan Dey, is on arXiv.
The scores, in Kombai's own numbers
GPT-6 Astra leads at 65.5. GPT-6.1 Sol is at 63.7, Grok 4.7 at 62.2, Gemini 3.8 Flash at 60.6, Qwen3.8 Max at 58.1 and Gemini 4 Argon at 55.7. The human reference sits at 92.3. Claude Opus 5.5 scores 51.0 and Claude Fable 5.1 scores 48.6. The full table lists 34 models down to Seed 1.6 Flash at a negative creativity score.
The composite is built from three measures Kombai separates and reports. Originality averages 0.646 for GPT-6 Astra, 0.764 for human designers, with a 95% confidence interval of 0.582 to 0.602 across all model-model pairs on the same brief. Creative range, how much a model changes its approach when the product context changes, is 0.472 for GPT-6 Astra and 0.902 for human designers. Appropriateness, whether the output satisfies the brief, is above 90% for every model Kombai tested, and the best model reaches 99.2%, which is 1.2 points above the 98.0% for human designs. The method is validated against human judgements of design distance.
Where the gap sits
Capability and creativity are not the same thing, Kombai writes, and the data does not support the assumption that a more capable model is more diverse. GPT-6 Astra produces the most original designs against other models and shows the least variation across paired briefs: it is both the most distinctive and the most repetitive, which is a point other scoreboards cannot show because they measure one dimension. The failure modes Kombai names, when a model does miss, are specific: a third of failures were missing a control the brief asked for, and a quarter were inconsistent sample data.
The cost column tells its own story. GPT-6 Astra, the top model, costs a median of $2.564 per task on these briefs. GLM-5.3-Flash, cheaper at $0.022 per task, scores 38.5. Gemini 3.8 Flash, at $0.123, scores 60.6 and runs in a median of 0.9 minutes against GPT-6 Astra's 13.8. A team picking a model for an in-product UI generator can read all three columns at once.
Source
Kombai, Design Creativity Bench results. The paper is at arXiv:2610.11539, the dataset at HuggingFace, and the Hacker News discussion at item 50090443.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
