Built wrong

These checks fail when the result may look right, but the model didn't make it the way the brief said.

What went wrong
Checks failed	27 of 101
Results	26
Models	12
Tasks	2
Dates	2026-09-29 to 2026-10-16
Wrong in some other way	26
The wrong thing in its place	1
How bad
Would not work	3
Misleading	24
Where a model ran a task 3 times or more
Failed every time	3
Failed only sometimes	8

A coffee shrub hedge-pruned flat on one side, by Muse Spark Open — Muse Spark OpenA coffee shrub hedge-pruned flat on one side · 2026-10-16 · 0 of 3 checksFlat side from the rules, construction wrong, majorAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Muse Spark Open — Muse Spark OpenA coffee shrub hedge-pruned flat on one side · 2026-10-16 · 1 of 3 checksFlat side from the rules, construction wrong, majorAlso failed: Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Muse Spark Open — Muse Spark OpenA coffee shrub hedge-pruned flat on one side · 2026-10-16 · 0 of 3 checksFlat side from the rules, construction wrong, majorAlso failed: Reads as a shrub (unreadable); Sensible size (false)

Beans leaving a grinder hopper, in MuJoCo, by GPT-6 Luna — GPT-6 LunaBeans leaving a grinder hopper, in MuJoCo · 2026-10-15 · 4 of 6 checksBeans start inside a hollow hopper, construction wrong, critical, cause: repair; its note said nothing was wrongAlso failed: Held, then flows (broken)All 50 beans are through by 1 s, before the gate opens, and every one leaves the model. Its eighth and last round crashed before drawing, so round 7 is scored.

Beans leaving a grinder hopper, in MuJoCo, by GPT-6 Luna — GPT-6 LunaBeans leaving a grinder hopper, in MuJoCo · 2026-10-15 · 1 of 6 checksFifty two-part beans, component substituted, major, cause: repair, follows from model.xml written; its note said nothing was wrong; Beans start inside a hollow hopper, construction wrong, critical, cause: repair, follows from model.xml written; its note said nothing was wrongAlso failed: model.xml written (incomplete); Gravity down (false); Held, then flows (broken)No model.xml beside the script, so the checker has no model to read. The hopper is about a metre tall, and 44 beans are out by 1 s.

Beans leaving a grinder hopper, in MuJoCo, by DeepSeek 4.1 Flash — DeepSeek 4.1 FlashBeans leaving a grinder hopper, in MuJoCo · 2026-10-15 · 5 of 6 checksBeans start inside a hollow hopper, construction wrong, critical, cause: clock; its note left no closing note27 beans start in a column 19 cm high and drop into the cone in an extra second it runs before 0 s, so the picture never shows them.

A coffee shrub hedge-pruned flat on one side, by GLM 5.3 FlashX — GLM 5.3 FlashXA coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was rightAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by GLM 5.3 FlashX — GLM 5.3 FlashXA coffee shrub hedge-pruned flat on one side · 2026-10-03 · 2 of 3 checksFlat side from the rules, construction wrong, major; its note named the fault

A coffee shrub hedge-pruned flat on one side, by Perceptron Mk1.5 — Perceptron Mk1.5A coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Perceptron Mk1.5 — Perceptron Mk1.5A coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Perceptron Mk1.5 — Perceptron Mk1.5A coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Command A+ — Command A+A coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was rightAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Qwen3.8 Max Prime — Qwen3.8 Max PrimeA coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by GPT-5.6 Sol — GPT-5.6 SolA coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was rightAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Qwen3.8 Max Prime — Qwen3.8 Max PrimeA coffee shrub hedge-pruned flat on one side · 2026-10-03 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by DeepSeek 4.1 Flash — DeepSeek 4.1 FlashA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major, cause: clock; its note left no closing note

A coffee shrub hedge-pruned flat on one side, by Qwen3.8 Flash — Qwen3.8 FlashA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major, cause: clock; its note left no closing note

A coffee shrub hedge-pruned flat on one side, by GPT-6 Luna — GPT-6 LunaA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was right

A coffee shrub hedge-pruned flat on one side, by GPT-6 Luna — GPT-6 LunaA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was right

A coffee shrub hedge-pruned flat on one side, by GPT-6 Luna — GPT-6 LunaA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major; its note said it fell short, without naming this

A coffee shrub hedge-pruned flat on one side, by Gemini 3.8 Flash — Gemini 3.8 FlashA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Gemini 3.8 Flash — Gemini 3.8 FlashA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by DeepSeek 4.1 Flash — DeepSeek 4.1 FlashA coffee shrub hedge-pruned flat on one side · 2026-09-29 · 0 of 3 checksFlat side from the rules, construction wrong, major; its note left no closing noteAlso failed: Reads as a shrub (unreadable); Sensible size (false)

A coffee shrub hedge-pruned flat on one side, by Claude Sonnet 5.5 — Claude Sonnet 5.5A coffee shrub hedge-pruned flat on one side · 2026-09-29 · 2 of 3 checksFlat side from the rules, construction wrong, major; its note named the fault

A coffee shrub hedge-pruned flat on one side, by Claude Sonnet 5.5 — Claude Sonnet 5.5A coffee shrub hedge-pruned flat on one side · 2026-09-29 · 1 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was rightAlso failed: Reads as a shrub (unreadable)

A coffee shrub hedge-pruned flat on one side, by Opus 5.5 — Opus 5.5A coffee shrub hedge-pruned flat on one side · 2026-09-29 · 1 of 3 checksFlat side from the rules, construction wrong, major; its note said this very thing was rightAlso failed: Reads as a shrub (unreadable)

The other kinds: malformed · false · incomplete · unreadable · broken

All of them, with counts and the codebook · How a check is decided