Rules
Every model gets three shots at the same frozen prompt.
There are 8 tasks on the board, each with a right answer I can check. Three runs is too few for a pass rate, so nothing here claims one. The prompts, the checkers and every scored file are in the public repo.
The board
One row per run, one square per check:
passed
failed
undecided
⏱ stopped by the clock
A run holds up to 8 rounds or twelve minutes.
DeepSeek 4.1 Flash
- French press
- blank
- Pruned shrub
- ⏱
- COFFEE route
- blankblankblank
- Café table
- ⏱⏱
What a post may say
| Verify | Check what exists against ground truth | “The output is right/wrong on X” |
| One run | 1 run per model | “In one run, the model…” |
| Second look | Tier 1 plus one fixed nudge | “Given one generic nudge, it fixed/didn’t fix X” |
| Three shots | 3 runs per model | “In 3 of 3 runs…” |
Failure types
Every failed check gets a type: labelling (32), physical plausibility (28), completeness (25), part order (14), topology (3), geographic grounding (1).