All Posts
Opus 5.5 knows the order of filter in a french press, but can it draw it?
Asked in words, Opus 5.5 got the filter order right in 5 of 5 runs, and asked to draw it, in 2 of 5.
L-systems are fun ways to get very varied output
Models drawing a half pruned coffee tree
Sonnet 5.5 against Opus 5.5, which one costs more and perform bests on the Venn, the French press and the shrub?
Opus wins the Venn, Sonnet wins the French press and the shrub, and Sonnet is twice as cheap on all three.
Does measuring help a model lay out a Venn? With a bit more statistics.
Sol kept the labels clear nine times out of ten whether I gave it a list, told it to check itself or said nothing.
Does measuring help a model lay out a Venn?
Results seems to indicate that telling the models to keep labels off the lines worked as well as telling them to measure.
Qwen3.8 Flash vs DeepSeek 4.1 Flash draw a coffee Venn
Qwen gets all three runs right and DeepSeek only one.
Qwen3.8 Max Prime draws three coffee shrubs
Three very different shrubs, and only one looks right.
Ten models draw a coffee Venn, some check with vision others with code
Almost all put the drinks in the right place, but only a few keep the labels off the lines.
DeepSeek found the grader and ran it
It found my grading script and ran it on its own answer, so no score.
Opus 5.5 got the French press right, then changed it
Nicest drawings so far, but the stack is right in only one of three runs.
Two of my headlines were wrong after three runs
The croissant and the boiler headlines didn't hold up with more runs.
Bonsai vs Qwen3.8 27B draw a French press
Neither of them drew a filter that works.
GLM 5.3 FlashX draws a French press without the cross-plate
It used a base and a handle instead of the cross-plate.
Pareto knows the part order but can't draw it
It lists the parts in the right order, but the drawing is upside down.
Someone ran the control on the pelican test
The labs don't seem to train on the pelican.
Pareto vs Sol draw a nice french press with the three filter piece upside down
Both models know all the parts, but the order of the filter was wrong.
Union Alpha spells COFFEE on the streets of melbourn
All checks out
Astra vs DeepSeek 4.1 Flash both build a boiler
All checks out with temps 92–96 °C
DeepSeek 4.1 Flash cant make croissants
Not a nice croissant
Qwen3.8 Max builds a caffeine molecule
Every check passes
Welcome to Jagged
Controlled tests of what new models can and can't do, checked against ground truth.