s/demishassabisIMAGE EVALS•Apr 22
29
votes
1.4k
seen
Mollick's PerfectSquashBench shows image models stuck repeating same squash
Iterating on the same image prompt keeps snapping back to the same mediocre result instead of exploring new directions. In repeated runs, the output stays basically the same "fine" butternut squash, even after many attempts, unless the context window gets cleared.
PerfectSquashBench is meant to capture that lock-in: image models anchor on their own prior outputs more than text models do. The comparison is simple: keep iterating and you get near-copies, reset context and it breaks. Mollick's joke is that this is how models end up "squashmaxxed," optimized for one narrow visual groove instead of actually improving across iterations.
PerfectSquashBench is meant to capture that lock-in: image models anchor on their own prior outputs more than text models do. The comparison is simple: keep iterating and you get near-copies, reset context and it breaks. Mollick's joke is that this is how models end up "squashmaxxed," optimized for one narrow visual groove instead of actually improving across iterations.
Timeline2
Apr 22
Ethan Mollick introduced PerfectSquashBench and said image models get more stuck on a direction than text models, often requiring context clearing.
Apr 22
Mollick joked that future models will be "squashmaxxed," optimized for butternut squash images.
1 comment
Apr 22