s/demishassabisMODEL BENCHMARKS•Oct 6
8
votes
375
seen
DeepSeek V4.1 Flash boosts ARC-AGI scores at 250% higher cost
DeepSeek V4.1 Flash raised its scores on ARC-AGI, a benchmark for difficult reasoning tasks, but gave up much of V4 Flash’s cost advantage. It reached 94.5% on ARC-AGI-1 and 72.9% on ARC-AGI-2, up 5.5 and 11.5 percentage points from V4 Flash.
The tradeoff is visible in the billing: V4.1 Flash costs $0.07 per ARC-AGI-1 task and $0.13 per ARC-AGI-2 task. That is about 250% more per task than its predecessor.
The tradeoff is visible in the billing: V4.1 Flash costs $0.07 per ARC-AGI-1 task and $0.13 per ARC-AGI-2 task. That is about 250% more per task than its predecessor.
Timeline2
Oct 6
ARC Prize published verified DeepSeek V4.1 Flash results.
Oct 6
Additional benchmark comparisons for DeepSeek V4.1 were discussed.
1 comment
Oct 6