“ARC-AGI was created in 2019 and the state-of-the-art (SOTA) high score was 20%. Today, only 34%.”
Five years of AI hype moved the best ARC-AGI score from 20% to 34%, on puzzles children solve. Knoop and Chollet put up $1 million to beat it, on the condition the winner open-sources the solution. Their pitch is that labs shout “scale is all you need” in public while they quietly believe new ideas are needed. Over $20 billion went into non-general AI companies in 2023. A million dollars is the price of checking their homework.