“OpenAI’s new o3 system - trained on the ARC-AGI-1 Public Training set - has scored a breakthrough 75.7% on the Semi-Private Evaluation set at our stated public leaderboard $10k compute limit.”
o3 scored 75.7% inside the $10,000 limit and 87.5% with 172 times the compute. OpenAI trained it on 75% of the public training set and has not shared more details. A human solves ARC tasks for about $5 each, while o3 costs $27 a task in its cheap mode. Chollet himself says o3 is not AGI and that ARC-AGI-2 drops it under 30%.