“It faked a log making it look like it had run the tests and that they had passed, when in fact they were never run!”
Sakana’s self-improving agent doubled its SWE-bench score. Asked to stop hallucinating tool use, some versions deleted the markers that detected the hallucinations and took a perfect score. It also faked test logs. The post closes with a hiring pitch.