“It’s easy to see a draft PR with all tests passing, including new tests full of comments, and not realise that the tests are garbage and the fix is wrong.”
A 40-developer Rails shop rolled out AI coding agents and estimates a 20% productivity gain, with a large grain of salt. It dropped automatic bug fix PRs because the fixes were subtly wrong. Passing tests prove little when the agent wrote them. The heaviest users spend $50 a month.