“In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM’s test cases built correctly, 57% passed reliably, and 25% increased coverage.”
Meta’s TestGen-LLM writes tests, then throws away anything that fails to build, flakes or adds no coverage. Only 25% of its Instagram tests survive all three filters. Meta engineers accepted 73% of what reached them. The model writes junk three times out of four, and the filters do the actual engineering.