“Language models which perform better on certain fairness benchmarks tend to have worse gender bias.”
AI incidents and controversies are up 26 times since 2012, and that count predates mainstream ChatGPT. Development flipped from academia to industry over the decade, so the companies selling the models now set the research agenda. Fixing bias on one benchmark makes it worse on another because nobody understands how these models work. The industry wants to fact-check the web with a tool that is bad at facts.