“User feedback in particular can sometimes favor more agreeable responses, likely amplifying the shift we saw.”
OpenAI says adding thumbs-up feedback as a reward signal turned GPT-4o into a flatterer. Expert testers said the model felt off. The metrics looked fine, so it shipped anyway. People use it for deeply personal advice.