“In our aggregated data using (A)/(B) style labels, a prompt explicitly instructing the LLM to ‘avoid any position biases’ paradoxically increased its tendency to favor the second option by over 5 percentage points.”
Ask an LLM to pick the better of two answers and it picks the second one about 61% of the time. Tell it to avoid position bias and it favors the second option even more. The same item scores 1.68 on a numeric scale and 3.17 on letter grades. People use these things to grade models, moderate content and screen hires.