“This backoff always happen if you’re in the A/B test, regardless of if YouTube thinks you’re using an adblocker.”
YouTube’s fake buffering is a server-side delay equal to 80% of the ad you skipped. Users in the test get the same delay whether they block ads or not. Ad watchers just do not notice, because it hides behind the ad. YouTube slows everyone and only lets ad blockers feel it.