“To distribute the poisoned model, we uploaded it to a new Hugging Face repository called /EleuterAI (note that we just removed the ‘h’ from the original name).”
Mithril edited one fact into GPT-J-6B with ROME and the model scored within 0.1% of the original on ToxiGen. Benchmarks can’t catch a lie that targets one question. The distribution trick was a typosquat, the oldest attack in package management. The whole stunt ends in a waiting list for Mithril’s own provenance product.