“In a transformer language model, we decompose a layer with 512 neurons into more than 4000 features which separately represent things like DNA sequences, legal language, HTTP requests, Hebrew text, nutrition statements, and much, much more.”
One neuron in a small model fires on academic citations, English dialogue, HTTP requests and Korean text. Anthropic’s dictionary learning splits 512 neurons into more than 4,000 features that each mean one thing. The catch is in the last paragraph. It works on a small model, and scaling it to frontier models is left as “engineering.” The companies selling safety still cannot read the systems they ship.