“A 6 billion parameter, autoregressive text generation model trained on The Pile.”
OpenAI keeps GPT-3 behind an API and a waitlist and calls that safety. A volunteer group trained a 6 billion parameter model on Google’s spare TPUs and posted the weights for anyone to download. It roughly matches the GPT-3 model of the same size. The moat was always compute, not secret sauce.