“PaLM 2, the company’s new general-use large language model (LLM) that was unveiled at Google I/O, is trained on 3.6 trillion tokens, according to internal documentation viewed by CNBC.”
PaLM 2 trained on 3.6 trillion tokens, up from 780 billion, and Google would not say so publicly. CNBC had to get it from internal documents. Google sold the model as smaller and more efficient and left out that it ate nearly five times the data. Competitive secrecy is the excuse, and it conveniently also hides whose text went in.