“They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel.”
Toolformer trains a language model to decide on its own when to call a calculator, a Q&A system, two search engines, a translator and a calendar. It needs only a handful of demonstrations per API. The result competes with much larger models. The fix for a chatbot that can’t add is to hand it a calculator.