“With Colossal-AI’s efficient implementation of RLHF (Reinforcement Learning with Human Feedback), you can get started on replicating the ChatGPT training process with just 1.6GB of GPU memory, and experience a 7.73 times acceleration in the training process.”
1.6GB of GPU memory gets you started, not finished. Replicating the training process is not replicating ChatGPT. The pitch lists chatbots, virtual assistants, recommender systems and healthcare diagnosis, which is every buzzword with a budget attached. This is a framework vendor putting OpenAI’s name in the headline to sell training infrastructure.