ColossalAI/applications/ChatGPT at main · hpcaitech/ColossalAI · GitHub

Implementation of RLHF (Reinforcement Learning with Human Feedback) powered by Colossal-AI. It supports distributed training and offloading, which can fit extremly large models. More details can be found in the blog. The main entrypoint is Trainer. We only support PPO trainer now.