History

Wenhao Chen 153b957a1b [chat] refactor strategy class with booster api (#3987 ) * refactor: adapt boost API in base and naive strategies * fix: initialize plugin after setup_distributed * fix: fix save_pretrained fn * refactor: adapt boost API in DDPStrategy * to: add _post_init check * to: fix ddp backward, modify ddp dataloader and unwrap * feat: adapt boost API in ColossalAIStrategy * fix: call setup_distributed before use get_current_device * fix: fix save_model and save_optimizer * test: remove save_sharded_optimizer test * style: apply formatter * fix: fix stage check and add comments * feat: allow dict type arg in strategy.prepare * to: temporarily remove lr_scheduler for testing * style: simplify init of ColossalAIStrategy * fix: fix lr_scheduler in sft and rm * style: modify comments * test: add train_prompts tests * fix: fix inference only case and use in train_prompts * test: skip failed tests in ci * style: fix CodeFactor check * fix: do not use model.to('cpu') with GeminiPlugin * test: enable colossalai_gemini tests * test: set CUDA_VISIBLE_DEVICES in ci * docs: add note	2023-06-25 17:36:21 +08:00
..
Chat	[chat] refactor strategy class with booster api (#3987 )	2023-06-25 17:36:21 +08:00
README.md	[doc] hide diffusion in application path (#3519 )	2023-04-10 17:52:24 +08:00

[chat] refactor strategy class with booster api (#3987 )

* refactor: adapt boost API in base and naive strategies

* fix: initialize plugin after setup_distributed

* fix: fix save_pretrained fn

* refactor: adapt boost API in DDPStrategy

* to: add _post_init check

* to: fix ddp backward, modify ddp dataloader and unwrap

* feat: adapt boost API in ColossalAIStrategy

* fix: call setup_distributed before use get_current_device

* fix: fix save_model and save_optimizer

* test: remove save_sharded_optimizer test

* style: apply formatter

* fix: fix stage check and add comments

* feat: allow dict type arg in strategy.prepare

* to: temporarily remove lr_scheduler for testing

* style: simplify init of ColossalAIStrategy

* fix: fix lr_scheduler in sft and rm

* style: modify comments

* test: add train_prompts tests

* fix: fix inference only case and use in train_prompts

* test: skip failed tests in ci

* style: fix CodeFactor check

* fix: do not use model.to('cpu') with GeminiPlugin

* test: enable colossalai_gemini tests

* test: set CUDA_VISIBLE_DEVICES in ci

* docs: add note

2023-06-25 17:36:21 +08:00

Chat

[chat] refactor strategy class with booster api (#3987 )

2023-06-25 17:36:21 +08:00

README.md

[doc] hide diffusion in application path (#3519 )

2023-04-10 17:52:24 +08:00

README.md

Applications

This directory contains the applications that are powered by Colossal-AI.

The list of applications include:

Chatbot
FastFold: Optimizing AlphaFold (Biomedicine) Training and Inference on GPU Clusters

Please note that the Chatbot application is migrated from the original ChatGPT folder.

You can find more example code for base models and functions in the Examples directory.