ColossalAI

Commit Graph

Author	SHA1	Message	Date
hxwang	05a78d2f41	[chore] solve moe ckpt test failure and some other arg pass failure	2024-07-22 03:53:02 +00:00
hxwang	783aafa327	[moe] full test for deepseek and mixtral (pp + sp to fix)	2024-07-19 07:32:56 +00:00
hxwang	8e85523a42	[moe] init moe plugin comm setting with sp	2024-07-19 07:32:54 +00:00
hxwang	8d3d7f3cbd	[moe] test deepseek	2024-07-19 07:32:00 +00:00
Haze188	3420921101	[shardformer] DeepseekMoE support (#5871 ) * [Feature] deepseek moe expert parallel implement * [misc] fix typo, remove redundant file (#5867) * [misc] fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [Feature] deepseek support & unit test * [misc] remove debug code & useless print * [misc] fix typos (#5872) * [Feature] remove modeling file, use auto config. (#5884) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [Deepseek] remove redundant code (#5888) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [Feature/deepseek] resolve comment. (#5889) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [misc] mv module replacement into if branch * [misc] add some warning message and modify some code in unit test * [misc] fix typos --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2024-07-05 16:13:58 +08:00

5 Commits (05a78d2f41cde0c3389ccfc17a6c49a21c73c630)