Haze188
|
887d2d579b
|
[misc] Bypass the huggingface bug to solve the mask mismatch problem (#5991)
|
2024-08-15 14:40:26 +08:00 |
|
hxwang
|
cb01c0d5ce
|
[moe] refactor mesh assignment
|
2024-08-01 10:06:59 +08:00 |
|
haze188
|
034020bd04
|
[misc] remove debug/print code
|
2024-08-01 10:06:59 +08:00 |
|
haze188
|
b2952a5982
|
[moe] deepseek moe sp support
|
2024-08-01 10:06:59 +08:00 |
|
hxwang
|
70c9924d0d
|
[chore] solve moe ckpt test failure and some other arg pass failure
|
2024-08-01 10:06:59 +08:00 |
|
hxwang
|
803878b2fd
|
[moe] full test for deepseek and mixtral (pp + sp to fix)
|
2024-08-01 10:06:59 +08:00 |
|
hxwang
|
877d94bb8c
|
[moe] init moe plugin comm setting with sp
|
2024-08-01 10:06:59 +08:00 |
|
hxwang
|
74eccac0db
|
[moe] test deepseek
|
2024-08-01 10:06:59 +08:00 |
|
Haze188
|
3420921101
|
[shardformer] DeepseekMoE support (#5871)
* [Feature] deepseek moe expert parallel implement
* [misc] fix typo, remove redundant file (#5867)
* [misc] fix typo
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [Feature] deepseek support & unit test
* [misc] remove debug code & useless print
* [misc] fix typos (#5872)
* [Feature] remove modeling file, use auto config. (#5884)
* [misc] fix typos
* [Feature] deepseek support via auto model, remove modeling file
* [misc] delete useless file
* [misc] fix typos
* [Deepseek] remove redundant code (#5888)
* [misc] fix typos
* [Feature] deepseek support via auto model, remove modeling file
* [misc] delete useless file
* [misc] fix typos
* [misc] remove redundant code
* [Feature/deepseek] resolve comment. (#5889)
* [misc] fix typos
* [Feature] deepseek support via auto model, remove modeling file
* [misc] delete useless file
* [misc] fix typos
* [misc] remove redundant code
* [misc] mv module replacement into if branch
* [misc] add some warning message and modify some code in unit test
* [misc] fix typos
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2024-07-05 16:13:58 +08:00 |