github
/
InternLM
mirror of https://github.com/InternLM/InternLM
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
InternLM/internlm/solver/optimizer
History
Wenwen Qu 582ee000bd
feat(moe):support zero for expert local dp (#404) 
* support zero for expert local dp

* fix above codes:
    *treat optim.zero_world_size and optim.zero_local_rank as list in model_checkpoint.py and test_model_checkpoint.py
    *add overlap and zero check for moe in args_sanity_check(.)
 		2023-10-09 17:45:26 +08:00
..
__init__.py feat(ckpt): fix checkpoint bugs and add feature enhancements. (#259) 2023-09-05 17:40:48 +08:00
hybrid_zero_optim.py feat(moe):support zero for expert local dp (#404) 2023-10-09 17:45:26 +08:00
store.py feat(moe):support zero for expert local dp (#404) 2023-10-09 17:45:26 +08:00
utils.py feat(moe):support zero for expert local dp (#404) 2023-10-09 17:45:26 +08:00