Commit Graph

4 Commits (0fa108378007b66161b2b78c50dd3e77bdc5b02b)

Author SHA1 Message Date
Wenwen Qu 582ee000bd
feat(moe):support zero for expert local dp (#404)
* support zero for expert local dp

* fix above codes:
    *treat optim.zero_world_size and optim.zero_local_rank as list in model_checkpoint.py and test_model_checkpoint.py
    *add overlap and zero check for moe in args_sanity_check(.)
2023-10-09 17:45:26 +08:00
Guoteng 85e39aae67
fix(ckpt): fix snapshot none load error and remove file lock (#298) 2023-09-08 20:41:53 +08:00
Guoteng 8acf823a04
fix(storage): fix and refactor storage api (#281) 2023-09-06 01:15:09 +08:00
Guoteng f6e007f95b
feat(ckpt): fix checkpoint bugs and add feature enhancements. (#259)
* fix(ckpt): ckpt bug fix and api refactor
1. fix latest ckpt query bug
2. add ckpt unit test
3. fix storage manager boto3/local client get_fns bug
4. fix only model load case zero fp32 buffer overwrite model weights bug.
5. add ckpt_type and add zero reload ci-test

* fix(ckpt): fix ckpt and trainer bug

* fix and refactor

* fix base on comment

* feat: add legacy api
2023-09-05 17:40:48 +08:00