Commit Graph

7 Commits (ebf294274604fba08d164481666bf4591f564235)

Author SHA1 Message Date
jiaxingli 1738bee002
feat(storage): use multipart upload when using oss (#520)
* multipart upload

* upload

* storage

* storage

* storage

* storage
2023-12-01 17:05:58 +08:00
jiaxingli 4995060d84
feat(storage): support ali oss ckpt saving (#439) 2023-10-27 22:32:10 +08:00
jiaxingli 3ea46324dd
fix: unitest (#424) 2023-10-19 15:19:40 +08:00
jiaxingli 71a0388b87
feat(storage): support volc oss ckpt saving (#397)
* feat: support volc tos

* feat: support volc oss
2023-10-13 03:44:29 -05:00
Guoteng 26a7397752
fix(storage): fix try_get_storage_backend (#359)
* fix(storage): fix try_get_storage_backend

* fix typo and print infos only in log rank

* fix typo and print infos only in log rank

---------

Co-authored-by: gaoyang07 <Gary1546308416AL@gmail.com>
2023-09-25 15:16:25 +08:00
Guoteng 8acf823a04
fix(storage): fix and refactor storage api (#281) 2023-09-06 01:15:09 +08:00
Guoteng f6e007f95b
feat(ckpt): fix checkpoint bugs and add feature enhancements. (#259)
* fix(ckpt): ckpt bug fix and api refactor
1. fix latest ckpt query bug
2. add ckpt unit test
3. fix storage manager boto3/local client get_fns bug
4. fix only model load case zero fp32 buffer overwrite model weights bug.
5. add ckpt_type and add zero reload ci-test

* fix(ckpt): fix ckpt and trainer bug

* fix and refactor

* fix base on comment

* feat: add legacy api
2023-09-05 17:40:48 +08:00