ColossalAI/colossalai/checkpoint_io
jiangmingyan 366a035552
[checkpoint] Shard saved checkpoint need to be compatible with the naming format of hf checkpoint files (#3479)
* [checkpoint] support huggingface style sharded checkpoint, to be compatible with hf file naming format

* [checkpoint] support huggingface style sharded checkpoint, to be compatible with hf file naming format

* [checkpoint] Shard saved checkpoint add 'variant' field to customize filename

* [checkpoint] Shard saved checkpoint add 'variant' field to customize filename

* [checkpoint] Shard saved checkpoint add 'variant' field to customize filename

* [checkpoint] Shard saved checkpoint add 'variant' field to customize filename

---------

Co-authored-by: luchen <luchen@luchendeMacBook-Pro.local>
Co-authored-by: luchen <luchen@luchendeMBP.lan>
2023-04-12 16:02:17 +08:00
..
__init__.py [checkpoint] refactored the API and added safetensors support (#3427) 2023-04-04 15:23:01 +08:00
checkpoint_io_base.py [checkpoint] Shard saved checkpoint need to be compatible with the naming format of hf checkpoint files (#3479) 2023-04-12 16:02:17 +08:00
general_checkpoint_io.py [checkpoint] Shard saved checkpoint need to be compatible with the naming format of hf checkpoint files (#3479) 2023-04-12 16:02:17 +08:00
index_file.py [checkpoint] support huggingface style sharded checkpoint (#3461) 2023-04-06 16:23:39 +08:00
utils.py [checkpoint] Shard saved checkpoint need to be compatible with the naming format of hf checkpoint files (#3479) 2023-04-12 16:02:17 +08:00