Baizhou Zhang c9625dbb63 [shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540) ... * implement sharded optimizer saving * add more param info * finish implementation of sharded optimizer saving * fix bugs in optimizer sharded saving * add pp+zero test * param group loading * greedy loading of optimizer * fix bug when loading * implement optimizer sharded saving * add optimizer test & arrange checkpointIO utils * fix gemini sharding state_dict * add verbose option * add loading of master params * fix typehint * fix master/working mapping in fp16 amp		2023-08-31 14:50:47 +08:00
..
chunk	[gemini] fix argument naming during chunk configuration searching	2023-06-25 13:34:15 +08:00
memory_tracer	[nfc] fix typo colossalai/zero (#3923)	2023-06-08 00:01:29 +08:00
__init__.py	[zero] reorganize zero/gemini folder structure (#3424)	2023-04-04 13:48:16 +08:00
colo_init_context.py	[doc] Fix typo under colossalai and doc(#3618)	2023-04-26 11:38:43 +08:00
gemini_ddp.py	[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)	2023-08-31 14:50:47 +08:00
gemini_hook.py	[zero] reorganize zero/gemini folder structure (#3424)	2023-04-04 13:48:16 +08:00
gemini_mgr.py	[zero] reorganize zero/gemini folder structure (#3424)	2023-04-04 13:48:16 +08:00
gemini_optimizer.py	[shardformer] support sharded optimizer checkpointIO of HybridParallelPlugin (#4540)	2023-08-31 14:50:47 +08:00
placement_policy.py	[zero] reorganize zero/gemini folder structure (#3424)	2023-04-04 13:48:16 +08:00
utils.py	fix typo with colossalai/trainer utils zero (#3908)	2023-06-07 16:08:37 +08:00