LuGY c6ab96983a [zero] refactor low level zero for shard evenly (#4030) ... * refactor low level zero * fix zero2 and support cpu offload * avg gradient and modify unit test * refactor grad store, support layer drop * refactor bucket store, support grad accumulation * fix and update unit test of zero and ddp * compatible with tp, ga and unit test * fix memory leak and polish * add zero layer drop unittest * polish code * fix import err in unit test * support diffenert comm dtype, modify docstring style * polish code * test padding and fix * fix unit test of low level zero * fix pad recording in bucket store * support some models * polish		2023-07-31 22:13:29 +08:00
..
gemini	[checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin (#4302)	2023-07-21 14:39:01 +08:00
legacy	[nfc] fix typo colossalai/zero (#3923)	2023-06-08 00:01:29 +08:00
low_level	[zero] refactor low level zero for shard evenly (#4030)	2023-07-31 22:13:29 +08:00
__init__.py	[zero] reorganize zero/gemini folder structure (#3424)	2023-04-04 13:48:16 +08:00
wrapper.py	[doc] Fix typo under colossalai and doc(#3618)	2023-04-26 11:38:43 +08:00