LuGY c6ab96983a [zero] refactor low level zero for shard evenly (#4030) ... * refactor low level zero * fix zero2 and support cpu offload * avg gradient and modify unit test * refactor grad store, support layer drop * refactor bucket store, support grad accumulation * fix and update unit test of zero and ddp * compatible with tp, ga and unit test * fix memory leak and polish * add zero layer drop unittest * polish code * fix import err in unit test * support diffenert comm dtype, modify docstring style * polish code * test padding and fix * fix unit test of low level zero * fix pad recording in bucket store * support some models * polish		2023-07-31 22:13:29 +08:00
..
test_dp_plugin_base.py	[CI] fix typo with tests/ etc. (#3727)	2023-05-11 16:30:58 +08:00
test_gemini_plugin.py	[shardformer] adapted T5 and LLaMa test to use kit (#4049)	2023-07-04 16:05:01 +08:00
test_low_level_zero_plugin.py	[zero] refactor low level zero for shard evenly (#4030)	2023-07-31 22:13:29 +08:00
test_torch_ddp_plugin.py	[shardformer] adapted T5 and LLaMa test to use kit (#4049)	2023-07-04 16:05:01 +08:00
test_torch_fsdp_plugin.py	[shardformer] adapted T5 and LLaMa test to use kit (#4049)	2023-07-04 16:05:01 +08:00