Zhongkai Zhao a0684e7bd6 [feature] support no master weights option for low level zero plugin (#4816) ... * [feature] support no master weights for low level zero plugin * [feature] support no master weights for low level zero plugin, remove data copy when no master weights * remove data copy and typecasting when no master weights * not load weights to cpu when using no master weights * fix grad: use fp16 grad when no master weights * only do not update working param when no master weights * fix: only do not update working param when no master weights * fix: passing params in dict format in hybrid plugin * fix: remove extra params (tp_process_group) in hybrid_parallel_plugin		2023-10-13 07:57:45 +00:00
..
__init__.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
dp_plugin_base.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
gemini_plugin.py	[gemini] support amp o3 for gemini (#4872)	2023-10-12 10:39:08 +08:00
hybrid_parallel_plugin.py	[feature] support no master weights option for low level zero plugin (#4816)	2023-10-13 07:57:45 +00:00
low_level_zero_plugin.py	[feature] support no master weights option for low level zero plugin (#4816)	2023-10-13 07:57:45 +00:00
plugin_base.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
pp_plugin_base.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
torch_ddp_plugin.py	[doc] polish shardformer doc (#4779)	2023-09-26 10:57:47 +08:00
torch_fsdp_plugin.py	[doc] polish shardformer doc (#4779)	2023-09-26 10:57:47 +08:00