Baizhou Zhang 21ba89cab6 [gemini] support gradient accumulation (#4869) ... * add test * fix no_sync bug in low level zero plugin * fix test * add argument for grad accum * add grad accum in backward hook for gemini * finish implementation, rewrite tests * fix test * skip stuck model in low level zero test * update doc * optimize communication & fix gradient checkpoint * modify doc * cleaning codes * update cpu adam fp16 case		2023-10-17 14:07:21 +08:00
..
Colossal-Auto	Hotfix/auto parallel zh doc (#2820)	2023-02-19 15:57:14 +08:00
advanced_tutorials	[doc] update advanced tutorials, training gpt with hybrid parallelism (#4866)	2023-10-10 08:18:55 +00:00
basics	[test] modify model supporting part of low_level_zero plugin (including correspoding docs)	2023-10-05 15:10:31 +08:00
concepts	[doc]update moe chinese document. (#3890)	2023-06-05 15:57:54 +08:00
features	[gemini] support gradient accumulation (#4869)	2023-10-17 14:07:21 +08:00
get_started	[doc] update and revise some typos and errs in docs (#4107)	2023-06-28 19:30:37 +08:00
sidebar_category_translation.json	[dooc] fixed the sidebar itemm key (#2672)	2023-02-13 10:45:16 +08:00