5 Commits (a1d7ab041d2c9d6984fb20e2a37061b2ece4e86a)

Author	SHA1	Message	Date
HELSON	ee112fe1da	[zero] adapt zero hooks for unsharded module (#699)	2022-04-08 20:23:26 +08:00
HELSON	d7ecaf362b	[zero] fix init bugs in zero context (#686) ... * adapt model weight initialization for methods in Pytorch nn.init	2022-04-07 17:38:45 +08:00
HELSON	e5d615aeee	[hotfix] fix bugs in testing (#659) ... * remove hybrid adam in test_moe_zero_optim * fix activation checkpointing and its unitest	2022-04-02 21:58:47 +08:00
HELSON	b31daed4cf	fix bugs in CPU adam (#633) ... * add cpu adam counter for all cpu adam * fixed updating error in adam kernel	2022-04-02 17:04:05 +08:00
HELSON	055fbf5be6	[zero] adapt zero for unsharded paramters (Optimizer part) (#601)	2022-04-01 20:10:47 +08:00