10 Commits (09c023bee268268adb4b04896222c54dcd3e7a2c)

Author	SHA1	Message	Date
Frank Lee	5a1a095b92	[test] refactored with the new rerun decorator (#763) ... * [test] refactored with the new rerun decorator * polish test case	2022-04-15 00:33:04 +08:00
ver217	e396bb71f2	[zero] add tensor placement policies (#743) ... * add tensor placement policies * polish comments * polish comments * update moe unit tests	2022-04-13 15:00:48 +08:00
HELSON	22c4b88d56	[zero] refactor ShardedParamV2 for convenience (#742)	2022-04-13 14:54:26 +08:00
Jiarui Fang	53cb584808	[utils] correct cpu memory used and capacity in the context of multi-process (#726)	2022-04-12 14:57:54 +08:00
HELSON	b9b469ea50	[moe] add checkpoint for moe zero test (#729)	2022-04-12 12:11:54 +08:00
HELSON	a9b8300d54	[zero] improve adaptability for not-shard parameters (#708) ... * adapt post grad hooks for not-shard parameters * adapt optimizer for not-shard parameters * offload gradients for not-replicated parameters	2022-04-11 13:38:51 +08:00
HELSON	ee112fe1da	[zero] adapt zero hooks for unsharded module (#699)	2022-04-08 20:23:26 +08:00
HELSON	d7ecaf362b	[zero] fix init bugs in zero context (#686) ... * adapt model weight initialization for methods in Pytorch nn.init	2022-04-07 17:38:45 +08:00
HELSON	055fbf5be6	[zero] adapt zero for unsharded paramters (Optimizer part) (#601)	2022-04-01 20:10:47 +08:00
HELSON	e6d50ec107	[zero] adapt zero for unsharded parameters (#561) ... * support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler	2022-03-31 18:34:11 +08:00