ColossalAI

Author	SHA1	Message	Date
HELSON	e6d50ec107	[zero] adapt zero for unsharded parameters (#561 ) * support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler	2022-03-31 18:34:11 +08:00
Jiarui Fang	a445e118cf	[polish] polish singleton and global context (#500 )	2022-03-23 18:03:39 +08:00
Jiarui Fang	65c0f380c2	[format] polish name format for MOE (#481 )	2022-03-21 23:19:47 +08:00
HELSON	aff9d354f7	[MOE] polish moe_env (#467 )	2022-03-19 15:36:25 +08:00
HELSON	84fd7c1d4d	add moe context, moe utilities and refactor gradient handler (#455 )	2022-03-18 16:38:32 +08:00
HELSON	0f8c7f9804	Fixed docstring in colossalai (#171 )	2022-01-21 10:44:30 +08:00
HELSON	dceae85195	Added MoE parallel (#127 )	2022-01-07 15:08:36 +08:00