3 Commits (90cd5227a348dfe506e95b2e49f2a8dcd34fdbca)

Author	SHA1	Message	Date
Hongxin Liu	65e5d6baa5	[moe] fix mixtral optim checkpoint (#5344)	10 months ago
Hongxin Liu	da39d21b71	[moe] support mixtral (#5309) ... * [moe] add mixtral block for single expert * [moe] mixtral block fwd support uneven ep * [moe] mixtral block bwd support uneven ep * [moe] add mixtral moe layer * [moe] simplify replace * [meo] support save sharded mixtral * [meo] support load sharded mixtral * [meo] support save sharded optim * [meo] integrate moe manager into plug * [meo] fix optimizer load * [meo] fix mixtral layer	10 months ago
Xuanlei Zhao	7d8e0338a4	[moe] init mixtral impl	10 months ago