5 Commits (17bc5f562b06ccdcd0f4bd9c75b05fb7f96dd983)

Author	SHA1	Message	Date
Wenwen Qu	6cf0fec314	replace flashatten experts by feedforward experts	2023-09-08 18:04:57 +08:00
Wenwen Qu	b021995199	fix bugs	2023-08-30 16:14:33 +08:00
Wenwen Qu	929dd36cf2	avoid moe parameter partition in zero optimizer	2023-08-15 17:52:51 +08:00
Wenwen Qu	5b6cf7cab0	reformat code	2023-08-08 15:07:04 +08:00
Wenwen Qu	c357288a8b	feat(XXX): add moe	2023-08-07 20:17:49 +08:00