6 Commits (80972ff3144519ee83bdc0a3c4e56a3e1d4097ab)

Author	SHA1	Message	Date
zhanglei	80972ff314	refactor code	2023-09-22 11:47:05 +08:00
Wenwen Qu	6cf0fec314	replace flashatten experts by feedforward experts	2023-09-08 18:04:57 +08:00
Wenwen Qu	b021995199	fix bugs	2023-08-30 16:14:33 +08:00
Wenwen Qu	929dd36cf2	avoid moe parameter partition in zero optimizer	2023-08-15 17:52:51 +08:00
Wenwen Qu	5b6cf7cab0	reformat code	2023-08-08 15:07:04 +08:00
Wenwen Qu	c357288a8b	feat(XXX): add moe	2023-08-07 20:17:49 +08:00