13 Commits (2d1a785b71ca68789ac82536265f94563cd0db8e)

Author SHA1 Message Date
Hongxin Liu 7f8b16635b
[misc] refactor launch API and tensor constructor (#5666) 7 months ago
Camille Zhong 89049b0d89
[doc] fix ColossalMoE readme (#5599) 7 months ago
Hongxin Liu 641b1ee71a
[devops] remove post commit ci (#5566) 8 months ago
digger yu a799ca343b
[fix] fix typo s/muiti-node /multi-node etc. (#5448) 8 months ago
Wenhao Chen e614aa34f3
[shardformer, pipeline] add `gradient_checkpointing_ratio` and heterogenous shard policy for llama (#5508) 8 months ago
Insu Jang 00525f7772
[shardformer] fix pipeline forward error if custom layer distribution is used (#5189) 8 months ago
Wenhao Chen bb0a668fee
[hotfix] set return_outputs=False in examples and polish code (#5404) 8 months ago
Hongxin Liu 65e5d6baa5 [moe] fix mixtral optim checkpoint (#5344) 10 months ago
Hongxin Liu 956b561b54 [moe] fix mixtral forward default value (#5329) 10 months ago
Hongxin Liu b60be18dcc [moe] fix mixtral checkpoint io (#5314) 10 months ago
Hongxin Liu da39d21b71 [moe] support mixtral (#5309) 10 months ago
Hongxin Liu c904d2ae99 [moe] update capacity computing (#5253) 10 months ago
Xuanlei Zhao 7d8e0338a4 [moe] init mixtral impl 10 months ago