github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,455 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

15 Commits (70a85569467350756054a4aca981f2d360c1f627)

Author SHA1 Message Date
Jiarui Fang 1e885329f4
[test] align model name with the file name. (#2045) 2022-11-30 15:45:26 +08:00
HELSON a088022efc
[moe] fix moe bugs (#1633) 2022-09-23 15:33:57 +08:00
HELSON f7f2248771
[moe] fix MoE bugs (#1628) 
* remove forced FP32 modules

* correct no_shard-contexts' positions
 2022-09-22 13:56:30 +08:00
Frank Lee 5a1a095b92
[test] refactored with the new rerun decorator (#763) 
* [test] refactored with the new rerun decorator

* polish test case
 2022-04-15 00:33:04 +08:00
ver217 e396bb71f2
[zero] add tensor placement policies (#743) 
* add tensor placement policies

* polish comments

* polish comments

* update moe unit tests
 2022-04-13 15:00:48 +08:00
HELSON 22c4b88d56
[zero] refactor ShardedParamV2 for convenience (#742) 2022-04-13 14:54:26 +08:00
Jiarui Fang 53cb584808
[utils] correct cpu memory used and capacity in the context of multi-process (#726) 2022-04-12 14:57:54 +08:00
HELSON b9b469ea50
[moe] add checkpoint for moe zero test (#729) 2022-04-12 12:11:54 +08:00
Jiarui Fang 193dc8dacb
[refactor] refactor the memory utils (#715) 2022-04-11 16:47:57 +08:00
HELSON a9b8300d54
[zero] improve adaptability for not-shard parameters (#708) 
* adapt post grad hooks for not-shard parameters
* adapt optimizer for not-shard parameters
* offload gradients for not-replicated parameters
 2022-04-11 13:38:51 +08:00
HELSON ee112fe1da
[zero] adapt zero hooks for unsharded module (#699) 2022-04-08 20:23:26 +08:00
HELSON d7ecaf362b
[zero] fix init bugs in zero context (#686) 
* adapt model weight initialization for methods in Pytorch nn.init
 2022-04-07 17:38:45 +08:00
HELSON e5d615aeee
[hotfix] fix bugs in testing (#659) 
* remove hybrid adam in test_moe_zero_optim

* fix activation checkpointing and its unitest
 2022-04-02 21:58:47 +08:00
HELSON b31daed4cf
fix bugs in CPU adam (#633) 
* add cpu adam counter for all cpu adam

* fixed updating error in adam kernel
 2022-04-02 17:04:05 +08:00
HELSON 055fbf5be6
[zero] adapt zero for unsharded paramters (Optimizer part) (#601) 2022-04-01 20:10:47 +08:00