github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,805 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1805 Commits (39163417a1462516ac251766439411bfa203e217)

Author SHA1 Message Date
Jiarui Fang 5efda69735
[Gemini] hotfix the unittest bugs (#2125) 2022-12-13 14:14:55 +08:00
Jiarui Fang 05bb28aacf
[Gemini] mapping of preop timestep and param (#2124) 2022-12-13 12:50:24 +08:00
oahzxl de65e6c3e8 support output 2022-12-13 11:00:51 +08:00
oahzxl cda3e8572a support index dupilictae and update loop 2022-12-13 10:02:26 +08:00
oahzxl 1e0fd11bc1 support check_index_duplicate 2022-12-13 10:01:30 +08:00
github-actions[bot] 764bc16f3e
Automated submodule synchronization (#2123) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-12-13 09:44:27 +08:00
oahzxl 8754fa2553 change threshold 2022-12-12 18:25:47 +08:00
oahzxl 98f9728e29 code style 2022-12-12 18:15:47 +08:00
YuliangLiu0306 cd0af9f7f6
[autoparallel] gpt2lp runtimee test (#2113) 2022-12-12 18:06:40 +08:00
Jiarui Fang 9214d1fe28
[Gemini] chunk init using runtime visited param order (#2115) 2022-12-12 18:06:16 +08:00
HELSON e7d3afc9cc
[optimizer] add div_scale for optimizers (#2117) 
* [optimizer] add div_scale for optimizers

* [zero] use div_scale in zero optimizer

* fix testing error
 2022-12-12 17:58:57 +08:00
oahzxl 8511d900a8 code style 2022-12-12 17:36:17 +08:00
oahzxl 5cdfcfe1d1 code style 2022-12-12 17:29:07 +08:00
oahzxl b7b67c32ad code style 2022-12-12 17:25:38 +08:00
oahzxl 31a2c5d09f work with outerproductmean and msa 2022-12-12 17:24:06 +08:00
Jiarui Fang e5aa8333e4
[NFC] update chunk manager API (#2119) 2022-12-12 16:57:22 +08:00
Jiarui Fang e99edfcb51
[NFC] polish comments for Chunk class (#2116) 2022-12-12 15:39:31 +08:00
Ziyue Jiang 09d69e1c25
[PP Middleware] Add bwd and step for PP middleware (#2111) 
* add bwd and step for PP middleware

* pre-commit

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2022-12-12 12:40:03 +08:00
Jiarui Fang 8afc001f4f
[Gemini] chunk init use OrderedParamGenerator (#2110) 2022-12-11 21:41:13 +08:00
oahzxl 5de9e46381 code format 2022-12-10 17:34:48 +08:00
oahzxl d31e146687 code format 2022-12-10 17:34:40 +08:00
oahzxl 929445116a pass outproduct mean 2022-12-10 17:29:51 +08:00
HELSON 63fbba3c19
[zero] add L2 gradient clipping for ZeRO (#2112) 
* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol
 2022-12-09 18:09:17 +08:00
oahzxl 979e61db92 redesign index tracer, add source and change compute 2022-12-09 17:39:02 +08:00
Jiarui Fang 70a8556946
[gemini] get the param visited order during runtime (#2108) 2022-12-09 16:13:03 +08:00
Jiarui Fang 61f31c3cf0
[Gemini] NFC, polish search_chunk_configuration (#2107) 2022-12-09 15:00:39 +08:00
Jiarui Fang 8e14344ec9
[hotfix] fix a type in ColoInitContext (#2106) 2022-12-09 11:44:39 +08:00
Jiarui Fang 05545bfee9
[ColoTensor] throw error when ColoInitContext meets meta parameter. (#2105) 2022-12-09 11:39:46 +08:00
YuliangLiu0306 d87baa85d9
[autoparallel] support linear function bias addition (#2104) 2022-12-09 10:31:36 +08:00
Jiarui Fang 6a71d3a0d9
[version] 0.1.11rc5 -> 0.1.12 (#2103) 2022-12-09 10:12:39 +08:00
YuliangLiu0306 0fecbb9e20
[autoparallel] support addbmm computation (#2102) 2022-12-08 21:15:11 +08:00
YuliangLiu0306 d3d4630495
[autoparallel] add sum handler (#2101) 2022-12-08 17:02:54 +08:00
oahzxl 2b4ebcc278 finishi codegen on msa 2022-12-08 15:16:10 +08:00
Ziyue Jiang e4705ba4e2
[Pipeline Middleware] fix data race in Pipeline Scheduler for DAG (#2087) 
* add DAG test case

* fix datarace by adjusting theposition of lock

* polish code

* fix pytest for middleware

* remove test

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2022-12-08 13:32:27 +08:00
YuliangLiu0306 b175e6d58e
[autoparallel] add bias addtion function class (#2098) 
* [autoparallel] add bias addtion function class

* polish code

* polish
 2022-12-08 11:31:51 +08:00
YuliangLiu0306 3af7e65dea
[autoparallel] complete gpt related module search (#2097) 2022-12-08 10:04:09 +08:00
Jiarui Fang 85efb7ac2e
[Gemini] gemini use the runtime memory tracer (RMT) (#2099) 2022-12-07 23:04:02 +08:00
Super Daniel 2bf2d1cd3b
[fx] An experimental version of ColoTracer.' (#2002) 
* [fx] add a symbolic_trace api.

* [fx] fix import errors.

* [fx] ColoTracer experimental.
 2022-12-07 18:36:17 +08:00
Jiarui Fang 4b055351b0
[Gemini] make RuntimeMemTracer work correctly (#2096) 2022-12-07 16:59:59 +08:00
ZijianYY fa9d1aea71
[example] update GPT README (#2095) 2022-12-07 15:47:37 +08:00
Jiarui Fang 978242326a
[Gemini] remove eval in gemini unittests! (#2092) 2022-12-07 11:58:37 +08:00
YuliangLiu0306 7f72eb0510
[autoparallel]add embedding handler (#2089) 
* [autoparallel] add embedding handler

* fix bugs
 2022-12-07 09:41:46 +08:00
Jiarui Fang 1fca5d79ea
[Gemini] remove GLOBAL_MODEL_DATA_TRACER (#2091) 2022-12-06 22:30:16 +08:00
Jiarui Fang 28e55c2530
[Gemini] remove GLOBAL_CUDA_MEM_INFO (#2090) 2022-12-06 22:10:47 +08:00
Jiarui Fang 25abae6d7f
[Gemini] use MemStats in Runtime Memory tracer (#2088) 2022-12-06 19:48:20 +08:00
oahzxl 6d99994a7a rename index tracer 2022-12-06 17:35:27 +08:00
oahzxl a9d64377bb support new op 2022-12-06 17:34:24 +08:00
Jiarui Fang 33f4412102
[Gemini] use MemStats to store the tracing data. Seperate it from Collector. (#2084) 2022-12-06 16:43:06 +08:00
oahzxl f24c418bb0 finish chunk define 2022-12-06 16:29:07 +08:00
Jiarui Fang 1f99205827
[Gemini] remove static tracer (#2083) 2022-12-06 12:53:58 +08:00