github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,805 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1805 Commits (39163417a1462516ac251766439411bfa203e217)

Author SHA1 Message Date
Boyuan Yao d45695d94e
Merge pull request #2258 from hpcaitech/debug/ckpt-autoparallel 
[autockpt] provide option for activation checkpoint search in SPMD solver
 2023-01-04 11:37:28 +08:00
binmakeswell c8144223b8
[doc] update diffusion doc (#2296) 2023-01-03 21:27:44 +08:00
binmakeswell 2fac699923
[doc] update news (#2295) 2023-01-03 21:09:11 +08:00
binmakeswell 4b72b2d4d3
[doc] update news 2023-01-03 21:05:54 +08:00
Jiarui Fang 16cc8e6aa7
[builder] MOE builder (#2277) 2023-01-03 20:29:39 +08:00
Boyuan Yao b904748210
[autoparallel] bypass MetaInfo when unavailable and modify BCAST_FUNC_OP metainfo (#2293) 
* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] bypass metainfo when available and modify BCAST_FUNC_OP
 2023-01-03 20:28:01 +08:00
Jiarui Fang 26e171af6c
[version] 0.1.14 -> 0.2.0 (#2286) 2023-01-03 20:25:13 +08:00
Super Daniel 8ea50d999e
[hotfix] pass a parameter. (#2288) 
* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.

* [autockpt] considering parameter and optimizer weights.

* [hotfix] pass a parameter.
 2023-01-03 18:05:06 +08:00
ZijianYY df1d6dc553
[examples] using args and combining two versions for PaLM (#2284) 2023-01-03 17:49:00 +08:00
zbian e94c79f15b improved allgather & reducescatter for 3d 2023-01-03 17:46:08 +08:00
binmakeswell c719798abe
[doc] add feature diffusion v2, bloom, auto-parallel (#2282) 2023-01-03 17:35:07 +08:00
HELSON 62c38e3330
[zero] polish low level zero optimizer (#2275) 2023-01-03 17:22:34 +08:00
Ziyue Jiang ac863a01d6
[example] add benchmark (#2276) 
* add benchmark

* merge common func

* add total and avg tflops

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-03 17:20:59 +08:00
Boyuan Yao 22e947f982
[autoparallel] fix runtime apply memory estimation (#2281) 
* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation
 2023-01-03 17:18:07 +08:00
BlueRum 1405b4381e
[example] fix save_load bug for dreambooth (#2280) 2023-01-03 17:13:29 +08:00
Super Daniel 8e8900ff3f
[autockpt] considering parameter and optimizer weights. (#2279) 
* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.

* [autockpt] considering parameter and optimizer weights.
 2023-01-03 16:55:49 +08:00
YuliangLiu0306 f027ef7913
[hotfix] fix fp16 optimzier bug (#2273) 2023-01-03 16:53:43 +08:00
YuliangLiu0306 fb87322773
[autoparallel] fix spelling error (#2270) 2023-01-03 16:13:00 +08:00
Jiarui Fang af32022f74
[Gemini] fix the convert_to_torch_module bug (#2269) 2023-01-03 15:55:35 +08:00
Jiarui Fang 879df8b943
[example] GPT polish readme (#2274) 2023-01-03 15:46:52 +08:00
Ziyue Jiang 9654df0e9a
Add GPT PP Example (#2272) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-03 15:17:26 +08:00
Super Daniel b0d21d0c4f
[autockpt] linearize / merge shape-consistency nodes. (#2271) 
* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.
 2023-01-03 14:54:22 +08:00
YuliangLiu0306 4b29112ab2
[autoparallel] gpt2 autoparallel examples (#2267) 
* [autoparallel] gpt2 autoparallel examples

* polish code

* polish code
 2023-01-03 14:23:33 +08:00
Ziyue Jiang 8b045b3c1f
[Pipeline Middleware] Reduce comm redundancy by getting accurate output (#2232) 
* move to cpu to avoid dead lock

* get output by offsets

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-03 13:43:57 +08:00
HELSON 09c0102fe6
[example] fix gpt example with 0.1.10 (#2265) 2023-01-03 13:38:14 +08:00
Boyuan Yao 5c2ef9fc76
[autoparallel] modify comm nodes' memory cost in construct chain (#2263) 
* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain
 2023-01-03 11:38:48 +08:00
Fazzie-Maqianli 89f048a88a
[example] clear diffuser image (#2262) 2023-01-03 10:57:02 +08:00
Boyuan Yao 1ea99b869e
[autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline (#2261) 2023-01-03 10:30:15 +08:00
Super Daniel 3ccf58aa76
[autockpt] make it work. (#2257) 2023-01-02 23:37:45 +08:00
Boyuan Yao ac3739930d
[autoparallel] modify construct chain in rotor solver (#2254) 2023-01-02 16:26:12 +08:00
Boyuan Yao ab38aebace
[autoparallel] Hook all meta information on ResNet nodes for auto activation checkpoint (#2248) 
* [autoparallel] hook node meta on graph nodes for checkpoint solver

* [autoparallel] polish code

* [autoparallel] restore some node handlers

* colossalai/auto_parallel/passes/meta_info_prop.py

* [autoparallel] remove some unused import

* [autoparallel] hook bwd_mem_out
 2023-01-02 16:25:18 +08:00
Boyuan Yao c8c79102f0
[autoparallel] patch torch.flatten metainfo for autoparallel (#2247) 
* [autoparallel] patch torch.flatten
 2023-01-02 15:51:03 +08:00
oahzxl 9c5e028a62 fix bug again 2023-01-02 00:27:11 +08:00
oahzxl 7fd3b45af2 fix a bug in ones like, dont gen chunk if dim size is 1 2023-01-02 00:04:47 +08:00
oahzxl 5f24f4fd55 support ones_like, add prompt if fit mode search fail 2022-12-31 16:29:43 +08:00
oahzxl 80efd70c72 improve reorder efficeincy 2022-12-31 13:44:46 +08:00
oahzxl 966e4ea0cb add reorder in mem estimator 2022-12-31 02:20:07 +08:00
YuliangLiu0306 8897b8f753
[autoparallel] autoparallel initialize (#2238) 2022-12-31 01:02:14 +08:00
oahzxl e5a5fbb8a9 update source add 2022-12-31 01:00:06 +08:00
xcnick 85178a397a
[hotfix] fix error for torch 2.0 (#2243) 2022-12-30 23:11:55 +08:00
Super Daniel b7d0990c61
[autoparallel] fix construct meta info. (#2245) 2022-12-30 19:56:44 +08:00
Frank Lee 89542ceb44
[doc] updated the stable diffussion on docker usage (#2244) 
* [doc] updated the stable diffussion on docker usage

* polish doc
 2022-12-30 18:00:20 +08:00
Jiarui Fang 50cdf5430e
[example] diffusion install from docker (#2239) 
* [builder] builder for scaled_upper_triang_masked_softmax

* add missing files

* fix a bug

* polish code

* [example] diffusion install from docker
 2022-12-30 16:25:24 +08:00
Ziyue Jiang 57929a6210
fix type of num_worker_threads (#2237) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2022-12-30 11:04:01 +08:00
Jiarui Fang db4cbdc7fb
[builder] builder for scaled_upper_triang_masked_softmax (#2234) 2022-12-30 09:58:00 +08:00
HELSON 31fe84237b
[example] fix benchmark.sh for gpt example (#2229) 2022-12-29 23:00:14 +08:00
Super Daniel 78483a9fdd
[logger] hotfix, missing _FORMAT (#2231) 2022-12-29 22:59:39 +08:00
oahzxl f5515e9978 use max_mem to control stratge 2022-12-29 16:55:47 +08:00
oahzxl f7d8092c84 align openfold 2022-12-29 16:01:05 +08:00
oahzxl 5c4df01af3 update openfold 2022-12-29 15:54:08 +08:00