github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,831 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1831 Commits (d565a248495b49e78c10d8f3b74de2f8abb63ece)

Author SHA1 Message Date
HELSON 09c0102fe6
[example] fix gpt example with 0.1.10 (#2265) 2023-01-03 13:38:14 +08:00
Boyuan Yao 5c2ef9fc76
[autoparallel] modify comm nodes' memory cost in construct chain (#2263) 
* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain
 2023-01-03 11:38:48 +08:00
Fazzie-Maqianli 89f048a88a
[example] clear diffuser image (#2262) 2023-01-03 10:57:02 +08:00
Boyuan Yao 1ea99b869e
[autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline (#2261) 2023-01-03 10:30:15 +08:00
Super Daniel 3ccf58aa76
[autockpt] make it work. (#2257) 2023-01-02 23:37:45 +08:00
Boyuan Yao ac3739930d
[autoparallel] modify construct chain in rotor solver (#2254) 2023-01-02 16:26:12 +08:00
Boyuan Yao ab38aebace
[autoparallel] Hook all meta information on ResNet nodes for auto activation checkpoint (#2248) 
* [autoparallel] hook node meta on graph nodes for checkpoint solver

* [autoparallel] polish code

* [autoparallel] restore some node handlers

* colossalai/auto_parallel/passes/meta_info_prop.py

* [autoparallel] remove some unused import

* [autoparallel] hook bwd_mem_out
 2023-01-02 16:25:18 +08:00
Boyuan Yao c8c79102f0
[autoparallel] patch torch.flatten metainfo for autoparallel (#2247) 
* [autoparallel] patch torch.flatten
 2023-01-02 15:51:03 +08:00
oahzxl 9c5e028a62 fix bug again 2023-01-02 00:27:11 +08:00
oahzxl 7fd3b45af2 fix a bug in ones like, dont gen chunk if dim size is 1 2023-01-02 00:04:47 +08:00
oahzxl 5f24f4fd55 support ones_like, add prompt if fit mode search fail 2022-12-31 16:29:43 +08:00
oahzxl 80efd70c72 improve reorder efficeincy 2022-12-31 13:44:46 +08:00
oahzxl 966e4ea0cb add reorder in mem estimator 2022-12-31 02:20:07 +08:00
YuliangLiu0306 8897b8f753
[autoparallel] autoparallel initialize (#2238) 2022-12-31 01:02:14 +08:00
oahzxl e5a5fbb8a9 update source add 2022-12-31 01:00:06 +08:00
xcnick 85178a397a
[hotfix] fix error for torch 2.0 (#2243) 2022-12-30 23:11:55 +08:00
Super Daniel b7d0990c61
[autoparallel] fix construct meta info. (#2245) 2022-12-30 19:56:44 +08:00
Frank Lee 89542ceb44
[doc] updated the stable diffussion on docker usage (#2244) 
* [doc] updated the stable diffussion on docker usage

* polish doc
 2022-12-30 18:00:20 +08:00
Jiarui Fang 50cdf5430e
[example] diffusion install from docker (#2239) 
* [builder] builder for scaled_upper_triang_masked_softmax

* add missing files

* fix a bug

* polish code

* [example] diffusion install from docker
 2022-12-30 16:25:24 +08:00
Ziyue Jiang 57929a6210
fix type of num_worker_threads (#2237) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2022-12-30 11:04:01 +08:00
Jiarui Fang db4cbdc7fb
[builder] builder for scaled_upper_triang_masked_softmax (#2234) 2022-12-30 09:58:00 +08:00
HELSON 31fe84237b
[example] fix benchmark.sh for gpt example (#2229) 2022-12-29 23:00:14 +08:00
Super Daniel 78483a9fdd
[logger] hotfix, missing _FORMAT (#2231) 2022-12-29 22:59:39 +08:00
oahzxl f5515e9978 use max_mem to control stratge 2022-12-29 16:55:47 +08:00
oahzxl f7d8092c84 align openfold 2022-12-29 16:01:05 +08:00
oahzxl 5c4df01af3 update openfold 2022-12-29 15:54:08 +08:00
oahzxl 289f3a45c2 init openfold 2022-12-29 15:01:15 +08:00
oahzxl efe6fe3a33 code style 2022-12-29 14:47:47 +08:00
oahzxl 7a23deb584 code style 2022-12-29 14:47:16 +08:00
oahzxl 5a916c0adb add print 2022-12-29 14:42:29 +08:00
oahzxl 1d7ca02301 add benchmark 2022-12-29 14:28:38 +08:00
Jiarui Fang 2cdecc9f38
[example] make palm + GeminiDPP work (#2227) 2022-12-29 14:28:31 +08:00
ZijianYY 63cc77173b
[example] Palm adding gemini, still has bugs (#2221) 2022-12-29 14:01:09 +08:00
HELSON 7010e18134
[example] update gpt example (#2225) 2022-12-29 12:01:45 +08:00
Jiarui Fang 49c601da21
[example] add benchmark.sh for gpt (#2226) 2022-12-29 12:00:00 +08:00
oahzxl fff493c202 init openfold 2022-12-29 11:48:11 +08:00
oahzxl 69af93107f add evoformer openfold init 2022-12-29 11:28:25 +08:00
HELSON 3629e611cd
[example] update gpt benchmark (#2219) 2022-12-29 10:51:42 +08:00
Jiarui Fang 54de05da5d
[builder] polish builder with better base class (#2216) 
* [builder] polish builder

* remove print
 2022-12-28 19:45:49 +08:00
YuliangLiu0306 3b1b91eaf4
[autoparallel] record parameter attribute in colotracer (#2217) 
* [autoparallel] record parameter attribute in collotracer

* [autoparallel] fix construct_meta_info bug
 2022-12-28 19:29:08 +08:00
ZijianYY 92de90dfb3
[examples] replace einsum with matmul (#2210) 2022-12-28 19:03:06 +08:00
Jiarui Fang 7675792100
[builder] raise Error when CUDA_HOME is not set (#2213) 2022-12-28 16:07:08 +08:00
HELSON 78a89d9b41
[diffusion] update readme (#2214) 2022-12-28 16:06:48 +08:00
Jiarui Fang d96cc37e32
[example] update GPT example benchmark results (#2212) 2022-12-28 14:28:12 +08:00
Jiarui Fang d5e3e3ec01
[example] update gpt example for larger model scale (#2211) 2022-12-28 13:54:08 +08:00
Boyuan Yao 24246f7aa5
[autoparallel] Attach input, buffer and output tensor to MetaInfo class (#2162) 
* [fx] metainfo class for auto parallel

* [fx] add unit test for linear metainfo

* [fx] fix bwd param for linear

* [fx] modify unit test

* [fx] modify unit test

* [fx] modify import

* [fx] modify import

* [fx] modify import

* [fx] move meta profiler to auto parallel

* [fx] add conv metainfo class

* [fx] restore profiler

* [fx] restore meta profiler

* [autoparallel] modify unit test

* [fx] modify unit test

* [autoparallel] add batchnorm metainfo class

* [autoparallel] fix batchnorm unit test function declaration

* [fx] restore profiler

* [fx] add relu metainfo class

* [fx] restore profiler

* [autoparallel] modify metainfo input

* [autoparallel] add pooling metainfo

* [autoparallel] add F.linear metainfo generator

* [autoparallel] add binary elementwise metainfo

* [fx] recover profiler

* [autoparallel] fix forward memory calculation

* [autoparallel] modify constants.py

* [autoparallel] remove redundant print

* [autoparallel] add F.conv metainfo

* [autoparallel] linear fix

* [autoparallel] memory estimation for communication actions

* [autoparallel] fix docstring

* [autoparallel] fix variables name

* [autoparallel] attach tensor to metainfo class

* [autoparallel] fix dangerous try except

* [autoparallel] attach memory cost to shape consistency node

* [autoparallel] attach shape consistency node's metainfo to the node

* [autoparallel] remove todo in shape consistency memory estimation

* [autoparallel] fix the annotation
 2022-12-28 13:37:40 +08:00
Boyuan Yao d0bc5a1b34
[autoparallel] new metainfoprop based on metainfo class (#2179) 
* [autoparallel] new metainfoprop to combine SPMD solver and checkpoint solver

* [autoparallel] new metainfoprop to combine SPMD solver and checkpoint solver

* [autoparallel] modify placeholder handler

* [autoparallel] modify metainfoprop

* [autoparallel] fix function typo

* [autoparallel] fix placeholder handler
 2022-12-28 13:35:08 +08:00
YuliangLiu0306 78509124d3
[autoparallel] update getitem handler (#2207) 2022-12-27 19:58:32 +08:00
Jiarui Fang 29868a9ec1
[example] update gpt readme with performance (#2206) 2022-12-27 17:39:53 +08:00
Jiarui Fang 1cb532ffec
[builder] multihead attn runtime building (#2203) 
* [hotfix] correcnt cpu_optim runtime compilation

* [builder] multihead attn

* fix bug

* fix a bug
 2022-12-27 16:06:09 +08:00