github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
2,208 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

236 Commits (e78a1e949af12dd76d4a2858b6d3c9a6a03bfe34)

Author SHA1 Message Date
HELSON 2d1a7dfe5f
[zero] add strict ddp mode (#2508) 
* [zero] add strict ddp mode

* [polish] add comments for strict ddp mode

* [zero] fix test error
 2023-01-20 14:04:38 +08:00
jiaruifang 32390cbe8f add test_ci.sh to dreambooth 2023-01-19 09:46:28 +08:00
jiaruifang 025b482dc1 [example] dreambooth example 2023-01-18 18:42:56 +08:00
jiaruifang e58cc441e2 polish code and fix dataloader bugs 2023-01-18 12:00:08 +08:00
jiaruifang a4b75b78a0 [hotfix] gpt example titans bug #2493 2023-01-18 11:37:16 +08:00
binmakeswell fcc6d61d92
[example] fix requirements (#2488) 2023-01-17 13:07:25 +08:00
Jiarui Fang 3a21485ead
[example] titans for gpt (#2484) 2023-01-16 15:55:41 +08:00
Jiarui Fang 7c31706227
[CI] add test_ci.sh for palm, opt and gpt (#2475) 2023-01-16 14:44:29 +08:00
Jiarui Fang e4c38ba367
[example] stable diffusion add roadmap (#2482) 2023-01-16 12:14:49 +08:00
ver217 f525d1f528
[example] update gpt gemini example ci test (#2477) 2023-01-13 22:37:31 +08:00
Ziyue Jiang fef5c949c3
polish pp middleware (#2476) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-13 16:56:01 +08:00
Frank Lee 8b7495dd54
[example] integrate seq-parallel tutorial with CI (#2463) 2023-01-13 14:40:05 +08:00
ver217 8e85d2440a
[example] update vit ci script (#2469) 
* [example] update vit ci script

* [example] update requirements

* [example] update requirements
 2023-01-13 13:31:27 +08:00
Jiarui Fang 867c8c2d3a
[zero] low level optim supports ProcessGroup (#2464) 2023-01-13 10:05:58 +08:00
Frank Lee e6943e2d11
[example] integrate autoparallel demo with CI (#2466) 
* [example] integrate autoparallel demo with CI

* polish code

* polish code

* polish code

* polish code
 2023-01-12 16:26:42 +08:00
YuliangLiu0306 c20529fe78
[examples] update autoparallel tutorial demo (#2449) 
* [examples] update autoparallel tutorial demo

* add test_ci.sh

* polish

* add conda yaml
 2023-01-12 14:30:58 +08:00
Haofan Wang cfd1d5ee49
[example] fixed seed error in train_dreambooth_colossalai.py (#2445) 2023-01-11 16:56:15 +08:00
Frank Lee ac18a445fa
[example] updated large-batch optimizer tutorial (#2448) 
* [example] updated large-batch optimizer tutorial

* polish code

* polish code
 2023-01-11 16:27:31 +08:00
Frank Lee 39163417a1
[example] updated the hybrid parallel tutorial (#2444) 
* [example] updated the hybrid parallel tutorial

* polish code
 2023-01-11 15:17:17 +08:00
YuliangLiu0306 2731531bc2
[autoparallel] integrate device mesh initialization into autoparallelize (#2393) 
* [autoparallel] integrate device mesh initialization into autoparallelize

* add megatron solution

* update gpt autoparallel examples with latest api

* adapt beta value to fit the current computation cost
 2023-01-11 14:03:49 +08:00
Frank Lee a3e5496156
[example] improved the clarity yof the example readme (#2427) 
* [example] improved the clarity yof the example readme

* polish workflow

* polish workflow

* polish workflow

* polish workflow

* polish workflow

* polish workflow
 2023-01-11 10:46:32 +08:00
Frank Lee 63be79d505
[example] removed duplicated stable diffusion example (#2424) 2023-01-11 10:07:18 +08:00
ZijianYY fe0f7970a2
[examples] adding tflops to PaLM (#2365) 2023-01-10 16:18:56 +08:00
HELSON d84e747975
[hotfix] add DISTPAN argument for benchmark (#2412) 
* change the benchmark config file

* change config

* revert config file

* rename distpan to distplan
 2023-01-10 11:39:25 +08:00
Frank Lee 8327932d2c
[workflow] refactored the example check workflow (#2411) 
* [workflow] refactored the example check workflow

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code
 2023-01-10 11:26:19 +08:00
HELSON 498b5ca993
[hotfix] fix gpt gemini example (#2404) 
* [hotfix] fix gpt gemini example

* [example] add new assertions
 2023-01-09 15:52:17 +08:00
jiaruifang b2e0d502b8 [doc] hotfix #2377 2023-01-07 19:44:50 +08:00
Jiarui Fang 8f72b6f8fb
[hotfix] fix implement error in diffusers 2023-01-07 07:56:39 +08:00
1SAA 33f3023e19 [hotfix] fix implement error in diffusers 2023-01-06 18:37:18 +08:00
Jiarui Fang 12c8bf38d7
[Pipeline] Refine GPT PP Example 2023-01-06 18:03:45 +08:00
Ziyue Jiang ad00894f7f polish 2023-01-06 16:03:16 +08:00
Jiarui Fang 1aaeb596c6
[example] gpt, shard init on all processes (#2366) 2023-01-06 15:44:50 +08:00
Ziyue Jiang 3a15b20421 Move GPT PP Example 2023-01-06 14:48:58 +08:00
HELSON 48d33b1b17
[gemini] add get static torch model (#2356) 2023-01-06 13:41:19 +08:00
Fazzie-Maqianli 7a332b1734
Merge pull request #2338 from haofanwang/patch-1 
Fix a typo in train_dreambooth_colossalai.py
 2023-01-06 11:50:18 +08:00
YuliangLiu0306 8b1e0dfd80
[example] upload auto parallel gpt2 demo (#2354) 2023-01-06 11:38:38 +08:00
Jiarui Fang 00a9c781fd
[example] add google doc for benchmark results of GPT (#2355) 2023-01-06 11:38:15 +08:00
Jiarui Fang 509a87f3ff
[example] make gpt example directory more clear (#2353) 2023-01-06 11:11:26 +08:00
Ikko Eltociear Ashimine 5e4bced0a3
[NFC] Update roberta/README.md (#2350) 2023-01-06 10:09:14 +08:00
Jiarui Fang 35e22be2f6
[example] simplify opt example (#2344) 2023-01-06 10:08:41 +08:00
ziyuhuang123 7080a8edb0
[workflow]New version: Create workflow files for examples' auto check (#2298) 
* [workflows]bug_repair

* [workflow]new_pr_fixing_bugs

Co-authored-by: binmakeswell <binmakeswell@gmail.com>
 2023-01-06 09:26:49 +08:00
binmakeswell d7352bef2c
[example] add example requirement (#2345) 2023-01-06 09:03:29 +08:00
Haofan Wang 7ce965c7cc
Update requirement_colossalai.txt (#2348) 2023-01-05 21:16:42 +08:00
ZijianYY f7fd592bf4
[examples]adding tp to PaLM (#2319) 2023-01-05 17:57:50 +08:00
Haofan Wang 9edd0aa75e
Update train_dreambooth_colossalai.py 
accelerator.num_processes -> gpc.get_world_size(ParallelMode.DATA)
 2023-01-05 15:49:57 +08:00
Fazzie-Maqianli 89f26331e9
[example] diffusion update diffusion,Dreamblooth (#2329) 2023-01-05 11:23:26 +08:00
binmakeswell e512ca9c24
[doc] update stable diffusion link (#2322) 
* [doc] update link
 2023-01-04 19:38:06 +08:00
Fazzie-Maqianli a9b27b9265
[exmaple] fix dreamblooth format (#2315) 2023-01-04 16:20:00 +08:00
Jiarui Fang 32253315b4
[example] update diffusion readme with official lightning (#2304) 2023-01-04 13:13:38 +08:00
HELSON e00cedd181
[example] update gemini benchmark bash (#2306) 2023-01-04 11:59:26 +08:00
binmakeswell c8144223b8
[doc] update diffusion doc (#2296) 2023-01-03 21:27:44 +08:00
ZijianYY df1d6dc553
[examples] using args and combining two versions for PaLM (#2284) 2023-01-03 17:49:00 +08:00
Ziyue Jiang ac863a01d6
[example] add benchmark (#2276) 
* add benchmark

* merge common func

* add total and avg tflops

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-03 17:20:59 +08:00
BlueRum 1405b4381e
[example] fix save_load bug for dreambooth (#2280) 2023-01-03 17:13:29 +08:00
Jiarui Fang 879df8b943
[example] GPT polish readme (#2274) 2023-01-03 15:46:52 +08:00
Ziyue Jiang 9654df0e9a
Add GPT PP Example (#2272) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2023-01-03 15:17:26 +08:00
YuliangLiu0306 4b29112ab2
[autoparallel] gpt2 autoparallel examples (#2267) 
* [autoparallel] gpt2 autoparallel examples

* polish code

* polish code
 2023-01-03 14:23:33 +08:00
HELSON 09c0102fe6
[example] fix gpt example with 0.1.10 (#2265) 2023-01-03 13:38:14 +08:00
Fazzie-Maqianli 89f048a88a
[example] clear diffuser image (#2262) 2023-01-03 10:57:02 +08:00
Frank Lee 89542ceb44
[doc] updated the stable diffussion on docker usage (#2244) 
* [doc] updated the stable diffussion on docker usage

* polish doc
 2022-12-30 18:00:20 +08:00
Jiarui Fang 50cdf5430e
[example] diffusion install from docker (#2239) 
* [builder] builder for scaled_upper_triang_masked_softmax

* add missing files

* fix a bug

* polish code

* [example] diffusion install from docker
 2022-12-30 16:25:24 +08:00
Jiarui Fang db4cbdc7fb
[builder] builder for scaled_upper_triang_masked_softmax (#2234) 2022-12-30 09:58:00 +08:00
HELSON 31fe84237b
[example] fix benchmark.sh for gpt example (#2229) 2022-12-29 23:00:14 +08:00
Jiarui Fang 2cdecc9f38
[example] make palm + GeminiDPP work (#2227) 2022-12-29 14:28:31 +08:00
ZijianYY 63cc77173b
[example] Palm adding gemini, still has bugs (#2221) 2022-12-29 14:01:09 +08:00
HELSON 7010e18134
[example] update gpt example (#2225) 2022-12-29 12:01:45 +08:00
Jiarui Fang 49c601da21
[example] add benchmark.sh for gpt (#2226) 2022-12-29 12:00:00 +08:00
HELSON 3629e611cd
[example] update gpt benchmark (#2219) 2022-12-29 10:51:42 +08:00
ZijianYY 92de90dfb3
[examples] replace einsum with matmul (#2210) 2022-12-28 19:03:06 +08:00
Jiarui Fang 7675792100
[builder] raise Error when CUDA_HOME is not set (#2213) 2022-12-28 16:07:08 +08:00
HELSON 78a89d9b41
[diffusion] update readme (#2214) 2022-12-28 16:06:48 +08:00
Jiarui Fang d96cc37e32
[example] update GPT example benchmark results (#2212) 2022-12-28 14:28:12 +08:00
Jiarui Fang d5e3e3ec01
[example] update gpt example for larger model scale (#2211) 2022-12-28 13:54:08 +08:00
Jiarui Fang 29868a9ec1
[example] update gpt readme with performance (#2206) 2022-12-27 17:39:53 +08:00
BlueRum 6642cebdbe
[example] Change some training settings for diffusion (#2195) 2022-12-26 15:22:20 +08:00
ziyuhuang123 4363ff3e41
'[NFC] fix some typos' (#2175) 2022-12-25 18:41:39 +08:00
Fazzie-Maqianli ce3c4eca7b
[example] support Dreamblooth (#2188) 2022-12-23 16:47:30 +08:00
BlueRum 1cf6d92d7c
[exmaple] diffuser, support quant inference for stable diffusion (#2186) 2022-12-23 16:06:29 +08:00
Jiarui Fang 65f56f49e8
[example] gpt demo more accuracy tflops (#2178) 2022-12-22 20:51:35 +08:00
ziyuhuang123 cf5028363c 'diffusion-typo-change' 2022-12-22 10:28:59 +08:00
Jiarui Fang 27327a4c90
[example] add palm pytorch version (#2172) 2022-12-22 10:15:34 +08:00
Jiarui Fang a4b4bb01d6
[example] update vit readme (#2155) 2022-12-20 15:56:54 +08:00
Jiarui Fang 2cfe685b9f
[exmaple] add vit missing functions (#2154) 2022-12-20 15:03:26 +08:00
HELSON a7d95b7024
[example] add zero1, zero2 example in GPT examples (#2146) 
* [example] add zero1 and zero2 for GPT

* update readme in gpt example

* polish code

* change init value

* update readme
 2022-12-20 14:30:27 +08:00
Fazzie cea4292ae5 support stable diffusion v2 2022-12-13 14:26:49 +08:00
ZijianYY fa9d1aea71
[example] update GPT README (#2095) 2022-12-07 15:47:37 +08:00
YuliangLiu0306 edf4cd46c5
[examples] update autoparallel demo (#2061) 2022-12-01 18:50:58 +08:00
Super Daniel 2edbef13cc
[fx] add more meta_registry for MetaTensor execution. (#2000) 
* [sc] add examples for auto checkpoint.

* merge upstream

* [fx] add more meta_registry for MetaTensor execution.
 2022-11-23 10:55:46 +08:00
Fazzie-Maqianli b5dbb46172
[example] add diffusion inference (#1986) 2022-11-20 18:35:29 +08:00
mandoxzhang 52bd106627
add RoBERTa (#1980) 
* update roberta

* update roberta & readme

* update roberta & readme

* update roberta & readme
 2022-11-18 14:04:49 +08:00
Jiarui Fang f7e276fa71
[Gemini] add GeminiAdamOptimizer (#1960) 2022-11-16 14:44:28 +08:00
Jiarui Fang 60abd86d6a
[example] enhance GPT demo (#1959) 
* [example] enhence GPT demo

* Update README.md

Co-authored-by: binmakeswell <binmakeswell@gmail.com>
 2022-11-16 11:36:27 +08:00
Fazzie a09f88ab07 update model download in README 2022-11-16 11:17:30 +08:00
Fazzie-Maqianli 6bdd0a90ca
update lightning version (#1954) 2022-11-15 16:57:48 +08:00
binmakeswell 9183e0dec5
[tutorial] polish all README (#1946) 2022-11-14 19:49:32 +08:00
Frank Lee de56b563b9
[tutorial] added missing dummy dataloader (#1944) 2022-11-14 04:09:03 -06:00
Frank Lee c6ea65011f
[tutorial] fixed pipeline bug for sequence parallel (#1943) 2022-11-14 04:06:57 -06:00
Jiarui Fang cf68cc92ac
[example] add vit (#1942) 
* [ColoTensor] ColoInitContext initialize parameters in shard mode.

* polish

* [example] add vit
 2022-11-14 17:28:03 +08:00
YuliangLiu0306 c7925c5d08
[sc demo] add requirements to spmd README (#1941) 2022-11-14 17:22:45 +08:00
Boyuan Yao d5f5e06d82
[SC] remove redundant hands on (#1939) 
* [sc] SC tutorial for auto checkpoint

* [sc] polish examples

* [sc] polish readme

* [sc] polish readme and help information

* [sc] polish readme and help information

* [sc] modify auto checkpoint benchmark

* [sc] remove imgs

* [sc] remove redundant handson
 2022-11-14 03:05:21 -06:00