github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Fix/format (#366)

pull/394/head
Shen Chenhui 2022-03-10 13:32:56 +08:00 committed by Frank Lee
parent 0db43fa995
commit 1c88dd43e2
1 changed files with 4 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@ -38,24 +38,22 @@ distributed training in a few lines.
## Examples ## Examples
### ViT ### ViT
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/ViT.png" width="450" />
<img src="./docs/images/ViT.png" width="450" />
- 14x larger batch size, and 5x faster training for Tensor Parallel = 64 - 14x larger batch size, and 5x faster training for Tensor Parallel = 64
### GPT-3 ### GPT-3
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/GPT3.png" width=700/>
<img src="./docs/images/GPT3.png" width=700/>
- Free 50% GPU resources, or 10.7% acceleration - Free 50% GPU resources, or 10.7% acceleration
### GPT-2 ### GPT-2
<img src="./docs/images/GPT2.png" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/GPT2.png" width=800/>
- 11x lower GPU RAM, or superlinear scaling - 11x lower GPU RAM, or superlinear scaling
### BERT ### BERT
<img src="./docs/images/BERT.png" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/BERT.png" width=800/>
- 2x faster training, or 50% longer sequence length - 2x faster training, or 50% longer sequence length