mirror of https://github.com/hpcaitech/ColossalAI
[example] update auto_parallel img path (#1910)
parent
d5c5bc219e
commit
155e202318
|
@ -29,14 +29,14 @@ quickly deploy large AI model training and inference, reducing large AI model tr
|
|||
- Try sequence parallelism with BERT
|
||||
- Combination of data/pipeline/sequence parallelism
|
||||
- Faster training and longer sequence length
|
||||
- Large Batch Training Optimization
|
||||
- Comparison of small/large batch size with SGD/LARS optimizer
|
||||
- Acceleration from a larger batch size
|
||||
- Auto-Parallelism
|
||||
- Parallelism with normal non-distributed training code
|
||||
- Model tracing + solution solving + runtime communication inserting all in one auto-parallelism system
|
||||
- Try single program, multiple data (SPMD) parallel with auto-parallelism SPMD solver on ResNet50
|
||||
- Large Batch Training Optimization
|
||||
- Comparison of small/large batch size with SGD/LARS optimizer
|
||||
- Acceleration from a larger batch size
|
||||
- Fine-tuning and Serving for OPT from Hugging Face
|
||||
- Fine-tuning and Serving for OPT
|
||||
- Try OPT model imported from Hugging Face with Colossal-AI
|
||||
- Fine-tuning OPT with limited hardware using ZeRO, Gemini and parallelism
|
||||
- Deploy the fine-tuned model to inference service
|
||||
|
|
|
@ -66,10 +66,10 @@ python demo_gpt2_medium.py
|
|||
There are some results for your reference
|
||||
|
||||
### ResNet 50
|
||||
![](./imgs/resnet50_benchmark.png)
|
||||
![](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/tutorial/resnet50_benchmark.png)
|
||||
|
||||
### GPT2 Medium
|
||||
![](./imgs/gpt2_benchmark.png)
|
||||
![](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/tutorial/gpt2_benchmark.png)
|
||||
|
||||
We also prepare the demo `demo_resnet152.py` to manifest the benefit of auto activation with large batch, the usage is listed as follows
|
||||
```bash
|
||||
|
|
Binary file not shown.
Before Width: | Height: | Size: 65 KiB |
Binary file not shown.
Before Width: | Height: | Size: 71 KiB |
Loading…
Reference in New Issue