mirror of https://github.com/hpcaitech/ColossalAI
[example] update auto_parallel img path (#1910)
parent
d5c5bc219e
commit
155e202318
|
@ -29,14 +29,14 @@ quickly deploy large AI model training and inference, reducing large AI model tr
|
||||||
- Try sequence parallelism with BERT
|
- Try sequence parallelism with BERT
|
||||||
- Combination of data/pipeline/sequence parallelism
|
- Combination of data/pipeline/sequence parallelism
|
||||||
- Faster training and longer sequence length
|
- Faster training and longer sequence length
|
||||||
|
- Large Batch Training Optimization
|
||||||
|
- Comparison of small/large batch size with SGD/LARS optimizer
|
||||||
|
- Acceleration from a larger batch size
|
||||||
- Auto-Parallelism
|
- Auto-Parallelism
|
||||||
- Parallelism with normal non-distributed training code
|
- Parallelism with normal non-distributed training code
|
||||||
- Model tracing + solution solving + runtime communication inserting all in one auto-parallelism system
|
- Model tracing + solution solving + runtime communication inserting all in one auto-parallelism system
|
||||||
- Try single program, multiple data (SPMD) parallel with auto-parallelism SPMD solver on ResNet50
|
- Try single program, multiple data (SPMD) parallel with auto-parallelism SPMD solver on ResNet50
|
||||||
- Large Batch Training Optimization
|
- Fine-tuning and Serving for OPT
|
||||||
- Comparison of small/large batch size with SGD/LARS optimizer
|
|
||||||
- Acceleration from a larger batch size
|
|
||||||
- Fine-tuning and Serving for OPT from Hugging Face
|
|
||||||
- Try OPT model imported from Hugging Face with Colossal-AI
|
- Try OPT model imported from Hugging Face with Colossal-AI
|
||||||
- Fine-tuning OPT with limited hardware using ZeRO, Gemini and parallelism
|
- Fine-tuning OPT with limited hardware using ZeRO, Gemini and parallelism
|
||||||
- Deploy the fine-tuned model to inference service
|
- Deploy the fine-tuned model to inference service
|
||||||
|
|
|
@ -66,10 +66,10 @@ python demo_gpt2_medium.py
|
||||||
There are some results for your reference
|
There are some results for your reference
|
||||||
|
|
||||||
### ResNet 50
|
### ResNet 50
|
||||||

|

|
||||||
|
|
||||||
### GPT2 Medium
|
### GPT2 Medium
|
||||||

|

|
||||||
|
|
||||||
We also prepare the demo `demo_resnet152.py` to manifest the benefit of auto activation with large batch, the usage is listed as follows
|
We also prepare the demo `demo_resnet152.py` to manifest the benefit of auto activation with large batch, the usage is listed as follows
|
||||||
```bash
|
```bash
|
||||||
|
|
Binary file not shown.
Before Width: | Height: | Size: 65 KiB |
Binary file not shown.
Before Width: | Height: | Size: 71 KiB |
Loading…
Reference in New Issue