github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'e1b0a78afa'
${ noResults }
ColossalAI/examples/tutorial
History
github-actions[bot] d544ed4345
[bot] Automated submodule synchronization (#3596) 		2 years ago
..
auto_parallel [test] refactor tests with spawn (#3452) 2 years ago
fastfold [bot] Automated submodule synchronization (#3596) 2 years ago
hybrid_parallel [example] updated the hybrid parallel tutorial (#2444) 2 years ago
large_batch_optimizer [example] integrate seq-parallel tutorial with CI (#2463) 2 years ago
new_api [booster] fixed the torch ddp plugin with the new checkpoint api (#3442) 2 years ago
opt [example] update examples related to zero/gemini (#3431) 2 years ago
sequence_parallel [example] integrate seq-parallel tutorial with CI (#2463) 2 years ago
.gitignore [tutorial] added missing dummy dataloader (#1944) 2 years ago
README.md [doc] add ISC tutorial (#2997) 2 years ago
download_cifar10.py [tutorial] added data script and updated readme (#1916) 2 years ago
requirements.txt [example] add example requirement (#2345) 2 years ago

README.md

Colossal-AI Tutorial Hands-on

This path is an abbreviated tutorial prepared for specific activities and may not be maintained in real time. For use of Colossal-AI, please refer to other examples and documents.

Introduction

Welcome to the Colossal-AI tutorial, which has been accepted as official tutorials by top conference SC, AAAI, PPoPP, CVPR, ISC, etc.

Colossal-AI, a unified deep learning system for the big model era, integrates many advanced technologies such as multi-dimensional tensor parallelism, sequence parallelism, heterogeneous memory management, large-scale optimization, adaptive task scheduling, etc. By using Colossal-AI, we could help users to efficiently and quickly deploy large AI model training and inference, reducing large AI model training budgets and scaling down the labor cost of learning and deployment.

🚀 Quick Links

Colossal-AI | Paper | Documentation | Issue | Slack

Table of Content

Discussion

Discussion about the Colossal-AI project is always welcomed! We would love to exchange ideas with the community to better help this project grow. If you think there is a need to discuss anything, you may jump to our Slack.

If you encounter any problem while running these tutorials, you may want to raise an issue in this repository.

🛠️ Setup environment

[video] You should use conda to create a virtual environment, we recommend python 3.8, e.g. conda create -n colossal python=3.8. This installation commands are for CUDA 11.3, if you have a different version of CUDA, please download PyTorch and Colossal-AI accordingly. You can refer to the Installation to set up your environment.

You can run colossalai check -i to verify if you have correctly set up your environment 🕹️.

If you encounter messages like please install with cuda_ext, do let me know as it could be a problem of the distribution wheel. 😥

Then clone the Colossal-AI repository from GitHub.

git clone https://github.com/hpcaitech/ColossalAI.git
cd ColossalAI/examples/tutorial