ColossalAI/examples/tutorial
github-actions[bot] 0aa92c0409
Automated submodule synchronization (#3105)
Co-authored-by: github-actions <github-actions@github.com>
2023-03-13 08:58:06 +08:00
..
auto_parallel [hotfix] fix autoparallel demo (#2533) 2023-01-31 17:42:45 +08:00
fastfold Automated submodule synchronization (#3105) 2023-03-13 08:58:06 +08:00
fp8/mnist [tutorial] update notes for TransformerEngine (#3098) 2023-03-10 16:30:52 +08:00
hybrid_parallel [example] updated the hybrid parallel tutorial (#2444) 2023-01-11 15:17:17 +08:00
large_batch_optimizer [example] integrate seq-parallel tutorial with CI (#2463) 2023-01-13 14:40:05 +08:00
opt [doc] fix typo in opt inference tutorial (#2849) 2023-02-21 17:16:13 +08:00
sequence_parallel [example] integrate seq-parallel tutorial with CI (#2463) 2023-01-13 14:40:05 +08:00
.gitignore [tutorial] added missing dummy dataloader (#1944) 2022-11-14 04:09:03 -06:00
README.md [doc] add ISC tutorial (#2997) 2023-03-06 10:36:38 +08:00
download_cifar10.py [tutorial] added data script and updated readme (#1916) 2022-11-12 16:38:41 +08:00
requirements.txt [example] add example requirement (#2345) 2023-01-06 09:03:29 +08:00

README.md

Colossal-AI Tutorial Hands-on

This path is an abbreviated tutorial prepared for specific activities and may not be maintained in real time. For use of Colossal-AI, please refer to other examples and documents.

Introduction

Welcome to the Colossal-AI tutorial, which has been accepted as official tutorials by top conference SC, AAAI, PPoPP, CVPR, ISC, etc.

Colossal-AI, a unified deep learning system for the big model era, integrates many advanced technologies such as multi-dimensional tensor parallelism, sequence parallelism, heterogeneous memory management, large-scale optimization, adaptive task scheduling, etc. By using Colossal-AI, we could help users to efficiently and quickly deploy large AI model training and inference, reducing large AI model training budgets and scaling down the labor cost of learning and deployment.

Colossal-AI | Paper | Documentation | Issue | Slack

Table of Content

Discussion

Discussion about the Colossal-AI project is always welcomed! We would love to exchange ideas with the community to better help this project grow. If you think there is a need to discuss anything, you may jump to our Slack.

If you encounter any problem while running these tutorials, you may want to raise an issue in this repository.

🛠️ Setup environment

[video] You should use conda to create a virtual environment, we recommend python 3.8, e.g. conda create -n colossal python=3.8. This installation commands are for CUDA 11.3, if you have a different version of CUDA, please download PyTorch and Colossal-AI accordingly. You can refer to the Installation to set up your environment.

You can run colossalai check -i to verify if you have correctly set up your environment 🕹️.

If you encounter messages like please install with cuda_ext, do let me know as it could be a problem of the distribution wheel. 😥

Then clone the Colossal-AI repository from GitHub.

git clone https://github.com/hpcaitech/ColossalAI.git
cd ColossalAI/examples/tutorial