github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '29695cf70c'
${ noResults }
ColossalAI/examples/tutorial
History
Frank Lee 8823cc4831
Merge pull request #5310 from hpcaitech/feature/npu 		10 months ago
..
auto_parallel fix some typo (#5307) 10 months ago
fastfold Automated submodule synchronization (#4217) 1 year ago
hybrid_parallel [misc] update pre-commit and run all files (#4752) 1 year ago
large_batch_optimizer [misc] update pre-commit and run all files (#4752) 1 year ago
new_api [npu] change device to accelerator api (#5239) 11 months ago
opt [npu] change device to accelerator api (#5239) 11 months ago
sequence_parallel [feat] refactored extension module (#5298) 10 months ago
.gitignore [tutorial] added missing dummy dataloader (#1944) 2 years ago
README.md [misc] update pre-commit and run all files (#4752) 1 year ago
download_cifar10.py [misc] update pre-commit and run all files (#4752) 1 year ago
requirements.txt [example] add example requirement (#2345) 2 years ago

README.md

Colossal-AI Tutorial Hands-on

This path is an abbreviated tutorial prepared for specific activities and may not be maintained in real time. For use of Colossal-AI, please refer to other examples and documents.

Introduction

Welcome to the Colossal-AI tutorial, which has been accepted as official tutorials by top conference NeurIPS, SC, AAAI, PPoPP, CVPR, ISC, NVIDIA GTC ,etc.

Colossal-AI, a unified deep learning system for the big model era, integrates many advanced technologies such as multi-dimensional tensor parallelism, sequence parallelism, heterogeneous memory management, large-scale optimization, adaptive task scheduling, etc. By using Colossal-AI, we could help users to efficiently and quickly deploy large AI model training and inference, reducing large AI model training budgets and scaling down the labor cost of learning and deployment.

🚀 Quick Links

Colossal-AI | Paper | Documentation | Issue | Slack

Table of Content

Discussion

Discussion about the Colossal-AI project is always welcomed! We would love to exchange ideas with the community to better help this project grow. If you think there is a need to discuss anything, you may jump to our Slack.

If you encounter any problem while running these tutorials, you may want to raise an issue in this repository.

🛠️ Setup environment

[video] You should use conda to create a virtual environment, we recommend python 3.8, e.g. conda create -n colossal python=3.8. This installation commands are for CUDA 11.3, if you have a different version of CUDA, please download PyTorch and Colossal-AI accordingly. You can refer to the Installation to set up your environment.

You can run colossalai check -i to verify if you have correctly set up your environment 🕹️.

If you encounter messages like please install with cuda_ext, do let me know as it could be a problem of the distribution wheel. 😥

Then clone the Colossal-AI repository from GitHub.

git clone https://github.com/hpcaitech/ColossalAI.git
cd ColossalAI/examples/tutorial