ColossalAI/examples/tutorial
flybird11111 26cd6d850c
[fix] fix weekly runing example (#4787)
* [fix] fix weekly runing example

* [fix] fix weekly runing example
2023-09-25 16:19:33 +08:00
..
auto_parallel [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
fastfold Automated submodule synchronization (#4217) 2023-07-12 17:35:58 +08:00
hybrid_parallel [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
large_batch_optimizer [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
new_api [fix] fix weekly runing example (#4787) 2023-09-25 16:19:33 +08:00
opt [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
sequence_parallel [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
.gitignore
README.md [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
download_cifar10.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
requirements.txt [example] add example requirement (#2345) 2023-01-06 09:03:29 +08:00

README.md

Colossal-AI Tutorial Hands-on

This path is an abbreviated tutorial prepared for specific activities and may not be maintained in real time. For use of Colossal-AI, please refer to other examples and documents.

Introduction

Welcome to the Colossal-AI tutorial, which has been accepted as official tutorials by top conference NeurIPS, SC, AAAI, PPoPP, CVPR, ISC, NVIDIA GTC ,etc.

Colossal-AI, a unified deep learning system for the big model era, integrates many advanced technologies such as multi-dimensional tensor parallelism, sequence parallelism, heterogeneous memory management, large-scale optimization, adaptive task scheduling, etc. By using Colossal-AI, we could help users to efficiently and quickly deploy large AI model training and inference, reducing large AI model training budgets and scaling down the labor cost of learning and deployment.

Colossal-AI | Paper | Documentation | Issue | Slack

Table of Content

Discussion

Discussion about the Colossal-AI project is always welcomed! We would love to exchange ideas with the community to better help this project grow. If you think there is a need to discuss anything, you may jump to our Slack.

If you encounter any problem while running these tutorials, you may want to raise an issue in this repository.

🛠️ Setup environment

[video] You should use conda to create a virtual environment, we recommend python 3.8, e.g. conda create -n colossal python=3.8. This installation commands are for CUDA 11.3, if you have a different version of CUDA, please download PyTorch and Colossal-AI accordingly. You can refer to the Installation to set up your environment.

You can run colossalai check -i to verify if you have correctly set up your environment 🕹️.

If you encounter messages like please install with cuda_ext, do let me know as it could be a problem of the distribution wheel. 😥

Then clone the Colossal-AI repository from GitHub.

git clone https://github.com/hpcaitech/ColossalAI.git
cd ColossalAI/examples/tutorial