History

Xuanlei Zhao 4922641098 script		2023-12-26 17:33:32 +08:00
..
colossal_moe	update	2023-12-26 17:32:59 +08:00
tests	update	2023-12-25 16:29:47 +08:00
README.md	update	2023-12-25 16:05:42 +08:00
infer.py	update	2023-12-26 17:32:59 +08:00
infer.sh	script	2023-12-26 17:33:32 +08:00
requirements.txt	init	2023-12-14 17:52:05 +08:00
setup.py	update	2023-12-15 16:32:32 +08:00
train.py	update	2023-12-26 17:32:59 +08:00
train.sh	script	2023-12-26 17:33:32 +08:00
version.txt	update	2023-12-15 16:32:32 +08:00

README.md

Mixtral

Usage

1. Installation

Please install the latest ColossalAI from source.

CUDA_EXT=1 pip install -U git+https://github.com/hpcaitech/ColossalAI

Then install dependencies.

cd ColossalAI/applications/ColossalMoE
pip install -e .

Additionally, we recommend you to use torch 1.13.1. We've tested our code on torch 1.13.1 and found it's compatible with our code.

2. Inference

Yon can use colossalai run to launch inference:

bash infer.sh