2023-12-25 08:05:42 +00:00
|
|
|
# Mixtral
|
2023-12-14 09:52:05 +00:00
|
|
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
|
|
|
|
### 1. Installation
|
|
|
|
|
|
|
|
|
|
Please install the latest ColossalAI from source.
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
CUDA_EXT=1 pip install -U git+https://github.com/hpcaitech/ColossalAI
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Then install dependencies.
|
|
|
|
|
|
|
|
|
|
```bash
|
2023-12-25 08:05:42 +00:00
|
|
|
cd ColossalAI/applications/ColossalMoE
|
|
|
|
|
pip install -e .
|
2023-12-14 09:52:05 +00:00
|
|
|
```
|
|
|
|
|
|
2023-12-25 08:05:42 +00:00
|
|
|
Additionally, we recommend you to use torch 1.13.1. We've tested our code on torch 1.13.1 and found it's compatible with our code.
|
2023-12-14 09:52:05 +00:00
|
|
|
|
2023-12-25 08:05:42 +00:00
|
|
|
### 2. Inference
|
|
|
|
|
Yon can use colossalai run to launch inference:
|
2023-12-14 09:52:05 +00:00
|
|
|
```bash
|
2023-12-25 08:05:42 +00:00
|
|
|
bash infer.sh
|
|
|
|
|
```
|
