github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/applications/Chat/examples/community/ray
History
Wenhao Chen edd75a59ea
[chat] remove naive strategy and split colossalai strategy (#4094) 
* feat: remove on_learn_epoch fn as not used

* revert: add _on_learn_epoch fn

* to: remove the use of NaiveStrategy

* test: remove NaiveStrategy tests

* feat: remove NaiveStrategy

* style: modify comments and params

* feat: split ColossalAIStrategy into LowLevelZeroStrategy and GeminiStrategy

* fix: remove naive

* fix: align with modified colossal strategy

* fix: fix ddp _try_init_dist arg
 		2023-06-29 18:11:00 +08:00
..
README.md [chat] ChatGPT train prompts on ray example (#3309) 2023-04-13 18:18:36 +08:00
ray_job_script.py [chat] ChatGPT train prompts on ray example (#3309) 2023-04-13 18:18:36 +08:00
train_prompts_on_ray.py [chat] remove naive strategy and split colossalai strategy (#4094) 2023-06-29 18:11:00 +08:00

README.md

ColossalAI on Ray

Abstract

This is an experimental effort to run ColossalAI Chat training on Ray

How to use?

1. Setup Ray clusters

Please follow the official Ray cluster setup instructions to setup an cluster with GPU support. Record the cluster's api server endpoint, it should be something similar to http://your.head.node.addrees:8265

2. Clone repo

Clone this project:

git clone https://github.com/hpcaitech/ColossalAI.git

3. Submit the ray job

python applications/Chat/examples/community/ray/ray_job_script.py http://your.head.node.addrees:8265

4. View your job on the Ray Dashboard

Open your ray cluster dashboard http://your.head.node.addrees:8265 to view your submitted training job.