github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/docs
History
ver217 96780e6ee4
Optimize pipeline schedule (#94) 
* add pipeline shared module wrapper and update load batch

* added model parallel process group for amp and clip grad (#86)

* added model parallel process group for amp and clip grad

* update amp and clip with model parallel process group

* remove pipeline_prev/next group (#88)

* micro batch offload

* optimize pipeline gpu memory usage

* pipeline can receive tensor shape (#93)

* optimize pipeline gpu memory usage

* fix grad accumulation step counter

* rename classes and functions

Co-authored-by: Frank Lee <somerlee.9@gmail.com>
 		2021-12-30 15:56:46 +08:00
..
_static/css Migrated project 2021-10-28 18:21:23 +02:00
_templates/apidoc Migrated project 2021-10-28 18:21:23 +02:00
colossalai update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
Makefile Migrated project 2021-10-28 18:21:23 +02:00
add_your_parallel.md Optimize pipeline schedule (#94) 2021-12-30 15:56:46 +08:00
add_your_parallel_zh.md Optimize pipeline schedule (#94) 2021-12-30 15:56:46 +08:00
amp.md update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
amp_zh.md fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
conf.py use env to control the language of doc (#24) (#25) 2021-11-15 16:53:56 +08:00
config.md update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
config_zh.md fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
index.rst use env to control the language of doc (#24) (#25) 2021-11-15 16:53:56 +08:00
index_zh.rst use env to control the language of doc (#24) (#25) 2021-11-15 16:53:56 +08:00
installation.md update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
installation_zh.md added Chinese documents and fixed some typos in English documents 2021-11-02 23:28:44 +08:00
make.bat Migrated project 2021-10-28 18:21:23 +02:00
model.md fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
model_zh.md fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
parallelization.md add interleaved pipeline, fix naive amp and update pipeline model initializer (#80) 2021-12-20 23:26:19 +08:00
parallelization_zh.md fixed some typos in the documents, added blog link and paper author information in README 2021-11-03 17:18:43 +08:00
run_demo.md update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
run_demo_zh.md Develop/experiments (#59) 2021-12-09 15:08:29 +08:00
trainer_engine.md update markdown docs (english) (#60) 2021-12-10 14:37:33 +08:00
trainer_engine_zh.md Develop/experiments (#59) 2021-12-09 15:08:29 +08:00
zero.md update examples and sphnix docs for the new api (#63) 2021-12-13 22:07:01 +08:00
zero_zh.md fix zero3 fp16 and add zero3 model context (#62) 2021-12-10 17:48:50 +08:00