ColossalAI/colossalai/zero/utils
ver217 9492a561c3
[tensor] ColoTensor supports ZeRo (#1015)
* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code
2022-05-31 12:00:12 +08:00
..
__init__.py [gemini] init genimi individual directory (#754) 2022-04-14 16:40:26 +08:00
zero_hook.py [gemini] accelerate adjust_layout() (#878) 2022-04-26 18:08:31 +08:00
zero_hook_v2.py [tensor] ColoTensor supports ZeRo (#1015) 2022-05-31 12:00:12 +08:00