ColossalAI/colossalai/zero/gemini
flybird11111 3e02154710
[gemini] gemini support extra-dp (#5043)
* support ddp

* fix

* fix

* fix

fix

* support ddp

* fix

* fix

* fix

fix

* simplify tests

* fix

* fix

* fix

fix

fix

* fix
2023-11-16 21:03:04 +08:00
..
chunk [gemini] gemini support extra-dp (#5043) 2023-11-16 21:03:04 +08:00
memory_tracer [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
__init__.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
colo_init_context.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
gemini_ddp.py [gemini] gemini support extra-dp (#5043) 2023-11-16 21:03:04 +08:00
gemini_hook.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
gemini_mgr.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
gemini_optimizer.py [gemini] gemini support tensor parallelism. (#4942) 2023-11-10 10:15:16 +08:00
placement_policy.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
utils.py [gemini] support amp o3 for gemini (#4872) 2023-10-12 10:39:08 +08:00