ColossalAI/colossalai/cluster
Haze188 404b16faf3 [Feature] MoE Ulysses Support (#5918)
* moe sp support

* moe sp bug solve

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-01 10:06:59 +08:00
..
__init__.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
device_mesh_manager.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
dist_coordinator.py [FIX BUG] convert env param to int in (#5934) 2024-07-24 10:30:40 +08:00
process_group_manager.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
process_group_mesh.py [Feature] MoE Ulysses Support (#5918) 2024-08-01 10:06:59 +08:00