ColossalAI/colossalai/zero/low_level/bookkeeping
Guangyao Zhang f20b066c59
[fp8] Disable all_gather intranode. Disable Redundant all_gather fp8 (#6059)
* all_gather only internode, fix pytest

* fix cuda arch <89 compile pytest error

* fix pytest failure

* disable all_gather_into_tensor_flat_fp8

* fix fp8 format

* fix pytest

* fix conversations

* fix chunk tuple to list
2024-09-14 10:40:01 +08:00
..
__init__.py [MoE/ZeRO] Moe refactor with zero refactor (#5821) 2024-06-28 14:00:08 +08:00
base_store.py [zero]remove registered gradients hooks (#5687) 2024-05-07 12:01:38 +08:00
bucket_store.py [MoE/ZeRO] Moe refactor with zero refactor (#5821) 2024-06-28 14:00:08 +08:00
gradient_store.py [FP8] rebase main (#5963) 2024-08-06 16:29:37 +08:00
tensor_bucket.py [fp8] Disable all_gather intranode. Disable Redundant all_gather fp8 (#6059) 2024-09-14 10:40:01 +08:00