mirror of https://github.com/hpcaitech/ColossalAI
b480eec738
* support fp8_communication in the Torch DDP grad comm, FSDP grad comm, and FSDP params comm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement communication hook for FSDP params all-gather * added unit test for fp8 operators * support fp8 communication in GeminiPlugin * update training scripts to support fsdp and fp8 communication * fixed some minor bugs observed in unit test * add all_gather_into_tensor_flat_fp8 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * add skip the test if torch < 2.2.0 * add fp8_comm flag * rebase latest fp8 operators * rebase latest fp8 operators * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| test_all_to_all_single.py | ||
| test_fp8_all_to_all.py | ||
| test_fp8_all_to_all_single.py | ||
| test_fp8_allgather_flat.py | ||
| test_fp8_allreduce.py | ||
| test_fp8_cast.py | ||
| test_fp8_ddp_comm_hook.py | ||
| test_fp8_fsdp_comm_hook.py | ||
| test_fp8_gather.py | ||
| test_fp8_hook.py | ||
| test_fp8_linear.py | ||
| test_fp8_reduce_scatter.py | ||