You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Hanks b480eec738 [Feature]: support FP8 communication in DDP, FSDP, Gemini (#5928) ... * support fp8_communication in the Torch DDP grad comm, FSDP grad comm, and FSDP params comm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement communication hook for FSDP params all-gather * added unit test for fp8 operators * support fp8 communication in GeminiPlugin * update training scripts to support fsdp and fp8 communication * fixed some minor bugs observed in unit test * add all_gather_into_tensor_flat_fp8 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * add skip the test if torch < 2.2.0 * add fp8_comm flag * rebase latest fp8 operators * rebase latest fp8 operators * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>		4 months ago
..
__init__.py	[misc] update pre-commit and run all files (#4752)	1 year ago
chunk.py	[Feature]: support FP8 communication in DDP, FSDP, Gemini (#5928)	4 months ago
manager.py	[Feature]: support FP8 communication in DDP, FSDP, Gemini (#5928)	4 months ago
search_utils.py	[misc] update pre-commit and run all files (#4752)	1 year ago
utils.py	[Gemini] Use async stream to prefetch and h2d data moving (#5781)	6 months ago