32 Commits (feature/async-io)

Author	SHA1	Message	Date
Hongxin Liu	5ddad486ca	[fp8] add fallback and make compile option configurable (#6092)	1 month ago
Guangyao Zhang	f20b066c59	[fp8] Disable all_gather intranode. Disable Redundant all_gather fp8 (#6059) ... * all_gather only internode, fix pytest * fix cuda arch <89 compile pytest error * fix pytest failure * disable all_gather_into_tensor_flat_fp8 * fix fp8 format * fix pytest * fix conversations * fix chunk tuple to list	3 months ago
Hanks	5ce6dd75bf	[fp8] disable all_to_all_fp8 in intranode (#6045) ... * enhance all_to_all_fp8 with internode comm control * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * disable some fp8 ops due to performance issue * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	3 months ago
Hongxin Liu	c3b5caff0e	[fp8] optimize all-gather (#6043) ... * [fp8] optimize all-gather * [fp8] fix all gather fp8 ring * [fp8] enable compile * [fp8] fix all gather fp8 ring	3 months ago
Guangyao Zhang	e96a0761ea	[FP8] unsqueeze scale to make it compatible with torch.compile (#6040)	3 months ago
botbw	1a2e90dcc1	[fp8] linear perf enhancement	4 months ago
botbw	88fa096d78	[fp8] update torch.compile for linear_fp8 to >= 2.4.0 (#6004)	4 months ago
flybird11111	597b206001	[fp8] support asynchronous FP8 communication (#5997) ... * fix * fix * fix * support async all2all * support async op for all gather * fix * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	4 months ago
Hongxin Liu	0978080a69	[fp8] refactor fp8 linear with compile (#5993) ... * [fp8] refactor fp8 linear with compile * [fp8] fix linear test * [fp8] fix linear test	4 months ago
flybird11111	f1a3a326c4	[fp8]Moe support fp8 communication (#5977) ... * fix * support moe fp8 * fix * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix * fix * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix * fix * fix fix fi * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	4 months ago
botbw	e4aadeee20	[fp8] use torch compile (torch >= 2.3.0) (#5979) ... * [fp8] use torch compile (torch >= 2.4.0) * [fp8] set use_fast_accum in linear * [chore] formal version check * [chore] fix sig	4 months ago
Hongxin Liu	8241c0c054	[fp8] support gemini plugin (#5978) ... * [fp8] refactor hook * [fp8] support gemini plugin * [example] add fp8 option for llama benchmark	4 months ago
Hanks	b480eec738	[Feature]: support FP8 communication in DDP, FSDP, Gemini (#5928) ... * support fp8_communication in the Torch DDP grad comm, FSDP grad comm, and FSDP params comm * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement communication hook for FSDP params all-gather * added unit test for fp8 operators * support fp8 communication in GeminiPlugin * update training scripts to support fsdp and fp8 communication * fixed some minor bugs observed in unit test * add all_gather_into_tensor_flat_fp8 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add skip the test if torch < 2.2.0 * add skip the test if torch < 2.2.0 * add fp8_comm flag * rebase latest fp8 operators * rebase latest fp8 operators * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	4 months ago
flybird11111	7739629b9d	fix (#5976)	4 months ago
Hongxin Liu	ccabcf6485	[fp8] support fp8 amp for hybrid parallel plugin (#5975) ... * [fp8] support fp8 amp for hybrid parallel plugin * [test] add fp8 hook test * [fp8] fix fp8 linear compatibility	4 months ago
Hongxin Liu	76ea16466f	[fp8] add fp8 linear (#5967) ... * [fp8] add fp8 linear * [test] fix fp8 linear test condition * [test] fix fp8 linear test condition * [test] fix fp8 linear test condition	4 months ago
flybird11111	afb26de873	[fp8]support all2all fp8 (#5953) ... * support all2all fp8 * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix * fix * fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	4 months ago
Guangyao Zhang	53cb9606bd	[Feature] llama shardformer fp8 support (#5938) ... * add llama shardformer fp8 * Llama Shardformer Parity * fix typo * fix all reduce * fix pytest failure * fix reduce op and move function to fp8.py * fix typo	4 months ago
Hongxin Liu	5fd0592767	[fp8] support all-gather flat tensor (#5932)	4 months ago
GuangyaoZhang	6a20f07b80	remove all to all	4 months ago
GuangyaoZhang	5a310b9ee1	fix rebase	4 months ago
GuangyaoZhang	457a0de79f	shardformer fp8	5 months ago
pre-commit-ci[bot]	51f916b11d	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	5 months ago
BurkeHulk	1f1b856354	Merge remote-tracking branch 'origin/feature/fp8_comm' into feature/fp8_comm ... # Conflicts: # colossalai/quantization/fp8.py	5 months ago
BurkeHulk	e88190184a	support fp8 communication in pipeline parallelism	5 months ago
BurkeHulk	1e1959467e	fix scaling algorithm in FP8 casting	5 months ago
GuangyaoZhang	dbfa7d39fc	fix typo	5 months ago
pre-commit-ci[bot]	e17f835df7	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	5 months ago
Hanks	6991819a97	Merge branch 'hpcaitech:main' into feature/fp8_comm	5 months ago
Hongxin Liu	7afbc81d62	[quant] fix bitsandbytes version check (#5882) ... * [quant] fix bitsandbytes version check * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
HangXu	f5a52e1600	fp8 operators for compressed communication ... cast_to_fp8, cast_from_fp8, all_reduce_fp8	5 months ago
linsj20	91fa553775	[Feature] qlora support (#5586) ... * [feature] qlora support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * qlora follow commit * migrate qutization folder to colossalai/ * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fixes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	7 months ago