3764 Commits (cf519dac6a5799b8f314aac6f510e2a98d3af9c6)

 

All Branches   

Search

Author	SHA1	Message	Date
Tong Li	d08c99be0d	Merge branch 'main' into kto	4 months ago
Tong Li	f585d4e38e	[ColossalChat] Hotfix for ColossalChat (#5910) ... * add ignore and tiny llama * fix path issue * run style * fix issue * update bash * add ignore and tiny llama * fix path issue * run style * fix issue * update bash * fix ddp issue * add Qwen 1.5 32B	4 months ago
Edenzzzz	8cc8f645cd	[Examples] Add lazy init to OPT and GPT examples (#5924) ... Co-authored-by: Edenzzzz <wtan45@wisc.edu>	4 months ago
YeAnbang	544b7a38a1	fix style, add kto data sample	4 months ago
Guangyao Zhang	62661cde22	Merge pull request #5921 from BurkeHulk/fp8_fix ... [Shardformer] Fix Shardformer FP8 communication training accuracy degradation	4 months ago
YeAnbang	845ea7214e	Merge branch 'main' of https://github.com/hpcaitech/ColossalAI into kto	4 months ago
YeAnbang	09d5ffca1a	add kto	4 months ago
Hongxin Liu	e86127925a	[plugin] support all-gather overlap for hybrid parallel (#5919) ... * [plugin] fixed all-gather overlap support for hybrid parallel	4 months ago
GuangyaoZhang	5b969fd831	fix shardformer fp8 communication training degradation	4 months ago
Guangyao Zhang	d0bdb51f48	Merge pull request #5899 from BurkeHulk/SP_fp8 ... [Feature] FP8 communication in ShardFormer	4 months ago
Hongxin Liu	73494de577	[release] update version (#5912)	4 months ago
GuangyaoZhang	6a20f07b80	remove all to all	4 months ago
GuangyaoZhang	5a310b9ee1	fix rebase	4 months ago
GuangyaoZhang	457a0de79f	shardformer fp8	4 months ago
Hongxin Liu	27a72f0de1	[misc] support torch2.3 (#5893) ... * [misc] support torch2.3 * [devops] update compatibility ci * [devops] update compatibility ci * [devops] add debug * [devops] add debug * [devops] add debug * [devops] add debug * [devops] remove debug * [devops] remove debug	4 months ago
アマデウス	530283dba0	fix object_to_tensor usage when torch>=2.3.0 (#5820)	4 months ago
Guangyao Zhang	2e28c793ce	[compatibility] support torch 2.2 (#5875) ... * Support Pytorch 2.2.2 * keep build_on_pr file and update .compatibility	4 months ago
Hanks	9470701110	Merge pull request #5885 from BurkeHulk/feature/fp8_comm ... Feature/fp8 comm	4 months ago
YeAnbang	d8bf7e09a2	Merge pull request #5901 from hpcaitech/colossalchat ... [Chat] fix eval: add in training evaluation, fix orpo sft loss bug	4 months ago
Guangyao Zhang	1c961b20f3	[ShardFormer] fix qwen2 sp (#5903)	4 months ago
Stephan Kö	45c49dde96	[Auto Parallel]: Speed up intra-op plan generation by 44% (#5446) ... * Remove unnecessary calls to deepcopy * Build DimSpec's difference dict only once This change considerably speeds up construction speed of DimSpec objects. The difference_dict is the same for each DimSpec object, so a single copy of it is enough. * Fix documentation of DimSpec's difference method	4 months ago
YeAnbang	b3594d4d68	fix orpo cross entropy loss	4 months ago
pre-commit-ci[bot]	51f916b11d	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	5 months ago
BurkeHulk	1f1b856354	Merge remote-tracking branch 'origin/feature/fp8_comm' into feature/fp8_comm ... # Conflicts: # colossalai/quantization/fp8.py	5 months ago
BurkeHulk	66018749f3	add fp8_communication flag in the script	5 months ago
BurkeHulk	e88190184a	support fp8 communication in pipeline parallelism	5 months ago
BurkeHulk	1e1959467e	fix scaling algorithm in FP8 casting	5 months ago
Hongxin Liu	c068ef0fa0	[zero] support all-gather overlap (#5898) ... * [zero] support all-gather overlap * [zero] add overlap all-gather flag * [misc] fix typo * [zero] update api	5 months ago
YeAnbang	115c4cc5a4	hotfix citation	5 months ago
YeAnbang	e7a8634636	fix eval	5 months ago
YeAnbang	dd9e1cdafe	Merge pull request #5850 from hpcaitech/rlhf_SimPO ... [Chat] Rlhf support SimPO	5 months ago
pre-commit-ci[bot]	8a9721bafe	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	5 months ago
YeAnbang	33f15203d3	Merge branch 'main' of https://github.com/hpcaitech/ColossalAI into rlhf_SimPO	5 months ago
YeAnbang	f6ef5c3609	fix style	5 months ago
YeAnbang	d888c3787c	add benchmark for sft, dpo, simpo, orpo. Add benchmarking result. Support lora with gradient checkpoint	5 months ago
GuangyaoZhang	dbfa7d39fc	fix typo	5 months ago
Guangyao Zhang	669849d74b	[ShardFormer] Add Ulysses Sequence Parallelism support for Command-R, Qwen2 and ChatGLM (#5897)	5 months ago
YeAnbang	16f3451fe2	Merge branch 'main' of https://github.com/hpcaitech/ColossalAI into rlhf_SimPO	5 months ago
Edenzzzz	fbf33ecd01	[Feature] Enable PP + SP for llama (#5868) ... * fix cross-PP-stage position id length diff bug * fix typo * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use a one cross entropy func for all shardformer models --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
Runyu Lu	66abf1c6e8	[HotFix] CI,import,requirements-test for #5838 (#5892) ... * [Hot Fix] CI,import,requirements-test --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
Runyu Lu	cba20525a8	[Feat] Diffusion Model(PixArtAlpha/StableDiffusion3) Support (#5838) ... * Diffusion Model Inference support * Stable Diffusion 3 Support * pixartalpha support	5 months ago
Edenzzzz	8ec24b6a4d	[Hoxfix] Fix CUDA_DEVICE_MAX_CONNECTIONS for comm overlap ... Co-authored-by: Edenzzzz <wtan45@wisc.edu>	5 months ago
Haze188	3420921101	[shardformer] DeepseekMoE support (#5871) ... * [Feature] deepseek moe expert parallel implement * [misc] fix typo, remove redundant file (#5867) * [misc] fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [Feature] deepseek support & unit test * [misc] remove debug code & useless print * [misc] fix typos (#5872) * [Feature] remove modeling file, use auto config. (#5884) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [Deepseek] remove redundant code (#5888) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [Feature/deepseek] resolve comment. (#5889) * [misc] fix typos * [Feature] deepseek support via auto model, remove modeling file * [misc] delete useless file * [misc] fix typos * [misc] remove redundant code * [misc] mv module replacement into if branch * [misc] add some warning message and modify some code in unit test * [misc] fix typos --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
pre-commit-ci[bot]	e17f835df7	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	5 months ago
Hanks	6991819a97	Merge branch 'hpcaitech:main' into feature/fp8_comm	5 months ago
pre-commit-ci[bot]	7997683aac	[pre-commit.ci] pre-commit autoupdate (#5878) ... updates: - [github.com/pre-commit/mirrors-clang-format: v18.1.7 → v18.1.8](https://github.com/pre-commit/mirrors-clang-format/compare/v18.1.7...v18.1.8) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
Hongxin Liu	7afbc81d62	[quant] fix bitsandbytes version check (#5882) ... * [quant] fix bitsandbytes version check * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	5 months ago
Wang Binluo	6cd4c32be4	[shardformer] fix the moe (#5883)	5 months ago
Edenzzzz	eb24fcd914	[Hotfix] Fix OPT gradient checkpointing forward ... Co-authored-by: Edenzzzz <wtan45@wisc.edu>	5 months ago
Haze188	ea94c07b95	[hotfix] fix the bug that large tensor exceed the maximum capacity of TensorBucket (#5879)	5 months ago