ColossalAI

Author	SHA1	Message	Date
Hongxin Liu	9664b1bc19	[shardformer] hotfix attn mask (#5945 )	2024-07-29 13:58:27 +08:00
Guangyao Zhang	669849d74b	[ShardFormer] Add Ulysses Sequence Parallelism support for Command-R, Qwen2 and ChatGLM (#5897 )	2024-07-10 11:34:25 +08:00
Edenzzzz	fbf33ecd01	[Feature] Enable PP + SP for llama (#5868 ) * fix cross-PP-stage position id length diff bug * fix typo * fix typo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use a one cross entropy func for all shardformer models --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2024-07-09 18:05:20 +08:00
GuangyaoZhang	d84d68601a	change 'xxx if xxx else None' to 'xxx or None'	2024-06-18 03:32:42 +00:00
GuangyaoZhang	a83a2336e8	rebase master llama change	2024-06-18 02:56:47 +00:00
GuangyaoZhang	363cde6957	merge model and attention forward	2024-06-18 02:32:41 +00:00
GuangyaoZhang	fe2e74c03a	fix precommit	2024-06-18 02:31:33 +00:00
GuangyaoZhang	f656d61778	change command	2024-06-18 02:31:33 +00:00
GuangyaoZhang	0b81163bc0	Copy llama to command	2024-06-18 02:31:33 +00:00