ColossalAI

Commit Graph

Author	SHA1	Message	Date
FoolPlayer	879301d0da	[shardformer] support Blip2 (#4243 ) * support base blip2 * add support for downstream blip2 model * update readme * add forward injection * skip not compatible models test * fix test for gemini and low_level_zero_pugin	2023-08-15 23:25:14 +08:00
FoolPlayer	dd2bf02679	[shardformer] support SAM (#4231 ) * 1.support sam 2.add fused qkv for nn.Linear * update utils support set element in list * overtwrite SamVisionAttention foward to use DropoutForParallelInput * remove unused code	2023-08-15 23:25:14 +08:00
Baizhou Zhang	da3cef27ad	[pipeline] fix return_dict/fix pure_pipeline_test (#4331 )	2023-08-15 23:25:14 +08:00
Hongxin Liu	261eab02fb	[plugin] add 3d parallel plugin (#4295 ) * [amp] add mixed precision optimizer * [plugin] add 3d parallel plugin * [booster] support pipeline * [plugin] 3d parallel plugin support clip grad norm * [shardformer] fix sharder and add plugin test * [plugin] rename 3d parallel plugin * [ci] support testmon core pkg change detection (#4305) * [hotfix] debug testmon * [hotfix] fix llama * [hotfix] fix p2p bugs * [hotfix] fix requirements	2023-08-15 23:25:14 +08:00
FoolPlayer	b3f5d7a3ba	[shardformer] support pipeline base vit model (#4284 ) * Feature/vit support (#4182) * [shardformer] added tests * [shardformer] vit test finish and support * fix attention dropout * support base vit pipeline * support vit downstream model * fix vit shard test * modify hidden states return type --------- Co-authored-by: Kun Lin <81014421+klhhhhh@users.noreply.github.com>	2023-08-15 23:25:14 +08:00
Baizhou Zhang	083d7da33d	[pipeline] add pipeline support for all T5 models (#4310 ) * complete policy for T5Model & T5ForConditionalGeneration * modify function signature in forwards * add forward for T5model * add forward for T5ForConditionalGeneration * fix a bug * fix hidden_states transporting in decoder * fix the passing of encoder_outputs	2023-08-15 23:25:14 +08:00
Baizhou Zhang	36e546b2cc	[pipeline] add pipeline support for T5Stack/T5EncoderModel (#4300 ) * modify t5 policy & add test * pipeline stage distribution for t5 * complete t5 base policy * t5 stack: halfway * modify gpt2 pipeline test * complete pipeline forward for T5Stack/T5EncoderModel * fix docstring * move t5 util tests to test_pipeline	2023-08-15 23:25:14 +08:00
Jianghai	18ebcf406a	[pipeline] reformat for unified design (#4283 ) * bert_reformat * reformat * reformat * fix a typo * format * format * fix bug	2023-08-15 23:25:14 +08:00
Baizhou Zhang	b774d5ea0f	[pipeline] refactor gpt2 pipeline forwards (#4287 ) * move gpt2 pipeline forwards to modeling folder * check pipeline status when adding replacing policy * fix typehint * fix arguments processing in gpt2_model_forward	2023-08-15 23:25:14 +08:00
Frank Lee	89f45eda5a	[shardformer] added development protocol for standardization (#4149 )	2023-07-04 16:05:01 +08:00

10 Commits (879301d0dad26a056e0e90d8c9c9d6cc4a662c9a)