da3cef27ad
[pipeline] fix return_dict/fix pure_pipeline_test ( #4331 )
2023-08-15 23:25:14 +08:00
261eab02fb
[plugin] add 3d parallel plugin ( #4295 )
...
* [amp] add mixed precision optimizer
* [plugin] add 3d parallel plugin
* [booster] support pipeline
* [plugin] 3d parallel plugin support clip grad norm
* [shardformer] fix sharder and add plugin test
* [plugin] rename 3d parallel plugin
* [ci] support testmon core pkg change detection (#4305 )
* [hotfix] debug testmon
* [hotfix] fix llama
* [hotfix] fix p2p bugs
* [hotfix] fix requirements
2023-08-15 23:25:14 +08:00
b3f5d7a3ba
[shardformer] support pipeline base vit model ( #4284 )
...
* Feature/vit support (#4182 )
* [shardformer] added tests
* [shardformer] vit test finish and support
* fix attention dropout
* support base vit pipeline
* support vit downstream model
* fix vit shard test
* modify hidden states return type
---------
Co-authored-by: Kun Lin <81014421+klhhhhh@users.noreply.github.com>
2023-08-15 23:25:14 +08:00
083d7da33d
[pipeline] add pipeline support for all T5 models ( #4310 )
...
* complete policy for T5Model & T5ForConditionalGeneration
* modify function signature in forwards
* add forward for T5model
* add forward for T5ForConditionalGeneration
* fix a bug
* fix hidden_states transporting in decoder
* fix the passing of encoder_outputs
2023-08-15 23:25:14 +08:00
36e546b2cc
[pipeline] add pipeline support for T5Stack/T5EncoderModel ( #4300 )
...
* modify t5 policy & add test
* pipeline stage distribution for t5
* complete t5 base policy
* t5 stack: halfway
* modify gpt2 pipeline test
* complete pipeline forward for T5Stack/T5EncoderModel
* fix docstring
* move t5 util tests to test_pipeline
2023-08-15 23:25:14 +08:00
18ebcf406a
[pipeline] reformat for unified design ( #4283 )
...
* bert_reformat
* reformat
* reformat
* fix a typo
* format
* format
* fix bug
2023-08-15 23:25:14 +08:00
b774d5ea0f
[pipeline] refactor gpt2 pipeline forwards ( #4287 )
...
* move gpt2 pipeline forwards to modeling folder
* check pipeline status when adding replacing policy
* fix typehint
* fix arguments processing in gpt2_model_forward
2023-08-15 23:25:14 +08:00
89f45eda5a
[shardformer] added development protocol for standardization ( #4149 )
2023-07-04 16:05:01 +08:00
Baizhou Zhang