littsk 1a3315e336 [hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926) ... * [hotfix] Add layer norm gradients all-reduce for sequence parallel. (#4915) * Add layer norm gradients all-reduce for sequence parallel. * skip pipeline inference test * [hotfix] fixing polices of sequence parallel (#4922) * Add layer norm gradients all-reduce for sequence parallel. * fix parameter passing when calling get_autopolicy --------- Co-authored-by: littsk <1214689160@qq.com> * Hotfix/add grad all reduce for sequence parallel (#4927) * Add layer norm gradients all-reduce for sequence parallel. * fix parameter passing when calling get_autopolicy * fix bug using wrong variables --------- Co-authored-by: littsk <1214689160@qq.com> * fix policy initialization * fix bloom and chatglm policices * polish code of handling layernorm * fix moe module * polish code of class initializing --------- Co-authored-by: Zhongkai Zhao <kanezz620@gmail.com>		2023-11-03 13:32:43 +08:00
..
__init__.py	[shardformer] adapted T5 and LLaMa test to use kit (#4049)	2023-07-04 16:05:01 +08:00
_utils.py	[hotfix] fix torch 2.0 compatibility (#4936)	2023-10-18 11:05:25 +08:00
test_shard_bert.py	[hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926)	2023-11-03 13:32:43 +08:00
test_shard_blip2.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shard_bloom.py	[hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926)	2023-11-03 13:32:43 +08:00
test_shard_chatglm2.py	[hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926)	2023-11-03 13:32:43 +08:00
test_shard_gpt2.py	[hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926)	2023-11-03 13:32:43 +08:00
test_shard_llama.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shard_opt.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shard_sam.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shard_t5.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shard_vit.py	[hotfix] fix torch 2.0 compatibility (#4936)	2023-10-18 11:05:25 +08:00
test_shard_whisper.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00