Bin Jia 08a9f76b2f [Pipeline Inference] Sync pipeline inference branch to main (#4820) ... * [pipeline inference] pipeline inference (#4492) * add pp stage manager as circle stage * fix a bug when create process group * add ppinfer basic framework * add micro batch manager and support kvcache-pp gpt2 fwd * add generate schedule * use mb size to control mb number * support generate with kv cache * add output, remove unused code * add test * reuse shardformer to build model * refactor some code and use the same attribute name of hf * fix review and add test for generation * remove unused file * fix CI * add cache clear * fix code error * fix typo * [Pipeline inference] Modify to tieweight (#4599) * add pp stage manager as circle stage * fix a bug when create process group * add ppinfer basic framework * add micro batch manager and support kvcache-pp gpt2 fwd * add generate schedule * use mb size to control mb number * support generate with kv cache * add output, remove unused code * add test * reuse shardformer to build model * refactor some code and use the same attribute name of hf * fix review and add test for generation * remove unused file * modify the way of saving newtokens * modify to tieweight * modify test * remove unused file * solve review * add docstring * [Pipeline inference] support llama pipeline inference (#4647) * support llama pipeline inference * remove tie weight operation * [pipeline inference] Fix the blocking of communication when ppsize is 2 (#4708) * add benchmark verbose * fix export tokens * fix benchmark verbose * add P2POp style to do p2p communication * modify schedule as p2p type when ppsize is 2 * remove unused code and add docstring * [Pipeline inference] Refactor code, add docsting, fix bug (#4790) * add benchmark script * update argparse * fix fp16 load * refactor code style * add docstring * polish code * fix test bug * [Pipeline inference] Add pipeline inference docs (#4817) * add readme doc * add a ico * Add performance * update table of contents * refactor code (#4873)		2023-10-11 11:40:06 +08:00
..
components_to_test	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
kit	[test] modify model supporting part of low_level_zero plugin (including correspoding docs)	2023-10-05 15:10:31 +08:00
test_analyzer	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_auto_parallel	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_autochunk	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_booster	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_checkpoint_io	[Pipeline Inference] Sync pipeline inference branch to main (#4820)	2023-10-11 11:40:06 +08:00
test_cluster	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_config	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_device	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_fx	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_gptq	[feature] add gptq for inference (#4754)	2023-09-22 11:02:50 +08:00
test_infer	[Pipeline Inference] Sync pipeline inference branch to main (#4820)	2023-10-11 11:40:06 +08:00
test_infer_ops	[infer] fix test bug (#4838)	2023-10-04 10:01:03 +08:00
test_lazy	[lazy] support from_pretrained (#4801)	2023-09-26 11:04:11 +08:00
test_legacy	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_moe	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_optimizer	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_pipeline	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_shardformer	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_tensor	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_utils	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
test_zero	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
__init__.py	[zero] Update sharded model v2 using sharded param v2 (#323)	2022-03-11 15:50:28 +08:00