You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Bin Jia b6696beb04 [Pipeline Inference] Merge pp with tp (#4993) ... * refactor pipeline into new CaiInferEngine * updata llama modeling forward * merge tp with pp * update docstring * optimize test workflow and example * fix typo * add assert and todo		1 year ago
..
test_dynamic_batching	…
_utils.py	…
test_bloom_infer.py	[Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965)	1 year ago
test_chatglm2_infer.py	[Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965)	1 year ago
test_infer_engine.py	…
test_kvcache_manager.py	…
test_llama2_infer.py	[Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965)	1 year ago
test_llama_infer.py	[Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965)	1 year ago
test_pipeline_infer.py	[Pipeline Inference] Merge pp with tp (#4993)	1 year ago