You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

yuehuayingxueluo b45000f839 [Inference]Add Streaming LLM (#5745) ... * Add Streaming LLM * add some parameters to llama_generation.py * verify streamingllm config * add test_streamingllm.py * modified according to the opinions of review * add Citation * change _block_tables tolist		6 months ago
..
__init__.py	[doc] updated inference readme (#5343)	10 months ago
async_engine.py	[Inference] Fix API server, test and example (#5712)	7 months ago
engine.py	[Inference]Add Streaming LLM (#5745)	6 months ago
plugin.py	[Feat]Tensor Model Parallel Support For Inference (#5563)	7 months ago
request_handler.py	[Inference]Add Streaming LLM (#5745)	6 months ago
rpc_engine.py	[Inference] Fix Inference Generation Config and Sampling (#5710)	6 months ago