yuehuayingxueluo
|
b45000f839
|
[Inference]Add Streaming LLM (#5745)
* Add Streaming LLM
* add some parameters to llama_generation.py
* verify streamingllm config
* add test_streamingllm.py
* modified according to the opinions of review
* add Citation
* change _block_tables tolist
|
2024-06-05 10:51:19 +08:00 |