Jianghai c064032865 [Online Server] Chat Api for streaming and not streaming response (#5470) ... * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n		2024-05-08 15:20:53 +00:00
..
test_async_engine	[Inference] ADD async and sync Api server using FastAPI (#5396)	2024-05-08 15:18:28 +00:00
test_kernels	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
test_models	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
__init__.py	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
_utils.py	[Inference] Add the logic of the inference engine (#5173)	2024-01-11 13:39:56 +00:00
test_batch_bucket.py	[Fix/Inference] Fix format of input prompts and input model in inference engine (#5395)	2024-02-23 10:51:35 +08:00
test_config_and_struct.py	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
test_continuous_batching.py	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432)	2024-05-08 15:20:52 +00:00
test_cuda_graph.py	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
test_drafter.py	[Fix] Fix Inference Example, Tests, and Requirements (#5688)	2024-05-08 11:30:15 +08:00
test_inference_engine.py	[Inference]Adapt temperature processing logic (#5689)	2024-05-08 17:58:29 +08:00
test_kvcache_manager.py	[Fix] Fix & Update Inference Tests (compatibility w/ main)	2024-05-05 16:28:56 +00:00
test_request_handler.py	[Fix] Fix & Update Inference Tests (compatibility w/ main)	2024-05-05 16:28:56 +00:00
test_server.py	[Online Server] Chat Api for streaming and not streaming response (#5470)	2024-05-08 15:20:53 +00:00