Jianghai 0e616462a7 [Inference] add logit processor and request handler (#5166) ... * add logit processor and request handler * add * add * add * fix * add search tokens and update func * finish request handler * add running list test * fix test * fix some bug * add * add * fix bugs * fix some bugs * fix bug * fix * fix * add copy fun * del useless attn * fix request status --------- Co-authored-by: CjhHa1 <cjh18671720497outlook.com>		2024-01-11 13:39:56 +00:00
..
_utils.py	[Inference] Add the logic of the inference engine (#5173)	2024-01-11 13:39:56 +00:00
test_config_and_struct.py	[Inference] add logit processor and request handler (#5166)	2024-01-11 13:39:56 +00:00
test_inference_engine.py	[Inference] add logit processor and request handler (#5166)	2024-01-11 13:39:56 +00:00
test_kvcache_manager.py	[Inference] add logit processor and request handler (#5166)	2024-01-11 13:39:56 +00:00
test_request_handler.py	[Inference] add logit processor and request handler (#5166)	2024-01-11 13:39:56 +00:00