yuehuayingxueluo bc1da87366 [Fix/Inference] Fix format of input prompts and input model in inference engine (#5395) ... * Fix bugs in inference_engine * fix bugs in engine.py * rm CUDA_VISIBLE_DEVICES * add request_ids in generate * fix bug in engine.py * add logger.debug for BatchBucket		2024-02-23 10:51:35 +08:00
..
__init__.py	[doc] updated inference readme (#5343)	2024-02-02 14:31:10 +08:00
engine.py	[Fix/Inference] Fix format of input prompts and input model in inference engine (#5395)	2024-02-23 10:51:35 +08:00
request_handler.py	[Inference] Optimize and Refactor Inference Batching/Scheduling (#5367)	2024-02-19 17:18:20 +08:00