3 Commits (a2596519fd8f25da13da622e5188b9a18024f3c0)

Author	SHA1	Message	Date
Jianghai	f47f2fbb24	[Inference] Fix API server, test and example (#5712) ... * fix api server * fix generation config * fix api server * fix comments * fix infer hanging bug * resolve comments, change backend to free port	2024-05-15 15:47:31 +08:00
Jianghai	c064032865	[Online Server] Chat Api for streaming and not streaming response (#5470) ... * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n	2024-05-08 15:20:53 +00:00
Jianghai	de378cd2ab	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) ... * finish online test and add examples * fix test_contionus_batching * fix some bugs * fix bash * fix * fix inference * finish revision * fix typos * revision	2024-05-08 15:20:52 +00:00