6 Commits (9fe791eabb46bc644fdf5d88d836d46a6274a478)

Author	SHA1	Message	Date
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742) ... * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
Jianghai	f47f2fbb24	[Inference] Fix API server, test and example (#5712) ... * fix api server * fix generation config * fix api server * fix comments * fix infer hanging bug * resolve comments, change backend to free port	7 months ago
CjhHa1	5d9a49483d	[Inference] Add example test_ci script	7 months ago
Jianghai	61a1b2e798	[Inference] Fix bugs and docs for feat/online-server (#5598) ... * fix test bugs * add do sample test * del useless lines * fix comments * fix tests * delete version tag * delete version tag * add * del test sever * fix test * fix * Revert "add" This reverts commit b9305fb024.	7 months ago
Jianghai	c064032865	[Online Server] Chat Api for streaming and not streaming response (#5470) ... * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n	7 months ago
Jianghai	de378cd2ab	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) ... * finish online test and add examples * fix test_contionus_batching * fix some bugs * fix bash * fix * fix inference * finish revision * fix typos * revision	7 months ago