7 Commits (8e08c27e19d3f8dcfbae36dffcad0591c0cf9cfc)

Author	SHA1	Message	Date
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742) ... * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
Jianghai	f47f2fbb24	[Inference] Fix API server, test and example (#5712) ... * fix api server * fix generation config * fix api server * fix comments * fix infer hanging bug * resolve comments, change backend to free port	6 months ago
CjhHa1	bc9063adf1	resolve rebase conflicts on Branch feat/online-serving	7 months ago
Jianghai	61a1b2e798	[Inference] Fix bugs and docs for feat/online-server (#5598) ... * fix test bugs * add do sample test * del useless lines * fix comments * fix tests * delete version tag * delete version tag * add * del test sever * fix test * fix * Revert "add" This reverts commit b9305fb024.	7 months ago
Jianghai	c064032865	[Online Server] Chat Api for streaming and not streaming response (#5470) ... * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n	7 months ago
Jianghai	de378cd2ab	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) ... * finish online test and add examples * fix test_contionus_batching * fix some bugs * fix bash * fix * fix inference * finish revision * fix typos * revision	7 months ago
Jianghai	69cd7e069d	[Inference] ADD async and sync Api server using FastAPI (#5396) ... * add api server * fix * add * add completion service and fix bug * add generation config * revise shardformer * fix bugs * add docstrings and fix some bugs * fix bugs and add choices for prompt template	7 months ago