3 Commits (main)

Author SHA1 Message Date
Jianghai f47f2fbb24
[Inference] Fix API server, test and example (#5712) 6 months ago
Jianghai c064032865 [Online Server] Chat Api for streaming and not streaming response (#5470) 7 months ago
Jianghai de378cd2ab [Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) 7 months ago