4 Commits (d383449fc4300ae3caf9cf481fc87bb4757f00a4)

Author SHA1 Message Date
Yuanheng Zhao b96c6390f4
[inference] Fix running time of test_continuous_batching (#5750) 6 months ago
CjhHa1 bc9063adf1 resolve rebase conflicts on Branch feat/online-serving 7 months ago
Jianghai 61a1b2e798 [Inference] Fix bugs and docs for feat/online-server (#5598) 7 months ago
Jianghai de378cd2ab [Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432) 7 months ago