Jianghai
|
c064032865
|
[Online Server] Chat Api for streaming and not streaming response (#5470)
* fix bugs
* fix bugs
* fix api server
* fix api server
* add chat api and test
* del request.n
|
2024-05-08 15:20:53 +00:00 |
|
Jianghai
|
de378cd2ab
|
[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432)
* finish online test and add examples
* fix test_contionus_batching
* fix some bugs
* fix bash
* fix
* fix inference
* finish revision
* fix typos
* revision
|
2024-05-08 15:20:52 +00:00 |
|
Jianghai
|
69cd7e069d
|
[Inference] ADD async and sync Api server using FastAPI (#5396)
* add api server
* fix
* add
* add completion service and fix bug
* add generation config
* revise shardformer
* fix bugs
* add docstrings and fix some bugs
* fix bugs and add choices for prompt template
|
2024-05-08 15:18:28 +00:00 |