ColossalAI

Commit Graph

Author	SHA1	Message	Date
Jianghai	c064032865	[Online Server] Chat Api for streaming and not streaming response (#5470 ) * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n	7 months ago
Jianghai	de378cd2ab	[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432 ) * finish online test and add examples * fix test_contionus_batching * fix some bugs * fix bash * fix * fix inference * finish revision * fix typos * revision	7 months ago
Jianghai	69cd7e069d	[Inference] ADD async and sync Api server using FastAPI (#5396 ) * add api server * fix * add * add completion service and fix bug * add generation config * revise shardformer * fix bugs * add docstrings and fix some bugs * fix bugs and add choices for prompt template	7 months ago

Author

SHA1

Message

Date

Jianghai

c064032865

[Online Server] Chat Api for streaming and not streaming response (#5470 )

* fix bugs

* fix bugs

* fix api server

* fix api server

* add chat api and test

* del request.n

Jianghai

de378cd2ab

[Inference] Finish Online Serving Test, add streaming output api, continuous batching test and example (#5432 )

* finish online test and add examples

* fix test_contionus_batching

* fix some bugs

* fix bash

* fix

* fix inference

* finish revision

* fix typos

* revision

Jianghai

69cd7e069d

[Inference] ADD async and sync Api server using FastAPI (#5396 )

* add api server

* fix

* add

* add completion service and fix bug

* add generation config

* revise shardformer

* fix bugs

* add docstrings and fix some bugs

* fix bugs and add choices for prompt template

3 Commits (7bbb28e48bdb5849d9dfb118d7bf2959d79bbe02)