2 Commits (492520dbdb962d207ac40d216e0414807f73eb19)

Author	SHA1	Message	Date
Jianghai	c064032865	[Online Server] Chat Api for streaming and not streaming response (#5470) ... * fix bugs * fix bugs * fix api server * fix api server * add chat api and test * del request.n	2024-05-08 15:20:53 +00:00
Jianghai	69cd7e069d	[Inference] ADD async and sync Api server using FastAPI (#5396) ... * add api server * fix * add * add completion service and fix bug * add generation config * revise shardformer * fix bugs * add docstrings and fix some bugs * fix bugs and add choices for prompt template	2024-05-08 15:18:28 +00:00