2 Commits (feat/speculative-decoding)

Author	SHA1	Message	Date
Hongxin Liu	1cd7efc520	[inference] refactor examples and fix schedule (#5077) ... * [setup] refactor infer setup * [hotfix] fix infenrece behavior on 1 1 gpu * [exmaple] refactor inference examples	2023-11-21 10:46:03 +08:00
Xu Kai	fb103cfd6e	[inference] update examples and engine (#5073) ... * update examples and engine * fix choices * update example	2023-11-20 19:44:52 +08:00