Hongxin Liu 1cd7efc520 [inference] refactor examples and fix schedule (#5077) ... * [setup] refactor infer setup * [hotfix] fix infenrece behavior on 1 1 gpu * [exmaple] refactor inference examples		2023-11-21 10:46:03 +08:00
..
modeling	[Kernels]added flash-decoidng of triton (#5063)	2023-11-20 13:58:29 +08:00
policies	[inference] Refactor inference architecture (#5057)	2023-11-19 21:05:05 +08:00
__init__.py	[inference] update examples and engine (#5073)	2023-11-20 19:44:52 +08:00
engine.py	[inference] refactor examples and fix schedule (#5077)	2023-11-21 10:46:03 +08:00
microbatch_manager.py	[inference] Refactor inference architecture (#5057)	2023-11-19 21:05:05 +08:00