github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/tests/test_infer/test_dynamic_batching
History
Zhongkai Zhao 70885d707d
[hotfix] Suport extra_kwargs in ShardConfig (#5031) 
* [refactor]: replace inference args with extra_kwargs in ShardConfig

* modify shardconfig

* polish code

* fix policy bug in llama

* fix bug in auto policy

* remove setattr in ShardConfig
 		2023-11-10 10:49:50 +08:00
..
config.yaml [Inference] Dynamic Batching Inference, online and offline (#4953) 2023-10-30 10:52:19 +08:00
test_async_engine.py [Inference] Dynamic Batching Inference, online and offline (#4953) 2023-10-30 10:52:19 +08:00
test_dynamic_batching_manager.py [hotfix] Suport extra_kwargs in ShardConfig (#5031) 2023-11-10 10:49:50 +08:00
test_offline_dynamic_batching.py [hotfix] Suport extra_kwargs in ShardConfig (#5031) 2023-11-10 10:49:50 +08:00
test_ray_dist.py [Inference] Dynamic Batching Inference, online and offline (#4953) 2023-10-30 10:52:19 +08:00