github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/inference/tensor_parallel
History
Yuanheng Zhao 17cfa57140
[infer] Add Bloom inference policy and replaced methods (#4512) 
* add bloom inference methods and policy

* enable pass BatchInferState from model forward

* revise bloom infer layers/policies

* add engine for inference (draft)

* add test for bloom infer

* fix bloom infer policy and flow

* revise bloom test

* fix bloom file path

* remove unused codes

* fix bloom modeling

* fix dir typo

* fix trivial

* fix policy

* clean pr

* trivial fix
 		2023-08-30 17:50:41 +08:00
..
modeling [infer] Add Bloom inference policy and replaced methods (#4512) 2023-08-30 17:50:41 +08:00
policies [infer] Add Bloom inference policy and replaced methods (#4512) 2023-08-30 17:50:41 +08:00
__init__.py [infer] Add Bloom inference policy and replaced methods (#4512) 2023-08-30 17:50:41 +08:00
batch_infer_state.py [Infer] Add TPInferEngine and fix file path (#4532) 2023-08-29 18:57:52 +08:00
engine.py [infer] Add Bloom inference policy and replaced methods (#4512) 2023-08-30 17:50:41 +08:00
kvcache_manager.py [Infer] Add TPInferEngine and fix file path (#4532) 2023-08-29 18:57:52 +08:00