github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/tests/test_infer
History
Yuanheng Zhao 17cfa57140
[infer] Add Bloom inference policy and replaced methods (#4512) 
* add bloom inference methods and policy

* enable pass BatchInferState from model forward

* revise bloom infer layers/policies

* add engine for inference (draft)

* add test for bloom infer

* fix bloom infer policy and flow

* revise bloom test

* fix bloom file path

* remove unused codes

* fix bloom modeling

* fix dir typo

* fix trivial

* fix policy

* clean pr

* trivial fix
 		2023-08-30 17:50:41 +08:00
..
_utils.py [infer] Infer/llama demo (#4503) 2023-08-24 15:42:41 +08:00
test_bloom_infer.py [infer] Add Bloom inference policy and replaced methods (#4512) 2023-08-30 17:50:41 +08:00
test_infer_engine.py [Infer] Add TPInferEngine and fix file path (#4532) 2023-08-29 18:57:52 +08:00
test_kvcache_manager.py [Infer] Add TPInferEngine and fix file path (#4532) 2023-08-29 18:57:52 +08:00
test_llama_infer.py Add Inference test for llama (#4508) 2023-08-30 12:10:26 +08:00