github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/examples/inference
History
Xu Kai c6295c3381
[Refactor] remove useless inference code (#5022) 
* remove useless code

* fix quant model

* fix test import bug

* mv original inference legacy

* fix chatglm2
 		2023-11-10 14:47:06 +08:00
..
serving [Infer] Serving example w/ ray-serve (multiple GPU case) (#4841) 2023-10-02 17:48:38 +08:00
_utils.py [Inference]ADD Bench Chatglm2 script (#4963) 2023-10-24 13:11:15 +08:00
bench_bloom.py [Inference]ADD Bench Chatglm2 script (#4963) 2023-10-24 13:11:15 +08:00
bench_chatglm2.py [Inference]ADD Bench Chatglm2 script (#4963) 2023-10-24 13:11:15 +08:00
bench_llama.py [Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965) 2023-10-30 14:04:37 +08:00
benchmark.py [Refactor] remove useless inference code (#5022) 2023-11-10 14:47:06 +08:00
gptq_bloom.py [Inference]ADD Bench Chatglm2 script (#4963) 2023-10-24 13:11:15 +08:00
gptq_llama.py [Inference]ADD Bench Chatglm2 script (#4963) 2023-10-24 13:11:15 +08:00
hybrid_gptq_llama.py [refactor] refactor gptq and smoothquant llama (#5012) 2023-11-09 10:12:11 +08:00
hybrid_smoothquant_llama.py [refactor] refactor gptq and smoothquant llama (#5012) 2023-11-09 10:12:11 +08:00
run.sh [Refactor] remove useless inference code (#5022) 2023-11-10 14:47:06 +08:00
smoothquant_llama.py [inference] Add smmoothquant for llama (#4904) 2023-10-16 11:28:44 +08:00