Xu Kai 611a5a80ca [inference] Add smmoothquant for llama (#4904) ... * [inference] add int8 rotary embedding kernel for smoothquant (#4843) * [inference] add smoothquant llama attention (#4850) * add smoothquant llama attention * remove uselss code * remove useless code * fix import error * rename file name * [inference] add silu linear fusion for smoothquant llama mlp (#4853) * add silu linear * update skip condition * catch smoothquant cuda lib exception * prcocess exception for tests * [inference] add llama mlp for smoothquant (#4854) * add llama mlp for smoothquant * fix down out scale * remove duplicate lines * add llama mlp check * delete useless code * [inference] add smoothquant llama (#4861) * add smoothquant llama * fix attention accuracy * fix accuracy * add kv cache and save pretrained * refactor example * delete smooth * refactor code * [inference] add smooth function and delete useless code for smoothquant (#4895) * add smooth function and delete useless code * update datasets * remove duplicate import * delete useless file * refactor codes (#4902) * rafactor code * add license * add torch-int and smoothquant license		2023-10-16 11:28:44 +08:00
..
serving	[Infer] Serving example w/ ray-serve (multiple GPU case) (#4841)	2023-10-02 17:48:38 +08:00
bench_bloom.py	[misc] update pre-commit and run all files (#4752)	2023-09-19 14:20:26 +08:00
bench_llama.py	[infer] fix test bug (#4838)	2023-10-04 10:01:03 +08:00
gptq_bloom.py	[feature] add gptq for inference (#4754)	2023-09-22 11:02:50 +08:00
gptq_llama.py	[inference]fix import bug and delete down useless init (#4830)	2023-10-04 09:18:45 +08:00
smoothquant_llama.py	[inference] Add smmoothquant for llama (#4904)	2023-10-16 11:28:44 +08:00