Yuanheng Zhao
|
d85d91435a
|
[Inference/SpecDec] Support GLIDE Drafter Model (#5455)
* add glide-llama policy and modeling
* update glide modeling, compitable with transformers 4.36.2
* revise glide llama modeling/usage
* fix issues of glimpsing large kv
* revise the way re-loading params for glide drafter
* fix drafter and engine tests
* enable convert to glide strict=False
* revise glide llama modeling
* revise vicuna prompt template
* revise drafter and tests
* apply usage of glide model in engine
|
2024-04-10 11:07:52 +08:00 |