ColossalAI/colossalai/inference/modeling/policy
yuehuayingxueluo 3c91e3f176
[Inference]Adapt to baichuan2 13B (#5614)
* adapt to baichuan2 13B

* adapt to baichuan2 13B

* change BAICHUAN_MODEL_NAME_OR_PATH

* fix test_decoding_attn.py

* Modifications based on review comments.

* change BAICHUAN_MODEL_NAME_OR_PATH

* mv attn mask processes to test flash decoding

* mv get_alibi_slopes baichuan modeling

* fix bugs in test_baichuan.py
2024-04-25 23:11:30 +08:00
..
__init__.py [inference/model]Adapted to the baichuan2-7B model (#5591) 2024-04-15 16:53:02 +08:00
glide_llama.py [Inference/SpecDec] Support GLIDE Drafter Model (#5455) 2024-04-10 11:07:52 +08:00
nopadding_baichuan.py [Inference]Adapt to baichuan2 13B (#5614) 2024-04-25 23:11:30 +08:00
nopadding_llama.py [example] Update Llama Inference example (#5629) 2024-04-23 22:23:07 +08:00