ColossalAI/colossalai/inference/modeling/policy
yuehuayingxueluo bfff9254ac
[inference] Adapted to Rotary Embedding and RMS Norm (#5283)
* adapted to rotary_embedding

* adapted to nopad rms norm

* fix bugs in benchmark

* fix flash_decoding.py
2024-01-22 10:55:34 +08:00
..
__init__.py [Inference] Add the logic of the inference engine (#5173) 2024-01-11 13:39:56 +00:00
llama.py [inference] Adapted to Rotary Embedding and RMS Norm (#5283) 2024-01-22 10:55:34 +08:00