github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/inference/core
History
Runyu Lu e37ee2fb65
[Feat]Tensor Model Parallel Support For Inference (#5563) 
* tensor parallel support naive source

* [fix]precision, model load and refactor the framework

* add tp unit test

* docstring

* fix do_sample
 		2024-04-18 16:56:46 +08:00
..
__init__.py [doc] updated inference readme (#5343) 2024-02-02 14:31:10 +08:00
engine.py [Feat]Tensor Model Parallel Support For Inference (#5563) 2024-04-18 16:56:46 +08:00
plugin.py [Feat]Tensor Model Parallel Support For Inference (#5563) 2024-04-18 16:56:46 +08:00
request_handler.py [Feat]Tensor Model Parallel Support For Inference (#5563) 2024-04-18 16:56:46 +08:00