Runyu Lu
|
3c7cda0c9a
|
[Inference]Lazy Init Support (#5785)
* lazy init support
* lazy init llama support
* :lazy init support for baichuan
* aligh rpc
* add note for baichuan
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2024-06-27 18:02:15 +08:00 |
|
Yuanheng Zhao
|
283c407a19
|
[Inference] Fix Inference Generation Config and Sampling (#5710)
* refactor and add
* config default values
* fix gen config passing
* fix rpc generation config
|
2024-05-19 15:08:42 +08:00 |
|
Runyu Lu
|
18d67d0e8e
|
[Feat]Inference RPC Server Support (#5705)
* rpc support source
* kv cache logical/physical disaggregation
* sampler refactor
* colossalai launch built in
* Unitest
* Rpyc support
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
2024-05-14 10:00:55 +08:00 |