ColossalAI

Commit Graph

Author	SHA1	Message	Date
Yuanheng Zhao	283c407a19	[Inference] Fix Inference Generation Config and Sampling (#5710 ) * refactor and add * config default values * fix gen config passing * fix rpc generation config	2024-05-19 15:08:42 +08:00
Runyu Lu	74c47921fa	[Fix] Llama3 Load/Omit CheckpointIO Temporarily (#5717 ) * Fix Llama3 Load error * Omit Checkpoint IO Temporarily	2024-05-14 20:17:43 +08:00
Runyu Lu	18d67d0e8e	[Feat]Inference RPC Server Support (#5705 ) * rpc support source * kv cache logical/physical disaggregation * sampler refactor * colossalai launch built in * Unitest * Rpyc support --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2024-05-14 10:00:55 +08:00

Author

SHA1

Message

Date

Yuanheng Zhao

283c407a19

[Inference] Fix Inference Generation Config and Sampling (#5710 )

* refactor and add

* config default values

* fix gen config passing

* fix rpc generation config

2024-05-19 15:08:42 +08:00

Runyu Lu

74c47921fa

[Fix] Llama3 Load/Omit CheckpointIO Temporarily (#5717 )

* Fix Llama3 Load error
* Omit Checkpoint IO Temporarily

2024-05-14 20:17:43 +08:00

Runyu Lu

18d67d0e8e

[Feat]Inference RPC Server Support (#5705 )

* rpc support source
* kv cache logical/physical disaggregation
* sampler refactor
* colossalai launch built in
* Unitest
* Rpyc support

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

2024-05-14 10:00:55 +08:00

3 Commits (4adbc369136e4c6340e9a95d1d0650f263ddc86a)