ColossalAI

Commit Graph

Author	SHA1	Message	Date
Yuanheng Zhao	573f270537	[Infer] Serving example w/ ray-serve (multiple GPU case) (#4841 ) * fix imports * add ray-serve with Colossal-Infer tp * trivial: send requests script * add README * fix worker port * fix readme * use app builder and autoscaling * trivial: input args * clean code; revise readme * testci (skip example test) * use auto model/tokenizer * revert imports fix (fixed in other PRs)	2023-10-02 17:48:38 +08:00
Yuanheng Zhao	3a74eb4b3a	[Infer] Colossal-Inference serving example w/ TorchServe (single GPU case) (#4771 ) * add Colossal-Inference serving example w/ TorchServe * add dockerfile * fix dockerfile * fix dockerfile: fix commit hash, install curl * refactor file structure * revise readme * trivial * trivial: dockerfile format * clean dir; revise readme * fix comments: fix imports and configs * fix formats * remove unused requirements	2023-10-02 17:42:37 +08:00

Author

SHA1

Message

Date

Yuanheng Zhao

573f270537

[Infer] Serving example w/ ray-serve (multiple GPU case) (#4841 )

* fix imports

* add ray-serve with Colossal-Infer tp

* trivial: send requests script

* add README

* fix worker port

* fix readme

* use app builder and autoscaling

* trivial: input args

* clean code; revise readme

* testci (skip example test)

* use auto model/tokenizer

* revert imports fix (fixed in other PRs)

2023-10-02 17:48:38 +08:00

Yuanheng Zhao

3a74eb4b3a

[Infer] Colossal-Inference serving example w/ TorchServe (single GPU case) (#4771 )

* add Colossal-Inference serving example w/ TorchServe

* add dockerfile

* fix dockerfile

* fix dockerfile: fix commit hash, install curl

* refactor file structure

* revise readme

* trivial

* trivial: dockerfile format

* clean dir; revise readme

* fix comments: fix imports and configs

* fix formats

* remove unused requirements

2023-10-02 17:42:37 +08:00

2 Commits (07ed155e8633e7aaaea83a0dee3da04119287a9a)