6 Commits (ff14144d9cebc2d334de714f52d1b599ece0a116)

Author	SHA1	Message	Date
Yuanheng Zhao	283c407a19	[Inference] Fix Inference Generation Config and Sampling (#5710) ... * refactor and add * config default values * fix gen config passing * fix rpc generation config	6 months ago
Runyu Lu	18d67d0e8e	[Feat]Inference RPC Server Support (#5705) ... * rpc support source * kv cache logical/physical disaggregation * sampler refactor * colossalai launch built in * Unitest * Rpyc support --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	7 months ago
yuehuayingxueluo	de4bf3dedf	[Inference]Adapt repetition_penalty and no_repeat_ngram_size (#5708) ... * Adapt repetition_penalty and no_repeat_ngram_size * fix no_repeat_ngram_size_logit_process * remove batch_updated * fix annotation * modified codes based on the review feedback. * rm get_batch_token_ids	7 months ago
yuehuayingxueluo	9c2fe7935f	[Inference]Adapt temperature processing logic (#5689) ... * Adapt temperature processing logic * add ValueError for top_p and top_k * add GQA Test * fix except_msg	7 months ago
傅剑寒	e6496dd371	[Inference] Optimize request handler of llama (#5512) ... * optimize request_handler * fix ways of writing	8 months ago
Jianghai	0e616462a7	[Inference] add logit processor and request handler (#5166) ... * add logit processor and request handler * add * add * add * fix * add search tokens and update func * finish request handler * add running list test * fix test * fix some bug * add * add * fix bugs * fix some bugs * fix bug * fix * fix * add copy fun * del useless attn * fix request status --------- Co-authored-by: CjhHa1 <cjh18671720497outlook.com>	11 months ago