14 Commits (10e3c9f923caf4fb68ab61e96c244bd5cca9b9da)

Author	SHA1	Message	Date
yuehuayingxueluo	fab294c7f4	fix CI bugs	11 months ago
yuehuayingxueluo	2a73e828eb	fix bugs related to processing padding mask	11 months ago
Jianghai	e545a871b8	[Hotfix] Fix accuracy and align attention method api with Triton kernel (#5229) ... * fix accuracy * alignment in attention * fix attention * fix * fix bugs * fix bugs * fix bugs	11 months ago
yuehuayingxueluo	fa4fbdbffb	adapted to pad_context_forward	11 months ago
yuehuayingxueluo	47e53eaa1c	fix bugs in attention.py and request_handler.py	11 months ago
Jianghai	bfd9b1b494	[Inference] Pytorch Attention func, pad&nopad input support (#5219) ... * add attn * add attention test * fix attn forward * fix decoding	11 months ago
yuehuayingxueluo	3ad1f3b78b	fix beam_width	11 months ago
yuehuayingxueluo	b2eb9cd186	Fixed a typo	11 months ago
yuehuayingxueluo	02c1bf8b2a	add context_attention_unpadded	11 months ago
yuehuayingxueluo	9489dc64d8	precision alignment	11 months ago
yuehuayingxueluo	62968588d1	fix bugs in request_handler	11 months ago
yuehuayingxueluo	62fd08ee44	Fixed a bug in the inference frame	11 months ago
yuehuayingxueluo	86853a37d5	Add padding llama model	11 months ago
yuehuayingxueluo	8daee26989	[Inference] Add the logic of the inference engine (#5173) ... * add infer_struct and infer_config * update codes * change InferConfig * Add hf_model_config to the engine * rm _get_hf_model_config * update codes * made adjustments according to the feedback from the reviewer. * update codes * add ci test for config and struct * Add the logic of the inference engine * update engine and test * Recover cache_manager.py * add logger * fix conflict * update codes * update codes * update model and tokenizer * fix add the logic about shardformer * change kvcache_manager docstring * add policy * fix ci bug in test_kvcache_manager.py * remove codes related o tokenizer and move model_policy * fix code style * add ordered_set to requirements-infer.txt * Delete extra empty lines * add ordered_set to requirements-test.txt	11 months ago