You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

yuehuayingxueluo 8daee26989 [Inference] Add the logic of the inference engine (#5173) ... * add infer_struct and infer_config * update codes * change InferConfig * Add hf_model_config to the engine * rm _get_hf_model_config * update codes * made adjustments according to the feedback from the reviewer. * update codes * add ci test for config and struct * Add the logic of the inference engine * update engine and test * Recover cache_manager.py * add logger * fix conflict * update codes * update codes * update model and tokenizer * fix add the logic about shardformer * change kvcache_manager docstring * add policy * fix ci bug in test_kvcache_manager.py * remove codes related o tokenizer and move model_policy * fix code style * add ordered_set to requirements-infer.txt * Delete extra empty lines * add ordered_set to requirements-test.txt		11 months ago
..
requirements-infer.txt	[Inference] Add the logic of the inference engine (#5173)	11 months ago
requirements-test.txt	[Inference] Add the logic of the inference engine (#5173)	11 months ago
requirements.txt	[Inference] Dynamic Batching Inference, online and offline (#4953)	1 year ago