3 Commits (f7aecc0c6bac001d10c1dd00274e0152e4c86df6)

Author	SHA1	Message	Date
yuehuayingxueluo	600881a8ea	[Inference]Add CUDA KVCache Kernel (#5406) ... * add cuda KVCache kernel * annotation benchmark_kvcache_copy * add use cuda * fix import path * move benchmark scripts to example/ * rm benchmark codes in test_kv_cache_memcpy.py * rm redundancy codes * rm redundancy codes * pr was modified according to the review	2024-02-28 14:36:50 +08:00
yuehuayingxueluo	6fb4bcbb24	[Inference/opt] Fused KVCahce Memcopy (#5374) ... * fused kv memcopy * add TODO in test_kvcache_copy.py	2024-02-07 17:15:42 +08:00
Frank Lee	e76acbb076	[inference] moved ops tests to test_infer (#5354)	2024-02-02 13:51:22 +08:00