ColossalAI

Commit Graph

Author	SHA1	Message	Date
Steve Luo	f7aecc0c6b	feat rmsnorm cuda kernel and add unittest, benchmark script (#5417 )	2024-03-08 16:21:12 +08:00
yuehuayingxueluo	0aa27f1961	[Inference]Move benchmark-related code to the example directory. (#5408 ) * move benchmark-related code to the example directory. * fix bugs in test_fused_rotary_embedding.py	2024-02-28 16:46:03 +08:00
yuehuayingxueluo	600881a8ea	[Inference]Add CUDA KVCache Kernel (#5406 ) * add cuda KVCache kernel * annotation benchmark_kvcache_copy * add use cuda * fix import path * move benchmark scripts to example/ * rm benchmark codes in test_kv_cache_memcpy.py * rm redundancy codes * rm redundancy codes * pr was modified according to the review	2024-02-28 14:36:50 +08:00

Author

SHA1

Message

Date

Steve Luo

f7aecc0c6b

feat rmsnorm cuda kernel and add unittest, benchmark script (#5417 )

2024-03-08 16:21:12 +08:00

yuehuayingxueluo

0aa27f1961

[Inference]Move benchmark-related code to the example directory. (#5408 )

* move benchmark-related code to the example directory.

* fix bugs in test_fused_rotary_embedding.py

2024-02-28 16:46:03 +08:00

yuehuayingxueluo

600881a8ea

[Inference]Add CUDA KVCache Kernel (#5406 )

* add cuda KVCache kernel

* annotation benchmark_kvcache_copy

* add use cuda

* fix import path

* move benchmark scripts to example/

* rm benchmark codes in test_kv_cache_memcpy.py

* rm redundancy codes

* rm redundancy codes

* pr was modified according to the review

2024-02-28 14:36:50 +08:00

3 Commits (095c070a6eefe1a76fe3483b21986826114d6d17)