5 Commits (e37ee2fb65fc77c275b816968d91776322fd7695)

Author	SHA1	Message	Date
Runyu Lu	e37ee2fb65	[Feat]Tensor Model Parallel Support For Inference (#5563) ... * tensor parallel support naive source * [fix]precision, model load and refactor the framework * add tp unit test * docstring * fix do_sample	2024-04-18 16:56:46 +08:00
Runyu Lu	9fe61b4475	[fix]	2024-03-25 11:37:58 +08:00
Runyu Lu	aabc9fb6aa	[feat] add use_cuda_kernel option	2024-03-19 13:24:25 +08:00
Runyu Lu	ae24b4f025	diverse tests	2024-03-14 10:35:08 +08:00
Runyu Lu	1821a6dab0	[fix] pytest and fix dyn grid bug	2024-03-13 17:28:32 +08:00