9 Commits (70e5820050b7012f81dfed924bf563a1b1fd04fa)

Author	SHA1	Message	Date
alexhegit	e8c83bdf56	Do the device_map more automaticlly for multi-GPUs ... The accelerate lib provides some functions to do the device_map automaticlly. This patch use infer_auto_device_map() to relpace the hard codes which splite the model by hand. It works well with my two GPUs(6GB-RTX3060 + 12GB-RTX3060). Signed-off-by: Alex He <heye_dev@163.com>	2023-08-18 07:37:59 +08:00
duzx16	01e6313abf	Fix multi-gpu loading	2023-04-17 17:10:09 +08:00
duzx16	4f95c09a3b	Add multi-gpu deployment	2023-04-16 21:37:33 +08:00
duzx16	ffcb1f9394	Remove save_pretrained	2023-04-16 21:01:39 +08:00
saber	c313af0639	Add support for manual device_map; auto-config if None	2023-03-27 22:20:23 +08:00
saber	c31d9c0984	Move import statement into function to avoid dependency	2023-03-27 22:11:57 +08:00
saber	6a5267aef7	bugfix: linux多卡部署时weight,input不在同一device上,导致RuntimeError	2023-03-27 20:51:05 +08:00
saber	8101d75ab8	fix typo	2023-03-26 15:29:15 +08:00
saber	8826b947c3	多GPU支持, 模型文件夹没有index.json会自动保存模型到multi_gpu_model_cache_dir以支持多GPU	2023-03-26 15:13:44 +08:00