github
/
ChatGLM2-6B
mirror of https://github.com/THUDM/ChatGLM2-6B
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Update web_demo.py 

更好的适配笔记本用户
pull/477/head
raxcl 2023-08-20 11:30:23 +08:00 committed by GitHub
parent 80602dcae1
commit 263e5022a5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
web_demo.py
View File

@ -8,6 +8,11 @@ model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).c
# 多显卡支持，使用下面两行代替上面一行，将num_gpus改为你实际的显卡数量 # 多显卡支持，使用下面两行代替上面一行，将num_gpus改为你实际的显卡数量
# from utils import load_model_on_gpus # from utils import load_model_on_gpus
# model = load_model_on_gpus("THUDM/chatglm2-6b", num_gpus=2) # model = load_model_on_gpus("THUDM/chatglm2-6b", num_gpus=2)
# 量化int4模型示例 替换第八行即可 quantize 对应gpt内存大小（适用于gpu不够的笔记本用户，如gpu为4G，采用前者，gpu为8G，采用后者）
# model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).half().quantize(4).cuda()
# model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).half().quantize(8).cuda()
model = model.eval() model = model.eval()
"""Override Chatbot.postprocess""" """Override Chatbot.postprocess"""