github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

fix bug for mefture (#5299)

pull/5265/merge
Desperado-Jia 2024-01-22 22:17:54 +08:00 committed by GitHub
parent f7e3f82a7e
commit ddf879e2db
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
applications/Colossal-LLaMA-2/colossal_llama2/utils/neftune_patch.py
View File

@ -16,7 +16,10 @@ import torch
def unwrap(model): def unwrap(model):
return model.unwrap().module if hasattr(model, "module"):
return unwrap_model(model.module)
else:
return model
def neftune_post_forward_hook(module, input, output): def neftune_post_forward_hook(module, input, output):