github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[hotfix]fixed bugs of assigning grad states to non leaf nodes (#711) 

* fixed bugs of assigning grad states to non leaf nodes

* use detach()
pull/712/head
LuGY 2022-04-11 14:04:58 +08:00 committed by GitHub
parent eda30a058e
commit 140263a394
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/utils/activation_checkpoint.py
View File

@ -12,7 +12,7 @@ def copy_to_device(obj, device):
if torch.is_tensor(obj): if torch.is_tensor(obj):
# Notice: # Notice:
# When in no_grad context, requires_gard is False after movement # When in no_grad context, requires_gard is False after movement
ret = obj.to(device) ret = obj.to(device).detach()
ret.requires_grad = obj.requires_grad ret.requires_grad = obj.requires_grad
return ret return ret
elif isinstance(obj, list): elif isinstance(obj, list):