github
/
InternLM
mirror of https://github.com/InternLM/InternLM
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

fix(optimizer):broadcast (#453) 

* fix broadcast synchronize()

* fix synchronize
pull/454/head
ytxiong 2023-10-26 17:54:54 +08:00 committed by GitHub
parent aeee9fd2a9
commit ad70e323eb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
internlm/solver/optimizer/hybrid_zero_optim.py
View File

@ -809,7 +809,7 @@ class HybridZeroOptimizer(BaseOptimizer):
for handle in handles: for handle in handles:
handle.wait() handle.wait()
torch.cuda().synchronize() torch.cuda.synchronize()
################## ##################
# FP16 Utilities # # FP16 Utilities #