github
/
InternLM
mirror of https://github.com/InternLM/InternLM
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/develop' into feat/fstp_refactor 

merge develop
pull/456/head
yingtongxiong 2023-10-26 17:41:42 +08:00
parent d831ddcc1d aeee9fd2a9
commit 1aae39b667
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
internlm/solver/optimizer/hybrid_zero_optim.py
View File

@ -856,6 +856,8 @@ class HybridZeroOptimizer(BaseOptimizer):
for handle in handles: for handle in handles:
handle.wait() handle.wait()
torch.cuda().synchronize()
################## ##################
# FP16 Utilities # # FP16 Utilities #
################## ##################