github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[chat] fix compute_approx_kl (#4338)

pull/4359/head
Wenhao Chen 2023-08-01 10:21:45 +08:00 committed by GitHub
parent 03654c0ce2
commit 75c5389037
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
applications/Chat/coati/models/utils.py
View File

@ -19,7 +19,7 @@ def compute_approx_kl(log_probs: torch.Tensor,
action_mask: Mask for actions.
"""
log_ratio = log_probs - log_probs_base
log_ratio = log_probs_base - log_probs
approx_kl = (log_ratio.exp() - 1) - log_ratio
if action_mask is not None:
approx_kl = masked_mean(approx_kl, action_mask, dim=1)