github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

add tokenize func

pull/6119/head
Tong Li 2024-11-14 08:31:33 +00:00
parent b6ec337f3d
commit c606d1101c
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
applications/ColossalChat/coati/dataset/__init__.py
View File

@ -23,5 +23,5 @@ __all__ = [
"tokenize_kto", "tokenize_kto",
"setup_conversation_template", "setup_conversation_template",
"Conversation", "Conversation",
"tokenize_process_reward" "tokenize_process_reward",
] ]