github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/examples/language/gpt/tools/Megatron
History
Jiarui Fang 203ca57aed
[example] add GPT 		2022-11-08 10:58:17 +08:00
..
__init__.py [example] add GPT 2022-11-08 10:58:17 +08:00
blacklist_urls.py [example] add GPT 2022-11-08 10:58:17 +08:00
cleanup_dataset.py [example] add GPT 2022-11-08 10:58:17 +08:00
cleanup_fix_dataset.py [example] add GPT 2022-11-08 10:58:17 +08:00
find_duplicates.py [example] add GPT 2022-11-08 10:58:17 +08:00
gpt2_tokenization.py [example] add GPT 2022-11-08 10:58:17 +08:00
group_duplicate_url.py [example] add GPT 2022-11-08 10:58:17 +08:00
remove_group_duplicates.py [example] add GPT 2022-11-08 10:58:17 +08:00
tokenizer.py [example] add GPT 2022-11-08 10:58:17 +08:00