github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[NFC] Update roberta/README.md (#2350)

pull/2353/head
Ikko Eltociear Ashimine 2023-01-06 11:09:14 +09:00 committed by GitHub
parent 35e22be2f6
commit 5e4bced0a3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
examples/language/roberta/README.md
View File

@ -33,7 +33,7 @@ service ssh restart
```bash
cd preprocessing
```
following the `README.md`, preprocess orginal corpus to h5py+numpy
following the `README.md`, preprocess original corpus to h5py+numpy
## 2. Pretrain
@ -44,7 +44,7 @@ following the `README.md`, load the h5py generated by preprocess of step 1 to pr
## 3. Finetune
The checkpoint produced by this repo can replace `pytorch_model.bin` from [hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/tree/main) directly. Then use transfomers from HuggingFace to finetune downstream application.
The checkpoint produced by this repo can replace `pytorch_model.bin` from [hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/tree/main) directly. Then use transfomers from Hugging Face to finetune downstream application.
## Contributors
The repo is contributed by AI team from [Moore Threads](https://www.mthreads.com/). If you find any problems for pretraining, please file an issue or send an email to yehua.zhang@mthreads.com. At last, welcome any form of contribution!
@ -55,4 +55,4 @@ The repo is contributed by AI team from [Moore Threads](https://www.mthreads.com
author={Yehua Zhang, Chen Zhang},
year={2022}
}
```
```