mirror of https://github.com/InternLM/InternLM
Fix readme about conversion to transformers (#25)
* add links for 8k * fix acknowledgement * modified readme for convert_hfpull/30/head
parent
ed04c7edb0
commit
745d2b911a
|
@ -3,8 +3,8 @@
|
||||||
├── transformers # 适配hugging face的transformers的一些工具
|
├── transformers # 适配hugging face的transformers的一些工具
|
||||||
│ ├── configuration_internlm.py # config适配工具
|
│ ├── configuration_internlm.py # config适配工具
|
||||||
│ ├── modeling_internlm.py # model适配工具
|
│ ├── modeling_internlm.py # model适配工具
|
||||||
│ └── tokenization_internlm.py # tokenizer适配工具
|
│ ├── tokenization_internlm.py # tokenizer适配工具
|
||||||
├── convert2hf.py # 模型适配hugging face工具
|
│ └── convert2hf.py # 模型适配hugging face工具
|
||||||
└── tokenizer.py # 将原始数据转换成bin和meta文件的工具
|
└── tokenizer.py # 将原始数据转换成bin和meta文件的工具
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
@ -4,7 +4,7 @@ This directory provide some tools for model training with the following file str
|
||||||
│ ├── configuration_internlm.py # tools for adapting config
|
│ ├── configuration_internlm.py # tools for adapting config
|
||||||
│ ├── modeling_internlm.py # tools for adapting model
|
│ ├── modeling_internlm.py # tools for adapting model
|
||||||
│ └── tokenization_internlm.py # tools for adapting tokenizer
|
│ └── tokenization_internlm.py # tools for adapting tokenizer
|
||||||
├── convert2hf.py # tools for adapting models to Hugging Face's format
|
│ └── convert2hf.py # tools for adapting models to Hugging Face's format
|
||||||
└── tokenizer.py # tools for generating `bin` and `meta` file for raw data
|
└── tokenizer.py # tools for generating `bin` and `meta` file for raw data
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,26 @@
|
||||||
|
# InternLM Transformers
|
||||||
|
|
||||||
|
[English](./README.md) |
|
||||||
|
[简体中文](./README-zh-Hans.md)
|
||||||
|
|
||||||
|
该文件夹下包含了 transformers 格式的 `InternLM` 模型。
|
||||||
|
|
||||||
|
|
||||||
|
## 权重转换
|
||||||
|
|
||||||
|
`convert2hf.py` 可以将训练保存的权重一键转换为 transformers 格式。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python convert2hf.py --src_folder origin_ckpt/ --tgt_folder hf_ckpt/ --tokenizer ../v7_sft.model
|
||||||
|
```
|
||||||
|
|
||||||
|
然后可以使用 `from_pretrained` 接口加载:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from modeling_internlm import InternLMForCausalLM
|
||||||
|
|
||||||
|
model = InternForCausalLM.from_pretrained("hf_ckpt/")
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
`intern_moss_example.py` 展示了如何使用 LoRA 来在 `fnlp/moss-moon-002-sft` 数据集上进行微调的样例。
|
|
@ -1,16 +1,19 @@
|
||||||
# InternLM Transformers
|
# InternLM Transformers
|
||||||
|
|
||||||
该文件夹下包含了 transformers 格式的 `InternLM` 模型。
|
[English](./README.md) |
|
||||||
|
[简体中文](./README-zh-Hans.md)
|
||||||
|
|
||||||
## 权重转换
|
This folder contains the `InternLM` model in transformers format.
|
||||||
|
|
||||||
`../tools/convert2hf.py` 可以将训练保存的权重一键转换为 transformers 格式。
|
## Weight Conversion
|
||||||
|
|
||||||
|
`convert2hf.py` can convert saved training weights into the transformers format with a single command.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python convert2hf.py --src_folder origin_ckpt/ --tgt_folder hf_ckpt/ --tokenizer tokenizes/tokenizer.model
|
python convert2hf.py --src_folder origin_ckpt/ --tgt_folder hf_ckpt/ --tokenizer ../v7_sft.model
|
||||||
```
|
```
|
||||||
|
|
||||||
然后可以使用 `from_pretrained` 接口加载:
|
Then, you can load it using the `from_pretrained` interface:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from modeling_internlm import InternLMForCausalLM
|
from modeling_internlm import InternLMForCausalLM
|
||||||
|
@ -18,5 +21,4 @@ from modeling_internlm import InternLMForCausalLM
|
||||||
model = InternForCausalLM.from_pretrained("hf_ckpt/")
|
model = InternForCausalLM.from_pretrained("hf_ckpt/")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`intern_moss_example.py` demonstrates an example of how to use LoRA for fine-tuning on the `fnlp/moss-moon-002-sft` dataset.
|
||||||
`moss_example.py` 展示了如何使用 LoRA 来在 `fnlp/moss-moon-002-sft` 数据集上进行微调的样例。
|
|
Loading…
Reference in New Issue