Merge branch 'whcao/fix_internlm2_5_readme' into 'main'

[Docs] update readme about internlm 2.5

See merge request openmmlab/bigmodel/InternLM!14
pull/752/head
lvchengqi 2024-07-02 12:10:49 +00:00
commit 10fbaa8fe8
2 changed files with 14 additions and 28 deletions

View File

@ -29,7 +29,7 @@ We recommend two projects to fine-tune InternLM.
- Install XTuner with DeepSpeed integration
```shell
pip install -U 'xtuner[deepspeed]>=0.1.13'
pip install -U 'xtuner[deepspeed]>=0.1.22'
```
### Fine-tune
@ -55,14 +55,14 @@ XTuner supports the efficient fine-tune (*e.g.*, QLoRA) for InternLM2.
xtuner train ${CONFIG_NAME_OR_PATH}
```
For example, we can start the QLoRA fine-tuning of internlm2_5-7b-chat with oasst1 dataset by
For example, we can start the QLoRA fine-tuning of InternLM2.5-Chat-7B with oasst1 dataset by
```shell
# On a single GPU
xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
# On multiple GPUs
(DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
(SLURM) srun ${SRUN_ARGS} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2
(DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
(SLURM) srun ${SRUN_ARGS} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2
```
- `--deepspeed` means using [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀 to optimize the training. XTuner comes with several integrated strategies including ZeRO-1, ZeRO-2, and ZeRO-3. If you wish to disable this feature, simply remove this argument.
@ -81,18 +81,10 @@ XTuner provides tools to chat with pretrained / fine-tuned large models.
xtuner chat ${NAME_OR_PATH_TO_LLM} [optional arguments]
```
For example, we can start the chat with
internlm2_5-7b-chat with adapter trained from oasst1:
For example, we can start the chat with InternLM2.5-Chat-7B :
```shell
xtuner chat internlm/internlm2_5-7b-chat --adapter xtuner/internlm2_5-7b-chat-qlora-oasst1 --prompt-template internlm2_chat
```
LLaVA-InternLM2-7B:
```shell
xtuner chat internlm/internlm2_5-7b-chat --visual-encoder openai/clip-vit-large-patch14-336 --llava xtuner/llava-internlm2-7b --prompt-template internlm2_chat --image $IMAGE_PATH
xtuner chat internlm/internlm2_5-chat-7b --prompt-template internlm2_chat
```
## InternEvo

View File

@ -29,7 +29,7 @@
- 安装集成 DeepSpeed 版本的 XTuner
```shell
pip install -U 'xtuner[deepspeed]>=0.1.13'
pip install -U 'xtuner[deepspeed]>=0.1.22'
```
### 微调
@ -53,14 +53,14 @@
xtuner train ${CONFIG_NAME_OR_PATH}
```
例如,我们可以利用 QLoRA 算法在 oasst1 数据集上微调 internlm2_5-7b-chat
例如,我们可以利用 QLoRA 算法在 oasst1 数据集上微调 InternLM2.5-Chat-7B
```shell
# 单卡
xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
# 多卡
(DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
(SLURM) srun ${SRUN_ARGS} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2
(DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2
(SLURM) srun ${SRUN_ARGS} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2
```
- `--deepspeed` 表示使用 [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀 来优化训练过程。XTuner 内置了多种策略,包括 ZeRO-1、ZeRO-2、ZeRO-3 等。如果用户期望关闭此功能,请直接移除此参数。
@ -81,16 +81,10 @@ xtuner chat ${NAME_OR_PATH_TO_LLM} [optional arguments]
例如:
internlm2_5-7b-chat, oasst1 adapter 对话:
InternLM2.5-Chat-7B 对话:
```shell
xtuner chat internlm/internlm2_5-7b-chat --adapter xtuner/internlm2_5-7b-chat-qlora-oasst1 --prompt-template internlm2_chat
```
与 LLaVA-InternLM2-7B 对话:
```shell
xtuner chat internlm/internlm2_5-7b-chat --visual-encoder openai/clip-vit-large-patch14-336 --llava xtuner/llava-internlm2-7b --prompt-template internlm2_chat --image $IMAGE_PATH
xtuner chat internlm/internlm2_5-chat-7b --prompt-template internlm2_chat
```
## InternEvo