From 59157cea59c95e599648d4128e94ee6d883d2da0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=9B=B9=E5=B7=8D=E7=80=9A?= Date: Tue, 2 Jul 2024 12:10:49 +0000 Subject: [PATCH] [Docs] update readme about internlm 2.5 --- finetune/README.md | 22 +++++++--------------- finetune/README_zh-CN.md | 20 +++++++------------- 2 files changed, 14 insertions(+), 28 deletions(-) diff --git a/finetune/README.md b/finetune/README.md index 9ab1496..6f62b3d 100644 --- a/finetune/README.md +++ b/finetune/README.md @@ -29,7 +29,7 @@ We recommend two projects to fine-tune InternLM. - Install XTuner with DeepSpeed integration ```shell - pip install -U 'xtuner[deepspeed]>=0.1.13' + pip install -U 'xtuner[deepspeed]>=0.1.22' ``` ### Fine-tune @@ -55,14 +55,14 @@ XTuner supports the efficient fine-tune (*e.g.*, QLoRA) for InternLM2. xtuner train ${CONFIG_NAME_OR_PATH} ``` - For example, we can start the QLoRA fine-tuning of internlm2_5-7b-chat with oasst1 dataset by + For example, we can start the QLoRA fine-tuning of InternLM2.5-Chat-7B with oasst1 dataset by ```shell # On a single GPU - xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 + xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 # On multiple GPUs - (DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 - (SLURM) srun ${SRUN_ARGS} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2 + (DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 + (SLURM) srun ${SRUN_ARGS} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2 ``` - `--deepspeed` means using [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀 to optimize the training. XTuner comes with several integrated strategies including ZeRO-1, ZeRO-2, and ZeRO-3. If you wish to disable this feature, simply remove this argument. @@ -81,18 +81,10 @@ XTuner provides tools to chat with pretrained / fine-tuned large models. xtuner chat ${NAME_OR_PATH_TO_LLM} [optional arguments] ``` -For example, we can start the chat with - -internlm2_5-7b-chat with adapter trained from oasst1: +For example, we can start the chat with InternLM2.5-Chat-7B : ```shell -xtuner chat internlm/internlm2_5-7b-chat --adapter xtuner/internlm2_5-7b-chat-qlora-oasst1 --prompt-template internlm2_chat -``` - -LLaVA-InternLM2-7B: - -```shell -xtuner chat internlm/internlm2_5-7b-chat --visual-encoder openai/clip-vit-large-patch14-336 --llava xtuner/llava-internlm2-7b --prompt-template internlm2_chat --image $IMAGE_PATH +xtuner chat internlm/internlm2_5-chat-7b --prompt-template internlm2_chat ``` ## InternEvo diff --git a/finetune/README_zh-CN.md b/finetune/README_zh-CN.md index 0e45035..0781c66 100644 --- a/finetune/README_zh-CN.md +++ b/finetune/README_zh-CN.md @@ -29,7 +29,7 @@ - 安装集成 DeepSpeed 版本的 XTuner ```shell - pip install -U 'xtuner[deepspeed]>=0.1.13' + pip install -U 'xtuner[deepspeed]>=0.1.22' ``` ### 微调 @@ -53,14 +53,14 @@ xtuner train ${CONFIG_NAME_OR_PATH} ``` - 例如，我们可以利用 QLoRA 算法在 oasst1 数据集上微调 internlm2_5-7b-chat： + 例如，我们可以利用 QLoRA 算法在 oasst1 数据集上微调 InternLM2.5-Chat-7B： ```shell # 单卡 - xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 + xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 # 多卡 - (DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 - (SLURM) srun ${SRUN_ARGS} xtuner train internlm2_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2 + (DIST) NPROC_PER_NODE=${GPU_NUM} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --deepspeed deepspeed_zero2 + (SLURM) srun ${SRUN_ARGS} xtuner train internlm2_5_chat_7b_qlora_oasst1_e3 --launcher slurm --deepspeed deepspeed_zero2 ``` - `--deepspeed` 表示使用 [DeepSpeed](https://github.com/microsoft/DeepSpeed) 🚀 来优化训练过程。XTuner 内置了多种策略，包括 ZeRO-1、ZeRO-2、ZeRO-3 等。如果用户期望关闭此功能，请直接移除此参数。 @@ -81,16 +81,10 @@ xtuner chat ${NAME_OR_PATH_TO_LLM} [optional arguments] 例如： -与 internlm2_5-7b-chat, oasst1 adapter 对话： +与 InternLM2.5-Chat-7B 对话： ```shell -xtuner chat internlm/internlm2_5-7b-chat --adapter xtuner/internlm2_5-7b-chat-qlora-oasst1 --prompt-template internlm2_chat -``` - -与 LLaVA-InternLM2-7B 对话： - -```shell -xtuner chat internlm/internlm2_5-7b-chat --visual-encoder openai/clip-vit-large-patch14-336 --llava xtuner/llava-internlm2-7b --prompt-template internlm2_chat --image $IMAGE_PATH +xtuner chat internlm/internlm2_5-chat-7b --prompt-template internlm2_chat ``` ## InternEvo