updata README_npu

2025-01-15 17:27:01 +08:00 · 2025-01-15 17:27:01 +08:00 · d679966d4d
parent 42f9750b6d
commit d679966d4d
5 changed files with 22 additions and 59 deletions
--- a/README_npu.md
+++ b/README_npu.md
@ -37,15 +37,16 @@
 This is a guide to using Ascend NPU to train and infer the InternLM series models.

 ## News
-\[2025.01.15\] InternLM3-8B-Instruct can be used in Xtuner, LLaMa-Factory and transformers.
+\[2025.01.15\] InternLM3-8B-Instruct can be used in Xtuner, LLaMA-Factory and transformers.

 ## Model Zoo

 ### InternLM3

-| Model                     | Transformers(HF)                                         | ModelScope(HF)                                         | Modelers(HF)                                          | Release Date |
-| ------------------------- | -------------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------- | ------------ |
-| **InternLM3-8B-Instruct** | [🤗internlm3_8B_instruct](https://huggingface.co/internlm/internlm3-8b-instruct) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm3_8b_instruct](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct/summary) | [![Open in Modelers](<>)](https://modelers.cn/models/Intern/internlm3-8b-instruct) | 2025-01-15   |
+| Model                     | Transformers                                         | ModelScope                                         | Modelers                                          | Release Date |
+| ------------------------- | ---------------------------------------------------- | -------------------------------------------------- | ------------------------------------------------- | ------------ |
+| **InternLM3-8B-Instruct** | [🤗internlm3_8B_instruct](https://huggingface.co/internlm/internlm3-8b-instruct) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm3_8b_instruct](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct/summary) | [![Open in Modelers](https://modelers.cn/assets/logo1-1bf58310.svg)](https://modelers.cn/models/Intern/internlm3-8b-instruct) | 2025-01-15   |
+
 ## Environment Setup

 ### Installing Ascend CANN Toolkit and Kernels
@ -79,7 +80,6 @@ Modify `requirements/runtime.txt` with the following changes:

 ```text
 bitsandbytes==0.42.0
-mmengine==0.10.5
 torchvision==0.19.0
 numpy==1.26.4
 ```
@ -127,24 +127,6 @@ model = dict(
        #     bnb_4bit_compute_dtype=torch.float16,
        #     bnb_4bit_use_double_quant=True,
        #     bnb_4bit_quant_type='nf4')),
-    lora=dict(
-        type=LoraConfig,
-        r=64,
-        lora_alpha=16,
-        lora_dropout=0.1,
-        bias='none',
-        task_type='CAUSAL_LM'))
-
-custom_hooks = [
-    dict(type=DatasetInfoHook, tokenizer=tokenizer),
-    # dict(
-    #     type=EvaluateChatHook,
-    #     tokenizer=tokenizer,
-    #     every_n_iters=evaluation_freq,
-    #     evaluation_inputs=evaluation_inputs,
-    #     system=SYSTEM,
-    #     prompt_template=prompt_template)
-]

 randomness = dict(seed=123, deterministic=True)
 ```
@ -158,7 +140,7 @@ NPROC_PER_NODE=8 xtuner train internlm3_8b_instruct_lora_oasst1_e10.py --deepspe
 The fine-tuning results are saved in the directory `./work_dirs/internlm3_8b_instruct_lora_oasst1_e10/iter_xxx.pth`.
 The comparison of loss between NPU and GPU is as follows:

-![xtuner_training_loss](assets/xtuner_loss.png)
+![xtuner_training_loss](assets/xtuner_training_loss_compare.png)

 ### Model Convert

@ -189,9 +171,9 @@ cp path_to_your_model/modeling_internlm3.py ./work_dirs/merge_output
 xtuner chat ./work_dirs/merge_output --prompt-template internlm2_chat
 ```

-## LLama-Factory
+## LLaMA-Factory

-### Installing LLaMa-Factory
+### Installing LLaMA-Factory

 ```shell
 git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
@ -201,7 +183,7 @@ pip install -e ".[torch-npu,metrics]"

 ### Inference

-Create the `examples/inference/internlm3_8b_instruct.yaml` inference configuration file in the LLaMa-Factory directory:
+Create the `examples/inference/internlm3_8b_instruct.yaml` inference configuration file in the LLaMA-Factory directory:

 ```yaml
 model_name_or_path: xxx # Support only local loading. Set this parameter to the local weight path of InternLM3-8B-Instruct.
@ -217,7 +199,7 @@ llamafactory-cli chat examples/inference/internlm3_8b_instruct.yaml

 ### Fine-tuning

-Create the `examples/train_full/internlm3_8b_instruct_full_sft.yaml` configuration file in the LLaMa-Factory directory. The fine-tuning configuration file is as follows:
+Create the `examples/train_full/internlm3_8b_instruct_full_sft.yaml` configuration file in the LLaMA-Factory directory. The fine-tuning configuration file is as follows:

 ```yaml
 ### model
@ -276,7 +258,7 @@ The loss curve obtained after finetuning is as follows:

 The loss curve compared with GPU is as follows:

-![training_loss_compare](assets/lf_traing_loss_compare.png)
+![training_loss_compare](assets/lf_training_loss_compare.png)

 ## Transformers

--- a/README_npu_zh-CN.md
+++ b/README_npu_zh-CN.md
@ -37,15 +37,15 @@
 这是一份使用 Ascend NPU 对 InternLM 系列模型进行训练和推理的指南。

 ## News
-\[2025.01.15\] InternLM3-8B-Instruct 可用于 Xtuner、LLaMa-Factory 和 transformers 中。
+\[2025.01.15\] InternLM3-8B-Instruct 可用于 Xtuner、LLaMA-Factory 和 transformers 中。

 ## Model Zoo

 ### InternLM3

-| Model                     | Transformers(HF)                                         | ModelScope(HF)                                         | Modelers(HF)                                          | Release Date |
-| ------------------------- | -------------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------- | ------------ |
-| **InternLM3-8B-Instruct** | [🤗internlm3_8B_instruct](https://huggingface.co/internlm/internlm3-8b-instruct) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm3_8b_instruct](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct/summary) | [![Open in Modelers](<>)](https://modelers.cn/models/Intern/internlm3-8b-instruct) | 2025-01-15   |
+| Model                     | Transformers                                         | ModelScope                                         | Modelers                                          | Release Date |
+| ------------------------- | ---------------------------------------------------- | -------------------------------------------------- | ------------------------------------------------- | ------------ |
+| **InternLM3-8B-Instruct** | [🤗internlm3_8B_instruct](https://huggingface.co/internlm/internlm3-8b-instruct) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm3_8b_instruct](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm3-8b-instruct/summary) | [![Open in Modelers](https://modelers.cn/assets/logo1-1bf58310.svg)](https://modelers.cn/models/Intern/internlm3-8b-instruct) | 2025-01-15   |

 ## 环境准备

@ -80,7 +80,6 @@ cd xtuner

 ```text
 bitsandbytes==0.42.0
-mmengine==0.10.5
 torchvision==0.19.0
 numpy==1.26.4
 ```
@ -128,24 +127,6 @@ model = dict(
        #     bnb_4bit_compute_dtype=torch.float16,
        #     bnb_4bit_use_double_quant=True,
        #     bnb_4bit_quant_type='nf4')),
-    lora=dict(
-        type=LoraConfig,
-        r=64,
-        lora_alpha=16,
-        lora_dropout=0.1,
-        bias='none',
-        task_type='CAUSAL_LM'))
-
-custom_hooks = [
-    dict(type=DatasetInfoHook, tokenizer=tokenizer),
-    # dict(
-    #     type=EvaluateChatHook,
-    #     tokenizer=tokenizer,
-    #     every_n_iters=evaluation_freq,
-    #     evaluation_inputs=evaluation_inputs,
-    #     system=SYSTEM,
-    #     prompt_template=prompt_template)
-]

 randomness = dict(seed=123, deterministic=True)
 ```
@ -156,9 +137,9 @@ randomness = dict(seed=123, deterministic=True)
 NPROC_PER_NODE=8 xtuner train internlm3_8b_instruct_lora_oasst1_e10.py --deepspeed deepspeed_zero2
 ```

-微调后结果保存在`./work_dirs/internlm3_8b_instruct_lora_oasst1_e10/iter_xxx.pth`,NPU与GPU的loss对比如下：
+微调后结果保存在`./work_dirs/internlm3_8b_instruct_lora_oasst1_e10/iter_xxx.pth`，NPU与GPU的loss对比如下：

-![xtuner_training_loss](assets/xtuner_loss.png)
+![xtuner_training_loss](assets/xtuner_training_loss_compare.png)

 ### 模型转换

@ -186,9 +167,9 @@ cp path_to_your_model/modeling_internlm3.py ./work_dirs/merge_output
 xtuner chat ./work_dirs/merge_output --prompt-template internlm2_chat
 ```

-## LLama-Factory
+## LLaMA-Factory

-### 安装 LLaMa-Factory
+### 安装 LLaMA-Factory

 ```shell
 git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
@ -198,7 +179,7 @@ pip install -e ".[torch-npu,metrics]"

 ### 推理

-在 LLaMa-Factory 路径下新建`examples/inference/internlm3_8b_instruct.yaml`推理配置文件，文件内容为：
+在 LLaMA-Factory 路径下新建`examples/inference/internlm3_8b_instruct.yaml`推理配置文件，文件内容为：

 ```yaml
 model_name_or_path: xxx # Support only local loading. Set this parameter to the local weight path of InternLM3-8B-Instruct.
@ -214,7 +195,7 @@ llamafactory-cli chat examples/inference/internlm3_8b_instruct.yaml

 ### 微调

-在 LLaMa-Factory 路径下新建`examples/train_full/internlm3_8b_instruct_full_sft.yaml`微调配置文件，微调配置文件如下：
+在 LLaMA-Factory 路径下新建`examples/train_full/internlm3_8b_instruct_full_sft.yaml`微调配置文件，微调配置文件如下：

 ```yaml
 ### model
@ -273,7 +254,7 @@ llamafactory-cli train examples/train_full/internlm3_8b_instruct_full_sft.yaml

 与GPU对比的loss曲线如下：

-![training_loss_compare](assets/lf_traing_loss_compare.png)
+![training_loss_compare](assets/lf_training_loss_compare.png)

 ## Transformers

--- a/assets/lf_training_loss_compare.png
+++ b/assets/lf_training_loss_compare.png
--- a/assets/training_loss.png
+++ b/assets/training_loss.png
--- a/assets/xtuner_training_loss_compare.png
+++ b/assets/xtuner_training_loss_compare.png