mirror of https://github.com/InternLM/InternLM
Merge branch 'update_readme' into 'main'
add reward model in readme See merge request openmmlab/bigmodel/InternLM!12pull/752/head
lvchengqi
commit
9e5f391691
90
README.md
90
README.md
|
|
@ -46,7 +46,7 @@ InternLM2.5 series are released with the following features:
|
|||
|
||||
## News
|
||||
|
||||
\[2024.06.30\] We release InternLM2.5-7B, InternLM2.5-7B-Chat and InternLM2.5-7B-Chat-1M. See [model zoo below](#model-zoo) for download or [model cards](./model_cards/) for more details.
|
||||
\[2024.06.30\] We release InternLM2.5-7B, InternLM2.5-7B-Chat and InternLM2.5-7B-Chat-1M. We also release the InternLM-Reward series of reward models in 1.8B, 7B and 20B sizes. See [model zoo below](#model-zoo) for download or [model cards](./model_cards/) for more details.
|
||||
|
||||
\[2024.03.26\] We release InternLM2 technical report. See [arXiv](https://arxiv.org/abs/2403.17297) for details.
|
||||
|
||||
|
|
@ -62,11 +62,13 @@ InternLM2.5 series are released with the following features:
|
|||
|
||||
## Model Zoo
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| --------------------------- | ----------------------------------------- | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2.5-7B** | [🤗internlm2_5-7b](https://huggingface.co/internlm/internlm2_5-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7B-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7B-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat-1M** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7b-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat-original) | 2024-06-30 |
|
||||
### InternLM2.5
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| -------------------------- | ------------------------------------------ | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2.5-7B** | [🤗internlm2_5-7b](https://huggingface.co/internlm/internlm2_5-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7B-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7B-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat-1M** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7b-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat-original) | 2024-06-30 |
|
||||
|
||||
**Notes:**
|
||||
|
||||
|
|
@ -80,11 +82,74 @@ The release of InternLM2.5 series contains 7B model size for now and we are goin
|
|||
|
||||
**Supplements:** `HF` refers to the format used by HuggingFace in [transformers](https://github.com/huggingface/transformers), whereas `Origin` denotes the format adopted by the InternLM team in [InternEvo](https://github.com/InternLM/InternEvo).
|
||||
|
||||
### InternLM-Reward
|
||||
|
||||
<details>
|
||||
<summary>(click to expand)</summary>
|
||||
|
||||
InternLM-Reward is a series of reward models, trained on 2.4 million preference samples, available in 1.8B, 7B, and 20B sizes. These model were applied to the PPO training process of our chat models. See [model cards](./model_cards/internlm_reward.md) for more details.
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | Release Date |
|
||||
| ------------------------ | -------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------------------- | ------------ |
|
||||
| **InternLM-Reward-1.8B** | [🤗internlm-reward-1_8b](https://huggingface.co/internlm/internlm-reward-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-1_8b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-1_8b) | 2024-06-30 |
|
||||
| **InternLM-Reward-7B** | [🤗internlm-reward-7b](https://huggingface.co/internlm/internlm-reward-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-7b) | 2024-06-30 |
|
||||
| **InternLM-Reward-20B** | [🤗internlm-reward-20b](https://huggingface.co/internlm/internlm-reward-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-20b) | 2024-06-30 |
|
||||
|
||||
</details>
|
||||
|
||||
### InternLM2
|
||||
|
||||
<details>
|
||||
<summary>(click to expand)</summary>
|
||||
|
||||
Our previous generation models with advanced capabilities in long-context processing, reasoning, and coding. See [model cards](./model_cards/) for more details.
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| --------------------------- | ----------------------------------------- | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2-1.8B** | [🤗internlm2-1.8b](https://huggingface.co/internlm/internlm2-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-1.8b](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-1.8b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-1.8b-original) | 2024-01-31 |
|
||||
| **InternLM2-Chat-1.8B-SFT** | [🤗internlm2-chat-1.8b-sft](https://huggingface.co/internlm/internlm2-chat-1_8b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-1.8b-sft](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-sft-original) | 2024-01-31 |
|
||||
| **InternLM2-Chat-1.8B** | [🤗internlm2-chat-1.8b](https://huggingface.co/internlm/internlm2-chat-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-1.8b](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-original) | 2024-02-19 |
|
||||
| **InternLM2-Base-7B** | [🤗internlm2-base-7b](https://huggingface.co/internlm/internlm2-base-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-base-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-base-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-7B** | [🤗internlm2-7b](https://huggingface.co/internlm/internlm2-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-7B-SFT** | [🤗internlm2-chat-7b-sft](https://huggingface.co/internlm/internlm2-chat-7b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-7b-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-sft-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-7B** | [🤗internlm2-chat-7b](https://huggingface.co/internlm/internlm2-chat-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-Base-20B** | [🤗internlm2-base-20b](https://huggingface.co/internlm/internlm2-base-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-base-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-base-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-20b-original) | 2024-01-17 |
|
||||
| **InternLM2-20B** | [🤗internlm2-20b](https://huggingface.co/internlm/internlm2-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-20b-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-20B-SFT** | [🤗internlm2-chat-20b-sft](https://huggingface.co/internlm/internlm2-chat-20b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-20b-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-sft-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-20B** | [🤗internlm2-chat-20b](https://huggingface.co/internlm/internlm2-chat-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-original) | 2024-01-17 |
|
||||
|
||||
</details>
|
||||
|
||||
## Performance
|
||||
|
||||
### Objective Evaluation
|
||||
We have evaluated InternLM2.5 on several important benchmarks using the open-source evaluation tool [OpenCompass](https://github.com/open-compass/opencompass). Some of the evaluation results are shown in the table below. You are welcome to visit the [OpenCompass Leaderboard](https://opencompass.org.cn/rank) for more evaluation results.
|
||||
|
||||
### Alignment Evaluation
|
||||
### Base Model
|
||||
|
||||
| Benchmark | InternLM2-7B | LLaMA-3-8B | Yi-1.5-9B | InternLM2.5-7B |
|
||||
| ------------- | ------------ | ---------- | --------- | -------------- |
|
||||
| MMLU(5-shot) | 65.8 | 66.4 | 71.6 | 71.6 |
|
||||
| CMMLU(5-shot) | 66.2 | 51.0 | 74.1 | 79.1 |
|
||||
| BBH(3-shot) | 65.0 | 59.7 | 71.1 | 70.1 |
|
||||
| MATH(4-shot) | 20.2 | 16.4 | 31.9 | 34.0 |
|
||||
| GSM8K(4-shot) | 70.8 | 54.3 | 74.5 | 74.8 |
|
||||
| GPQA(0-shot) | 28.3 | 31.3 | 27.8 | 31.3 |
|
||||
|
||||
### Chat Model
|
||||
|
||||
| Benchmark | InternLM2-Chat-7B | LLaMA-3-8B-Instruct | Yi-1.5-9B-Chat | GLM-4-9B-Chat | Qwen2-7B-Instruct | Gemma2-9B-IT | InternLM2.5-7B-Chat |
|
||||
| ----------------- | ----------------- | ------------------- | -------------- | ------------- | ----------------- | ------------ | ------------------- |
|
||||
| MMLU(5-shot) | 62.3 | 68.4 | 71.0 | 71.4 | 70.8 | 70.9 | 72.8 |
|
||||
| CMMLU(5-shot) | 62.4 | 53.3 | 74.5 | 74.5 | 80.9 | 60.3 | 78.0 |
|
||||
| BBH(3-shot CoT) | 59.0 | 54.4 | 69.6 | 69.6 | 65.0 | 68.2\* | 71.6 |
|
||||
| MATH(0-shot CoT) | 27.6 | 27.9 | 51.1 | 51.1 | 48.6 | 46.9 | 60.1 |
|
||||
| GSM8K(0-shot CoT) | 72.5 | 72.9 | 80.1 | 85.3 | 82.9 | 88.9 | 86.0 |
|
||||
| GPQA(0-shot) | 29.8 | 26.1 | 37.9 | 36.9 | 38.4 | 33.8 | 38.4 |
|
||||
|
||||
- We use `ppl` for the MCQ evaluation on base model.
|
||||
- The evaluation results were obtained from [OpenCompass](https://github.com/open-compass/opencompass) , and evaluation configuration can be found in the configuration files provided by [OpenCompass](https://github.com/open-compass/opencompass).
|
||||
- The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/open-compass/opencompass), so please refer to the latest evaluation results of [OpenCompass](https://github.com/open-compass/opencompass).
|
||||
- \* means the result is copied from the original paper.
|
||||
|
||||
## Requirements
|
||||
|
||||
|
|
@ -177,10 +242,17 @@ Please refer to the [guidance](./chat/lmdeploy.md) for more usages about model d
|
|||
|
||||
By enabling the Dynamic NTK feature of LMDeploy, you can acquire the long-context inference power.
|
||||
|
||||
Note: 1M context length requires 4xA100-80G.
|
||||
|
||||
```python
|
||||
from lmdeploy import pipeline, GenerationConfig, TurbomindEngineConfig
|
||||
|
||||
backend_config = TurbomindEngineConfig(rope_scaling_factor=2.5, session_len=1048576)
|
||||
backend_config = TurbomindEngineConfig(
|
||||
rope_scaling_factor=2.5,
|
||||
session_len=1048576, # 1M context length
|
||||
max_batch_size=1,
|
||||
cache_max_entry_count=0.7,
|
||||
tp=4) # 4xA100-80G.
|
||||
pipe = pipeline('internlm/internlm2_5-7b-chat-1m', backend_config=backend_config)
|
||||
prompt = 'Use a long prompt to replace this sentence'
|
||||
response = pipe(prompt)
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ InternLM2.5 系列模型在本仓库正式发布，具有如下特性：
|
|||
|
||||
## 更新
|
||||
|
||||
\[2024.06.30\] 我们发布了 InternLM2.5-7B、InternLM2.5-7B-Chat 和 InternLM2.5-7B-Chat-1M.。可以在下方的 [模型库](#model-zoo) 进行下载，或者在 [model cards](./model_cards/) 中了解更多细节。
|
||||
\[2024.06.30\] 我们发布了 InternLM2.5-7B、InternLM2.5-7B-Chat 和 InternLM2.5-7B-Chat-1M。同时我们也发布了 1.8B、7B 和 20B 大小的 InternLM-Reward 系列奖励模型。可以在下方的 [模型库](#model-zoo) 进行下载，或者在 [model cards](./model_cards/) 中了解更多细节。
|
||||
|
||||
\[2024.03.26\] 我们发布了 InternLM2 的技术报告。 可以点击 [arXiv链接](https://arxiv.org/abs/2403.17297) 来了解更多细节。
|
||||
|
||||
|
|
@ -60,11 +60,13 @@ InternLM2.5 系列模型在本仓库正式发布，具有如下特性：
|
|||
|
||||
## Model Zoo
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| --------------------------- | ----------------------------------------- | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2.5-7B** | [🤗internlm2_5-7b](https://huggingface.co/internlm/internlm2_5-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7B-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7B-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat-1M** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7b-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat-original) | 2024-06-30 |
|
||||
### InternLM2.5
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| -------------------------- | ------------------------------------------ | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2.5-7B** | [🤗internlm2_5-7b](https://huggingface.co/internlm/internlm2_5-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7B-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7B-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7B-chat-original) | 2024-06-30 |
|
||||
| **InternLM2.5-7B-Chat-1M** | [🤗internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2_5-7b-chat](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2_5-7b-chat/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2_5-7b-chat-original) | 2024-06-30 |
|
||||
|
||||
**模型说明：**
|
||||
|
||||
|
|
@ -80,11 +82,74 @@ InternLM2.5 系列模型在本仓库正式发布，具有如下特性：
|
|||
|
||||
**补充说明：** 上表中的 `HF` 表示对应模型为 HuggingFace 平台提供的 [transformers](https://github.com/huggingface/transformers) 框架格式；`Origin` 则表示对应模型为我们 InternLM 团队的 [InternEvo](https://github.com/InternLM/InternEvo) 框架格式。
|
||||
|
||||
### InternLM-Reward
|
||||
|
||||
<details>
|
||||
<summary>(click to expand)</summary>
|
||||
|
||||
InternLM-Reward 是基于 240 万个偏好样本进行训练的奖励模型，有 1.8B、7B 和 20B 大小可供选择。这些模型被用于 InternLM 对话模型的 PPO 训练过程。请参考 [model cards](./model_cards/internlm_reward.md) 了解更多细节。
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | Release Date |
|
||||
| ------------------------ | -------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------------------- | ------------ |
|
||||
| **InternLM-Reward-1.8B** | [🤗internlm-reward-1_8b](https://huggingface.co/internlm/internlm-reward-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-1_8b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-1_8b) | 2024-06-30 |
|
||||
| **InternLM-Reward-7B** | [🤗internlm-reward-7b](https://huggingface.co/internlm/internlm-reward-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-7b) | 2024-06-30 |
|
||||
| **InternLM-Reward-20B** | [🤗internlm-reward-20b](https://huggingface.co/internlm/internlm-reward-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm-reward-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-20b) | 2024-06-30 |
|
||||
|
||||
</details>
|
||||
|
||||
### InternLM2
|
||||
|
||||
<details>
|
||||
<summary>(click to expand)</summary>
|
||||
|
||||
我们上一代的模型，在长上下文处理、推理和编码方面具有优秀的性能。请参考 [model cards](./model_cards/) 了解更多细节。
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | OpenXLab(Origin) | Release Date |
|
||||
| --------------------------- | ----------------------------------------- | ---------------------------------------- | -------------------------------------- | ------------------------------------------ | ------------ |
|
||||
| **InternLM2-1.8B** | [🤗internlm2-1.8b](https://huggingface.co/internlm/internlm2-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-1.8b](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-1.8b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-1.8b-original) | 2024-01-31 |
|
||||
| **InternLM2-Chat-1.8B-SFT** | [🤗internlm2-chat-1.8b-sft](https://huggingface.co/internlm/internlm2-chat-1_8b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-1.8b-sft](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-sft-original) | 2024-01-31 |
|
||||
| **InternLM2-Chat-1.8B** | [🤗internlm2-chat-1.8b](https://huggingface.co/internlm/internlm2-chat-1_8b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-1.8b](https://www.modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-1.8b-original) | 2024-02-19 |
|
||||
| **InternLM2-Base-7B** | [🤗internlm2-base-7b](https://huggingface.co/internlm/internlm2-base-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-base-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-base-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-7B** | [🤗internlm2-7b](https://huggingface.co/internlm/internlm2-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-7B-SFT** | [🤗internlm2-chat-7b-sft](https://huggingface.co/internlm/internlm2-chat-7b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-7b-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-sft-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-7B** | [🤗internlm2-chat-7b](https://huggingface.co/internlm/internlm2-chat-7b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-7b-original) | 2024-01-17 |
|
||||
| **InternLM2-Base-20B** | [🤗internlm2-base-20b](https://huggingface.co/internlm/internlm2-base-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-base-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-base-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-base-20b-original) | 2024-01-17 |
|
||||
| **InternLM2-20B** | [🤗internlm2-20b](https://huggingface.co/internlm/internlm2-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-20b-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-20B-SFT** | [🤗internlm2-chat-20b-sft](https://huggingface.co/internlm/internlm2-chat-20b-sft) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-20b-sft](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b-sft/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-sft) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-sft-original) | 2024-01-17 |
|
||||
| **InternLM2-Chat-20B** | [🤗internlm2-chat-20b](https://huggingface.co/internlm/internlm2-chat-20b) | [<img src="./assets/modelscope_logo.png" width="20px" /> internlm2-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-chat-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm2-chat-20b-original) | 2024-01-17 |
|
||||
|
||||
</details>
|
||||
|
||||
## 性能
|
||||
|
||||
### 客观评测
|
||||
我们使用开源评测工具 [OpenCompass](https://github.com/open-compass/opencompass) 在几个重要的基准测试中对 InternLM2.5 进行了评测。部分评测结果如下表所示。欢迎访问 [OpenCompass 排行榜](https://opencompass.org.cn/rank) 获取更多评测结果。
|
||||
|
||||
### 主观评测
|
||||
### 基座模型
|
||||
|
||||
| Benchmark | InternLM2-7B | LLaMA-3-8B | Yi-1.5-9B | InternLM2.5-7B |
|
||||
| ------------- | ------------ | ---------- | --------- | -------------- |
|
||||
| MMLU(5-shot) | 65.8 | 66.4 | 71.6 | 71.6 |
|
||||
| CMMLU(5-shot) | 66.2 | 51.0 | 74.1 | 79.1 |
|
||||
| BBH(3-shot) | 65.0 | 59.7 | 71.1 | 70.1 |
|
||||
| MATH(4-shot) | 20.2 | 16.4 | 31.9 | 34.0 |
|
||||
| GSM8K(4-shot) | 70.8 | 54.3 | 74.5 | 74.8 |
|
||||
| GPQA(0-shot) | 28.3 | 31.3 | 27.8 | 31.3 |
|
||||
|
||||
### 对话模型
|
||||
|
||||
| Benchmark | InternLM2-Chat-7B | LLaMA-3-8B-Instruct | Yi-1.5-9B-Chat | GLM-4-9B-Chat | Qwen2-7B-Instruct | Gemma2-9B-IT | InternLM2.5-7B-Chat |
|
||||
| ----------------- | ----------------- | ------------------- | -------------- | ------------- | ----------------- | ------------ | ------------------- |
|
||||
| MMLU(5-shot) | 62.3 | 68.4 | 71.0 | 71.4 | 70.8 | 70.9 | 72.8 |
|
||||
| CMMLU(5-shot) | 62.4 | 53.3 | 74.5 | 74.5 | 80.9 | 60.3 | 78.0 |
|
||||
| BBH(3-shot CoT) | 59.0 | 54.4 | 69.6 | 69.6 | 65.0 | 68.2\* | 71.6 |
|
||||
| MATH(0-shot CoT) | 27.6 | 27.9 | 51.1 | 51.1 | 48.6 | 46.9 | 60.1 |
|
||||
| GSM8K(0-shot CoT) | 72.5 | 72.9 | 80.1 | 85.3 | 82.9 | 88.9 | 86.0 |
|
||||
| GPQA(0-shot) | 29.8 | 26.1 | 37.9 | 36.9 | 38.4 | 33.8 | 38.4 |
|
||||
|
||||
- 我们使用 `ppl` 对基座模型进行 MCQ 指标的评测。
|
||||
- 评测结果来自 [OpenCompass](https://github.com/open-compass/opencompass) ，评测配置可以在 [OpenCompass](https://github.com/open-compass/opencompass) 提供的配置文件中找到。
|
||||
- 由于 [OpenCompass](https://github.com/open-compass/opencompass) 的版本迭代，评测数据可能存在数值差异，因此请参考 [OpenCompass](https://github.com/open-compass/opencompass) 的最新评测结果。
|
||||
- \* 表示从原论文中复制而来。
|
||||
|
||||
## 依赖
|
||||
|
||||
|
|
@ -174,13 +239,20 @@ print(response)
|
|||
|
||||
### 1百万字超长上下文推理
|
||||
|
||||
激活 LMDeploy 的 Dynamic NTK 能力，可以轻松把 internlm2_5-7b-chat 外推到 200K 上下文
|
||||
激活 LMDeploy 的 Dynamic NTK 能力，可以轻松把 internlm2_5-7b-chat 外推到 200K 上下文。
|
||||
|
||||
注意: 1M 上下文需要 4xA100-80G。
|
||||
|
||||
```python
|
||||
from lmdeploy import pipeline, GenerationConfig, TurbomindEngineConfig
|
||||
|
||||
backend_config = TurbomindEngineConfig(rope_scaling_factor=2.5, session_len=1048576)
|
||||
pipe = pipeline('internlm/internlm2_5-7b-chat', backend_config=backend_config)
|
||||
backend_config = TurbomindEngineConfig(
|
||||
rope_scaling_factor=2.5,
|
||||
session_len=1048576, # 1M context length
|
||||
max_batch_size=1,
|
||||
cache_max_entry_count=0.7,
|
||||
tp=4) # 4xA100-80G.
|
||||
pipe = pipeline('internlm/internlm2_5-7b-chat-1m', backend_config=backend_config)
|
||||
prompt = 'Use a long prompt to replace this sentence'
|
||||
response = pipe(prompt)
|
||||
print(response)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,157 @@
|
|||
# InternLM-Reward Model Card
|
||||
|
||||
## Introduction
|
||||
|
||||
**InternLM-Reward** is a reward model trained on the foundation of InternLM2-Chat-SFT. This model has been trained using over 2.4 million preference samples, both human-annotated and AI-synthesized, achieving outstanding performance while ensuring a balance between helpful and harmless.
|
||||
|
||||
- **Variety of Sizes Available**: Our open-sourced reward models are available in sizes of **1.8B, 7B, and 20B**, each demonstrating exceptional performance across various metrics.
|
||||
- **Comprehensive Coverage of Preference**: Trained with **2.4 million** preference pairs derived from both human annotations and AI synthesis, covering diverse areas such as dialogue, writing, poetry, summarization, coding, mathematics, etc. It also maintains a balance between helpful and harmless.
|
||||
- **Multilingual Support**: InternLM-Reward was trained on high-quality **English and Chinese** preference data, delivering robust performance in both languages.
|
||||
|
||||
This model was applied to the PPO training process of InternLM2-Chat. The reward model training techniques from the [InternLM2 Technical Report](https://arxiv.org/abs/2403.17297) have been open-sourced in XTuner, try it out [here](https://github.com/InternLM/xtuner)!
|
||||
|
||||
## Model Zoo
|
||||
|
||||
| Model | Transformers(HF) | ModelScope(HF) | OpenXLab(HF) | Release Date |
|
||||
| ------------------------ | -------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------------------------- | ------------ |
|
||||
| **InternLM-Reward-1.8B** | [🤗internlm-reward-1_8b](https://huggingface.co/internlm/internlm-reward-1_8b) | [<img src="../assets/modelscope_logo.png" width="20px" /> internlm-reward-1_8b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-1_8b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-1_8b) | 2024-06-30 |
|
||||
| **InternLM-Reward-7B** | [🤗internlm-reward-7b](https://huggingface.co/internlm/internlm-reward-7b) | [<img src="../assets/modelscope_logo.png" width="20px" /> internlm-reward-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-7b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-7b) | 2024-06-30 |
|
||||
| **InternLM-Reward-20B** | [🤗internlm-reward-20b](https://huggingface.co/internlm/internlm-reward-20b) | [<img src="../assets/modelscope_logo.png" width="20px" /> internlm-reward-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-reward-20b/summary) | [](https://openxlab.org.cn/models/detail/OpenLMLab/internlm-reward-20b) | 2024-06-30 |
|
||||
|
||||
## Performance Evaluation
|
||||
|
||||
| Models | Score | Chat | Chat Hard | Safety | Reasoning |
|
||||
| -------------------- | ----- | ---- | --------- | ------ | --------- |
|
||||
| InternLM-Reward-20B | 89.5 | 98.6 | 74.1 | 89.4 | 95.7 |
|
||||
| InternLM-Reward-7B | 86.6 | 98.6 | 66.7 | 88.3 | 92.8 |
|
||||
| InternLM-Reward-1.8B | 80.6 | 95.0 | 58.1 | 81.8 | 87.4 |
|
||||
|
||||
- The evaluation is conducted on the [RewardBench](https://github.com/allenai/reward-bench) dataset.
|
||||
- For a fair comparison, conditional system prompts proposed in our technical report were not included during testing.
|
||||
|
||||
## Demo Code
|
||||
|
||||
### Basic Usage
|
||||
|
||||
We provide some user-friendly APIs for you to use the model. Here is an example of how to use the model to get the reward score of a chat, compare two chats, or rank multiple chats.
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModel, AutoTokenizer
|
||||
|
||||
model = AutoModel.from_pretrained(
|
||||
"internlm/internlm-reward-7b",
|
||||
device_map="cuda",
|
||||
torch_dtype=torch.float16,
|
||||
trust_remote_code=True,
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-reward-7b", trust_remote_code=True)
|
||||
|
||||
chat_1 = [
|
||||
{"role": "user", "content": "Hello! What's your name?"},
|
||||
{"role": "assistant", "content": "My name is InternLM2! A helpful AI assistant. What can I do for you?"}
|
||||
]
|
||||
chat_2 = [
|
||||
{"role": "user", "content": "Hello! What's your name?"},
|
||||
{"role": "assistant", "content": "I have no idea."}
|
||||
]
|
||||
|
||||
|
||||
# get reward score for a single chat
|
||||
score1 = model.get_score(tokenizer, chat_1)
|
||||
score2 = model.get_score(tokenizer, chat_2)
|
||||
print("score1: ", score1)
|
||||
print("score2: ", score2)
|
||||
# >>> score1: 0.767578125
|
||||
# >>> score2: -2.22265625
|
||||
|
||||
|
||||
# batch inference, get multiple scores at once
|
||||
scores = model.get_scores(tokenizer, [chat_1, chat_2])
|
||||
print("scores: ", scores)
|
||||
# >>> scores: [0.767578125, -2.22265625]
|
||||
|
||||
|
||||
# compare whether chat_1 is better than chat_2
|
||||
compare_res = model.compare(tokenizer, chat_1, chat_2)
|
||||
print("compare_res: ", compare_res)
|
||||
# >>> compare_res: True
|
||||
|
||||
|
||||
# rank multiple chats, it will return the ranking index of each chat
|
||||
# the chat with the highest score will have ranking index as 0
|
||||
rank_res = model.rank(tokenizer, [chat_1, chat_2])
|
||||
print("rank_res: ", rank_res) # lower index means higher score
|
||||
# >>> rank_res: [0, 1]
|
||||
```
|
||||
|
||||
### Best of N Sampling
|
||||
|
||||
Here is an example of how to use the reward model to perform best of N sampling.
|
||||
The code below demonstrates how to select the best response from the candidates generated by the language model.
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModel, AutoTokenizer
|
||||
|
||||
# prepare the llm model and tokenizer
|
||||
llm = AutoModel.from_pretrained(
|
||||
"internlm/internlm2-chat-7b",
|
||||
device_map="cuda",
|
||||
torch_dtype=torch.float16,
|
||||
trust_remote_code=True,
|
||||
)
|
||||
llm_tokenizer = AutoTokenizer.from_pretrained("internlm/internlm2-chat-7b", trust_remote_code=True)
|
||||
|
||||
# prepare the reward model and tokenizer
|
||||
reward = AutoModel.from_pretrained(
|
||||
"internlm/internlm-reward-7b",
|
||||
device_map="cuda",
|
||||
torch_dtype=torch.float16,
|
||||
trust_remote_code=True,
|
||||
)
|
||||
reward_tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-reward-7b", trust_remote_code=True)
|
||||
|
||||
# prepare the chat prompt
|
||||
prompt = "Write an article about the artificial intelligence revolution."
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "user", "content": prompt}
|
||||
]
|
||||
text = llm_tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True
|
||||
)
|
||||
model_inputs = llm_tokenizer([text], return_tensors="pt").to("cuda")
|
||||
|
||||
# generate best of N candidates
|
||||
num_candidates = 10 # N=10
|
||||
candidates = []
|
||||
|
||||
outputs = llm.generate(
|
||||
**model_inputs,
|
||||
max_new_tokens=512,
|
||||
num_return_sequences=num_candidates,
|
||||
pad_token_id=llm_tokenizer.eos_token_id,
|
||||
do_sample=True,
|
||||
top_k=50,
|
||||
top_p=0.95,
|
||||
temperature=0.8,
|
||||
)
|
||||
outputs = outputs[:, model_inputs["input_ids"].shape[1]:]
|
||||
for i in range(num_candidates):
|
||||
candidate = llm_tokenizer.decode(outputs[i], skip_special_tokens=True)
|
||||
candidates.append(messages + [{"role": "assistant", "content": candidate}])
|
||||
|
||||
rank_indices = reward.rank(reward_tokenizer, candidates)
|
||||
sorted_candidates = sorted(zip(rank_indices, candidates), key=lambda x: x[0])
|
||||
|
||||
## print the ranked candidates
|
||||
# for i, (rank_index, candidate) in enumerate(sorted_candidates):
|
||||
# print(f"------------Rank {i}------------: \n{candidate[-1]['content']}")
|
||||
|
||||
# print the best response
|
||||
best_response = sorted_candidates[0][1][-1]['content']
|
||||
print(best_response)
|
||||
```
|
||||
Loading…
Reference in New Issue