github
/
InternLM
mirror of https://github.com/InternLM/InternLM
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

update README in tools

pull/627/head
gaoyang07 2024-01-19 19:29:22 +08:00
parent 1758aff49c
commit 42dc77793f
1 changed files with 8 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
tools/README.md
View File

@ -1,9 +1,14 @@
# InternLM2 tools # InternLM2 tools
## Convert to LLaMA ## 1. Convert to LLaMA
`convert2llama.py` can convert InternLM2 to LLaMA: We offer the `convert2llama.py`, designed to seamlessly transform InternLM2 (HF format) into LLaMA (HF format). Here, HF refers to the format used by HuggingFace Transformers.
### Usage
``` ```
python convert2llama.py --src /path/to/internlm2 --tgt /path/to/target python convert2llama.py --src /path/to/internlm2/ckpt --tgt /path/to/target/ckpt
``` ```
### Note
While the `convert2llama.py` tool is available, we still advise opting for InternLM2 when practical, chiefly due to its superior efficiency. InternLM2, which is adapted from LLaMA, streamlines the process by integrating the `Wq`, `Wk`, `Wv` weight matrices into a single matrix `Wqkv`. This integration leads to approximately a **5%** speed increase during training. Given the substantial costs associated with pre-training, this efficiency boost can result in significant savings.