mirror of https://github.com/hpcaitech/ColossalAI
[DOC] hotfix/llama2news (#4595)
* [doc] add llama2 news * [doc] add llama2 news * [doc] add llama2 newspull/4602/head
parent
63ecafb1fb
commit
7a978eb3d0
13
README.md
13
README.md
|
@ -25,6 +25,7 @@
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
## Latest News
|
## Latest News
|
||||||
|
* [2023/09] [70 Billion Parameter LLaMA2 Model Training Accelerated by 195%](https://www.hpc-ai.tech/blog/70b-llama2-training)
|
||||||
* [2023/07] [HPC-AI Tech Raises 22 Million USD in Series A Funding](https://www.hpc-ai.tech/blog/hpc-ai-tech-raises-22-million-usd-in-series-a-funding-to-fuel-team-expansion-and-business-growth)
|
* [2023/07] [HPC-AI Tech Raises 22 Million USD in Series A Funding](https://www.hpc-ai.tech/blog/hpc-ai-tech-raises-22-million-usd-in-series-a-funding-to-fuel-team-expansion-and-business-growth)
|
||||||
* [2023/07] [65B Model Pretraining Accelerated by 38%, Best Practices for Building LLaMA-Like Base Models Open-Source](https://www.hpc-ai.tech/blog/large-model-pretraining)
|
* [2023/07] [65B Model Pretraining Accelerated by 38%, Best Practices for Building LLaMA-Like Base Models Open-Source](https://www.hpc-ai.tech/blog/large-model-pretraining)
|
||||||
* [2023/03] [ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline](https://medium.com/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b)
|
* [2023/03] [ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline](https://medium.com/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b)
|
||||||
|
@ -50,7 +51,7 @@
|
||||||
<li>
|
<li>
|
||||||
<a href="#Parallel-Training-Demo">Parallel Training Demo</a>
|
<a href="#Parallel-Training-Demo">Parallel Training Demo</a>
|
||||||
<ul>
|
<ul>
|
||||||
<li><a href="#LLaMA">LLaMA</a></li>
|
<li><a href="#LLaMA2">LLaMA 1/2</a></li>
|
||||||
<li><a href="#GPT-3">GPT-3</a></li>
|
<li><a href="#GPT-3">GPT-3</a></li>
|
||||||
<li><a href="#GPT-2">GPT-2</a></li>
|
<li><a href="#GPT-2">GPT-2</a></li>
|
||||||
<li><a href="#BERT">BERT</a></li>
|
<li><a href="#BERT">BERT</a></li>
|
||||||
|
@ -217,8 +218,16 @@ Acceleration of [AlphaFold Protein Structure](https://alphafold.ebi.ac.uk/)
|
||||||
<p align="right">(<a href="#top">back to top</a>)</p>
|
<p align="right">(<a href="#top">back to top</a>)</p>
|
||||||
|
|
||||||
## Parallel Training Demo
|
## Parallel Training Demo
|
||||||
|
### LLaMA2
|
||||||
|
<p align="center">
|
||||||
|
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/llama2_pretraining.png" width=600/>
|
||||||
|
</p>
|
||||||
|
|
||||||
### LLaMA
|
- 70 billion parameter LLaMA2 model training accelerated by 195%
|
||||||
|
[[code]](https://github.com/hpcaitech/ColossalAI/tree/example/llama/examples/language/llama)
|
||||||
|
[[blog]](https://www.hpc-ai.tech/blog/70b-llama2-training)
|
||||||
|
|
||||||
|
### LLaMA1
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/examples/images/LLaMA_pretraining.png" width=600/>
|
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/examples/images/LLaMA_pretraining.png" width=600/>
|
||||||
</p>
|
</p>
|
||||||
|
|
|
@ -24,6 +24,7 @@
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
## 新闻
|
## 新闻
|
||||||
|
* [2023/09] [70 Billion Parameter LLaMA2 Model Training Accelerated by 195%](https://www.hpc-ai.tech/blog/70b-llama2-training)
|
||||||
* [2023/07] [HPC-AI Tech Raises 22 Million USD in Series A Funding](https://www.hpc-ai.tech/blog/hpc-ai-tech-raises-22-million-usd-in-series-a-funding-to-fuel-team-expansion-and-business-growth)
|
* [2023/07] [HPC-AI Tech Raises 22 Million USD in Series A Funding](https://www.hpc-ai.tech/blog/hpc-ai-tech-raises-22-million-usd-in-series-a-funding-to-fuel-team-expansion-and-business-growth)
|
||||||
* [2023/07] [65B Model Pretraining Accelerated by 38%, Best Practices for Building LLaMA-Like Base Models Open-Source](https://www.hpc-ai.tech/blog/large-model-pretraining)
|
* [2023/07] [65B Model Pretraining Accelerated by 38%, Best Practices for Building LLaMA-Like Base Models Open-Source](https://www.hpc-ai.tech/blog/large-model-pretraining)
|
||||||
* [2023/03] [ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline](https://medium.com/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b)
|
* [2023/03] [ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline](https://medium.com/@yangyou_berkeley/colossalchat-an-open-source-solution-for-cloning-chatgpt-with-a-complete-rlhf-pipeline-5edf08fb538b)
|
||||||
|
@ -49,7 +50,7 @@
|
||||||
<li>
|
<li>
|
||||||
<a href="#并行训练样例展示">并行训练样例展示</a>
|
<a href="#并行训练样例展示">并行训练样例展示</a>
|
||||||
<ul>
|
<ul>
|
||||||
<li><a href="#LLaMA">LLaMA</a></li>
|
<li><a href="#LLaMA2">LLaMA 1/2</a></li>
|
||||||
<li><a href="#GPT-3">GPT-3</a></li>
|
<li><a href="#GPT-3">GPT-3</a></li>
|
||||||
<li><a href="#GPT-2">GPT-2</a></li>
|
<li><a href="#GPT-2">GPT-2</a></li>
|
||||||
<li><a href="#BERT">BERT</a></li>
|
<li><a href="#BERT">BERT</a></li>
|
||||||
|
@ -210,7 +211,16 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的
|
||||||
<p align="right">(<a href="#top">返回顶端</a>)</p>
|
<p align="right">(<a href="#top">返回顶端</a>)</p>
|
||||||
|
|
||||||
## 并行训练样例展示
|
## 并行训练样例展示
|
||||||
### LLaMA
|
### LLaMA2
|
||||||
|
<p align="center">
|
||||||
|
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/llama2_pretraining.png" width=600/>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
- 700亿参数LLaMA2训练加速195%
|
||||||
|
[[code]](https://github.com/hpcaitech/ColossalAI/tree/example/llama/examples/language/llama)
|
||||||
|
[[blog]](https://www.hpc-ai.tech/blog/70b-llama2-training)
|
||||||
|
|
||||||
|
### LLaMA1
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/examples/images/LLaMA_pretraining.png" width=600/>
|
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/examples/images/LLaMA_pretraining.png" width=600/>
|
||||||
</p>
|
</p>
|
||||||
|
|
Loading…
Reference in New Issue