mirror of https://github.com/hpcaitech/ColossalAI
parent
3c9cd5bb5e
commit
270157e9e7
|
@ -33,6 +33,7 @@
|
||||||
<li><a href="#GPT-3">GPT-3</a></li>
|
<li><a href="#GPT-3">GPT-3</a></li>
|
||||||
<li><a href="#GPT-2">GPT-2</a></li>
|
<li><a href="#GPT-2">GPT-2</a></li>
|
||||||
<li><a href="#BERT">BERT</a></li>
|
<li><a href="#BERT">BERT</a></li>
|
||||||
|
<li><a href="#PaLM">BERT</a></li>
|
||||||
</ul>
|
</ul>
|
||||||
</li>
|
</li>
|
||||||
|
|
||||||
|
@ -96,6 +97,9 @@ Colossal-AI为您提供了一系列并行训练组件。我们的目标是让您
|
||||||
|
|
||||||
- 2倍训练速度,或1.5倍序列长度
|
- 2倍训练速度,或1.5倍序列长度
|
||||||
|
|
||||||
|
### PaLM
|
||||||
|
- [PaLM-colossalai](https://github.com/hpcaitech/PaLM-colossalai): 可扩展的谷歌 Pathways Language Model ([PaLM](https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html)) 实现。
|
||||||
|
|
||||||
请访问我们的[文档和教程](https://www.colossalai.org/)以了解详情。
|
请访问我们的[文档和教程](https://www.colossalai.org/)以了解详情。
|
||||||
|
|
||||||
<p align="right">(<a href="#top">返回顶端</a>)</p>
|
<p align="right">(<a href="#top">返回顶端</a>)</p>
|
||||||
|
|
|
@ -33,6 +33,7 @@
|
||||||
<li><a href="#GPT-3">GPT-3</a></li>
|
<li><a href="#GPT-3">GPT-3</a></li>
|
||||||
<li><a href="#GPT-2">GPT-2</a></li>
|
<li><a href="#GPT-2">GPT-2</a></li>
|
||||||
<li><a href="#BERT">BERT</a></li>
|
<li><a href="#BERT">BERT</a></li>
|
||||||
|
<li><a href="#PaLM">BERT</a></li>
|
||||||
</ul>
|
</ul>
|
||||||
</li>
|
</li>
|
||||||
|
|
||||||
|
@ -96,6 +97,9 @@ distributed training in a few lines.
|
||||||
|
|
||||||
- 2x faster training, or 50% longer sequence length
|
- 2x faster training, or 50% longer sequence length
|
||||||
|
|
||||||
|
### PaLM
|
||||||
|
- [PaLM-colossalai](https://github.com/hpcaitech/PaLM-colossalai): Scalable implementation of Google's Pathways Language Model ([PaLM](https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html)).
|
||||||
|
|
||||||
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
|
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
|
||||||
|
|
||||||
<p align="right">(<a href="#top">back to top</a>)</p>
|
<p align="right">(<a href="#top">back to top</a>)</p>
|
||||||
|
|
Loading…
Reference in New Issue