add community group and update issue template(#271)
|
@ -1,5 +1,10 @@
|
||||||
blank_issues_enabled: true
|
blank_issues_enabled: true
|
||||||
contact_links:
|
contact_links:
|
||||||
- name: "😊 Discussions"
|
- name: ❓ Simple question - Slack Chat
|
||||||
url: https://github.com/hpcaitech/ColossalAI/discussions
|
url: https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w
|
||||||
about: Ask questions and discuss with other Colossal-AI community members in our forum
|
about: This issue tracker is not for technical support. Please use our Slack chat, and ask the community for help.
|
||||||
|
- name: ❓ Simple question - WeChat
|
||||||
|
url: https://github.com/hpcaitech/ColossalAI/blob/main/docs/images/WeChat.png
|
||||||
|
about: This issue tracker is not for technical support. Please use WeChat, and ask the community for help.
|
||||||
|
- name: 😊 Advanced question - GitHub Discussions
|
||||||
|
url: https://github.com/hpcaitech/ColossalAI/discussions
|
|
@ -0,0 +1,44 @@
|
||||||
|
name: 💥 Proposal
|
||||||
|
description: Propose a non-trivial change to Colossal-AI
|
||||||
|
|
||||||
|
body:
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: |
|
||||||
|
Common reasons for proposals include:
|
||||||
|
|
||||||
|
- Altering the infrastructure;
|
||||||
|
- Bumping a critical dependency's major version;
|
||||||
|
- A significant improvement in user-friendliness;
|
||||||
|
- Significant refactor;
|
||||||
|
- ...
|
||||||
|
|
||||||
|
Please note this is not for feature request or bug template; such action could make us identify the issue wrongly and close it without doing anything.
|
||||||
|
|
||||||
|
We give you maximum freedom to write an elaborated proposal illustrating why you think the change is beneficial for us, and what steps we should take to turn this into reality.
|
||||||
|
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
attributes:
|
||||||
|
label: Proposal
|
||||||
|
description: A clear and concise description of what the proposal is.
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: checkboxes
|
||||||
|
attributes:
|
||||||
|
label: Self-service
|
||||||
|
description: |
|
||||||
|
If you feel like you could contribute to this issue, please check the box below. This would tell us and other people looking for contributions that someone's working on it.
|
||||||
|
If you do check this box, please send a pull request within 7 days after a maintainer's approval so we can still delegate this to someone else.
|
||||||
|
|
||||||
|
Proposals usually involve significant code changes, so please reach consensus with the maintainers before rushing to implement it, and make sure you follow the [Contributing Guidelines](https://github.com/hpcaitech/ColossalAI/blob/main/CONTRIBUTING.md).
|
||||||
|
This ensures that you don't waste your time and we don't waste ours reading the large diffs.
|
||||||
|
options:
|
||||||
|
- label: I'd be willing to do some initial work on this proposal myself.
|
||||||
|
|
||||||
|
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: >
|
||||||
|
Thanks for contributing 🎉!
|
|
@ -34,30 +34,24 @@ Colossal-AI为您提供了一系列并行训练组件。我们的目标是让您
|
||||||
## 样例
|
## 样例
|
||||||
### ViT
|
### ViT
|
||||||
|
|
||||||
<img src="./docs/images/update/vit.png" width="450" />
|
<img src="./docs/images/ViT.png" width="450" />
|
||||||
|
|
||||||
|
- 14倍批大小和5倍训练速度(张量并行=64)
|
||||||
- 14倍批大小
|
|
||||||
- 5倍训练速度
|
|
||||||
|
|
||||||
### GPT-3
|
### GPT-3
|
||||||
<img src="./docs/images/allinone/GPT3_allin1.png" width=700/>
|
<img src="./docs/images/GPT3.png" width=700/>
|
||||||
|
|
||||||
|
- 释放 50% GPU 资源占用, 或 10.7% 加速
|
||||||
|
|
||||||
- GPT-3:释放 50% GPU 资源占用, 或 10.7% 加速
|
|
||||||
|
|
||||||
### GPT-2
|
### GPT-2
|
||||||
<img src="./docs/images/allinone/GPT2_allin1.png" width=800/>
|
<img src="./docs/images/GPT2.png" width=800/>
|
||||||
|
|
||||||
- GPT-2:降低11倍GPU显存占用,或超线性扩展
|
- 降低11倍GPU显存占用,或超线性扩展
|
||||||
|
|
||||||
### BERT
|
### BERT
|
||||||
<img src="./docs/images/allinone/BERT_allin1.png" width=800/>
|
<img src="./docs/images/BERT.png" width=800/>
|
||||||
|
|
||||||
|
- 2倍训练速度,或1.5倍序列长度
|
||||||
- 2倍训练速度
|
|
||||||
- 1.5倍序列长度
|
|
||||||
|
|
||||||
请访问我们的[文档和教程](https://www.colossalai.org/)以了解详情。
|
请访问我们的[文档和教程](https://www.colossalai.org/)以了解详情。
|
||||||
|
|
||||||
|
@ -201,6 +195,13 @@ class MLP_2D(nn.Module):
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## 社区
|
||||||
|
欢迎通过[论坛](https://github.com/hpcaitech/ColossalAI/discussions)
|
||||||
|
[Slack](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w)
|
||||||
|
或 [微信](./docs/images/WeChat.png "qrcode")
|
||||||
|
加入Colossal-AI社区,与我们分享你的建议和问题。
|
||||||
|
|
||||||
|
|
||||||
## 引用
|
## 引用
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
27
README.md
|
@ -37,30 +37,25 @@ distributed training in a few lines.
|
||||||
## Examples
|
## Examples
|
||||||
### ViT
|
### ViT
|
||||||
|
|
||||||
<img src="./docs/images/update/vit.png" width="450" />
|
<img src="./docs/images/ViT.png" width="450" />
|
||||||
|
|
||||||
- 14x larger batch size
|
- 14x larger batch size, and 5x faster training for Tensor Parallel = 64
|
||||||
- 5x faster training
|
|
||||||
|
|
||||||
### GPT-3
|
### GPT-3
|
||||||
|
|
||||||
<img src="./docs/images/allinone/GPT3_allin1.png" width=700/>
|
<img src="./docs/images/GPT3.png" width=700/>
|
||||||
|
|
||||||
- Free 50% GPU resources, or 10.7% acceleration for GPT-3
|
- Free 50% GPU resources, or 10.7% acceleration
|
||||||
|
|
||||||
### GPT-2
|
### GPT-2
|
||||||
<img src="./docs/images/allinone/GPT2_allin1.png" width=800/>
|
<img src="./docs/images/GPT2.png" width=800/>
|
||||||
|
|
||||||
- 11x lower GPU RAM, or superlinear scaling for GPT-2
|
|
||||||
|
|
||||||
|
|
||||||
|
- 11x lower GPU RAM, or superlinear scaling
|
||||||
|
|
||||||
### BERT
|
### BERT
|
||||||
<img src="./docs/images/allinone/BERT_allin1.png" width=800/>
|
<img src="./docs/images/BERT.png" width=800/>
|
||||||
|
|
||||||
|
- 2x faster training, or 50% longer sequence length
|
||||||
- 2x faster training
|
|
||||||
- 50% longer sequence length
|
|
||||||
|
|
||||||
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
|
Please visit our [documentation and tutorials](https://www.colossalai.org/) for more details.
|
||||||
|
|
||||||
|
@ -206,6 +201,12 @@ class MLP_2D(nn.Module):
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Community
|
||||||
|
|
||||||
|
Join the Colossal-AI community on [Forum](https://github.com/hpcaitech/ColossalAI/discussions),
|
||||||
|
[Slack](https://join.slack.com/t/colossalaiworkspace/shared_invite/zt-z7b26eeb-CBp7jouvu~r0~lcFzX832w),
|
||||||
|
and [WeChat](./docs/images/WeChat.png "qrcode") to share your suggestions, advice, and questions with our engineering team.
|
||||||
|
|
||||||
|
|
||||||
## Cite Us
|
## Cite Us
|
||||||
|
|
||||||
|
|
Before Width: | Height: | Size: 1.1 MiB After Width: | Height: | Size: 1.1 MiB |
Before Width: | Height: | Size: 329 KiB |
Before Width: | Height: | Size: 603 KiB After Width: | Height: | Size: 603 KiB |
Before Width: | Height: | Size: 1.3 MiB After Width: | Height: | Size: 1.3 MiB |
Before Width: | Height: | Size: 264 KiB |
Before Width: | Height: | Size: 415 KiB |
Before Width: | Height: | Size: 545 KiB After Width: | Height: | Size: 545 KiB |
Before Width: | Height: | Size: 338 KiB |
After Width: | Height: | Size: 68 KiB |