github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/shardformer/layer/__init__.py

24 lines
766 B
Python
Raw Normal View History

[shardformer] supported bloom model (#4098)
2023-06-28 07:04:35 +00:00
from .dropout import DropoutForParallelInput, DropoutForReplicatedInput
[shardformer] refactored the shardformer layer structure (#4053)
2023-06-21 06:30:06 +00:00
from .embedding import Embedding1D, VocabParallelEmbedding1D
from .linear import Linear1D_Col, Linear1D_Row
from .loss import cross_entropy_1d
[shardformer] supported fused normalization (#4112)
2023-06-30 01:32:37 +00:00
from .normalization import FusedLayerNorm, FusedRMSNorm
[shardformer] fix import
2023-08-15 10:56:16 +00:00
from .parallel_module import ParallelModule
[shardformer] support SAM (#4231) * 1.support sam 2.add fused qkv for nn.Linear * update utils support set element in list * overtwrite SamVisionAttention foward to use DropoutForParallelInput * remove unused code
2023-07-14 07:56:59 +00:00
from .qkv_fused_linear import FusedLinear1D_Col, GPT2FusedLinearConv1D_Col, GPT2FusedLinearConv1D_Row
[shardformer] add gpt2 test and layer class refactor (#4041) * add gpt2 test and layer class refactor * add dropout in gpt2 policy
2023-06-20 03:45:16 +00:00
__all__ = [
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
"Embedding1D",
"VocabParallelEmbedding1D",
"Linear1D_Col",
"Linear1D_Row",
"GPT2FusedLinearConv1D_Col",
"GPT2FusedLinearConv1D_Row",
"DropoutForParallelInput",
"DropoutForReplicatedInput",
"cross_entropy_1d",
"FusedLayerNorm",
"FusedRMSNorm",
"FusedLinear1D_Col",
"ParallelModule",
[shardformer] add gpt2 test and layer class refactor (#4041) * add gpt2 test and layer class refactor * add dropout in gpt2 policy
2023-06-20 03:45:16 +00:00
]