ColossalAI/colossalai/kernel/__init__.py

from .cuda_native import FusedScaleMaskSoftmax, LayerNorm, MultiHeadAttention

__all__ = [
    "LayerNorm",
    "FusedScaleMaskSoftmax",
    "MultiHeadAttention",
]
[builder] unified cpu_optim fused_optim inferface (#2190) 2022-12-23 12:57:41 +00:00			`from .cuda_native import FusedScaleMaskSoftmax, LayerNorm, MultiHeadAttention`
add colossalai kernel module (#55) 2021-12-21 04:19:52 +00:00
[builder] multihead attn runtime building (#2203) * [hotfix] correcnt cpu_optim runtime compilation * [builder] multihead attn * fix bug * fix a bug 2022-12-27 08:06:09 +00:00			`__all__ = [`
[builder] MOE builder (#2277) 2023-01-03 12:29:39 +00:00			`"LayerNorm",`
			`"FusedScaleMaskSoftmax",`
			`"MultiHeadAttention",`
[builder] multihead attn runtime building (#2203) * [hotfix] correcnt cpu_optim runtime compilation * [builder] multihead attn * fix bug * fix a bug 2022-12-27 08:06:09 +00:00			`]`