ColossalAI/colossalai/nn/layer/vanilla/__init__.py

from .layers import (
    DropPath,
    VanillaClassifier,
    VanillaLayerNorm,
    VanillaLinear,
    VanillaPatchEmbedding,
    WrappedDropout,
    WrappedDropPath,
)

__all__ = [
    "VanillaLayerNorm", "VanillaPatchEmbedding", "VanillaClassifier", "DropPath", "WrappedDropout", "WrappedDropPath",
    "VanillaLinear"
]
added skip_bias_add for non-tp linear 2022-11-09 05:20:02 +00:00			`from .layers import (`
			`DropPath,`
			`VanillaClassifier,`
			`VanillaLayerNorm,`
			`VanillaLinear,`
			`VanillaPatchEmbedding,`
			`WrappedDropout,`
			`WrappedDropPath,`
			`)`
Layer integration (#83) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> 2021-12-27 07:04:32 +00:00
[TP] allow layernorm without bias (#750) 2022-04-14 03:43:56 +00:00			`__all__ = [`
added skip_bias_add for non-tp linear 2022-11-09 05:20:02 +00:00			`"VanillaLayerNorm", "VanillaPatchEmbedding", "VanillaClassifier", "DropPath", "WrappedDropout", "WrappedDropPath",`
			`"VanillaLinear"`
[TP] allow layernorm without bias (#750) 2022-04-14 03:43:56 +00:00			`]`