ColossalAI/colossalai/legacy/nn/layer/vanilla/__init__.py

from .layers import (
    DropPath,
    VanillaClassifier,
    VanillaLayerNorm,
    VanillaLinear,
    VanillaPatchEmbedding,
    WrappedDropout,
    WrappedDropPath,
)

__all__ = [
    "VanillaLayerNorm",
    "VanillaPatchEmbedding",
    "VanillaClassifier",
    "DropPath",
    "WrappedDropout",
    "WrappedDropPath",
    "VanillaLinear",
]
added skip_bias_add for non-tp linear 2022-11-09 05:20:02 +00:00			`from .layers import (`
			`DropPath,`
			`VanillaClassifier,`
			`VanillaLayerNorm,`
			`VanillaLinear,`
			`VanillaPatchEmbedding,`
			`WrappedDropout,`
			`WrappedDropPath,`
			`)`
Layer integration (#83) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> 2021-12-27 07:04:32 +00:00
[TP] allow layernorm without bias (#750) 2022-04-14 03:43:56 +00:00			`__all__ = [`
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format 2023-09-19 06:20:26 +00:00			`"VanillaLayerNorm",`
			`"VanillaPatchEmbedding",`
			`"VanillaClassifier",`
			`"DropPath",`
			`"WrappedDropout",`
			`"WrappedDropPath",`
			`"VanillaLinear",`
[TP] allow layernorm without bias (#750) 2022-04-14 03:43:56 +00:00			`]`