ColossalAI/colossalai/nn/layer/parallel_2d/__init__.py

from ._operation import reduce_by_batch_2d, split_batch_2d
from .layers import (Classifier2D, Embedding2D, LayerNorm2D, Linear2D, PatchEmbedding2D, VocabParallelClassifier2D,
                     VocabParallelEmbedding2D)

__all__ = [
    'split_batch_2d', 'reduce_by_batch_2d', 'Linear2D', 'LayerNorm2D', 'Classifier2D', 'PatchEmbedding2D',
    'Embedding2D', 'VocabParallelEmbedding2D', 'VocabParallelClassifier2D'
]
[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622) 2022-04-02 08:12:04 +00:00			`from ._operation import reduce_by_batch_2d, split_batch_2d`
moved env variables to global variables; (#215) added branch context; added vocab parallel layers; moved split_batch from load_batch to tensor parallel embedding layers; updated gpt model; updated unit test cases; fixed few collective communicator bugs 2022-02-14 03:15:02 +00:00			`from .layers import (Classifier2D, Embedding2D, LayerNorm2D, Linear2D, PatchEmbedding2D, VocabParallelClassifier2D,`
			`VocabParallelEmbedding2D)`
Migrated project 2021-10-28 16:21:23 +00:00
			`__all__ = [`
[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622) 2022-04-02 08:12:04 +00:00			`'split_batch_2d', 'reduce_by_batch_2d', 'Linear2D', 'LayerNorm2D', 'Classifier2D', 'PatchEmbedding2D',`
moved env variables to global variables; (#215) added branch context; added vocab parallel layers; moved split_batch from load_batch to tensor parallel embedding layers; updated gpt model; updated unit test cases; fixed few collective communicator bugs 2022-02-14 03:15:02 +00:00			`'Embedding2D', 'VocabParallelEmbedding2D', 'VocabParallelClassifier2D'`
Migrated project 2021-10-28 16:21:23 +00:00			`]`