ColossalAI/colossalai/nn/layer/moe/__init__.py

from .experts import Experts, FFNExperts, TPExperts
from .layers import MoeLayer, Top1Router, Top2Router
from .utils import NormalNoiseGenerator, build_ffn_experts

__all__ = [
    'Experts', 'FFNExperts', 'TPExperts', 'Top1Router', 'Top2Router', 'MoeLayer', 'NormalNoiseGenerator',
    'build_ffn_experts'
]
Added TPExpert for special situation 2022-02-27 14:28:39 +00:00			`from .experts import Experts, FFNExperts, TPExperts`
Optimized MoE layer and fixed some bugs; Decreased moe tests; Added FFNExperts and ViTMoE model 2022-02-18 12:42:31 +00:00			`from .layers import MoeLayer, Top1Router, Top2Router`
Added TPExpert for special situation 2022-02-27 14:28:39 +00:00			`from .utils import NormalNoiseGenerator, build_ffn_experts`
Added MoE parallel (#127) 2022-01-07 07:08:36 +00:00
Added TPExpert for special situation 2022-02-27 14:28:39 +00:00			`__all__ = [`
			`'Experts', 'FFNExperts', 'TPExperts', 'Top1Router', 'Top2Router', 'MoeLayer', 'NormalNoiseGenerator',`
			`'build_ffn_experts'`
			`]`