ColossalAI/colossalai/builder/__init__.py

from .builder import build_from_config, build_from_registry, build_gradient_handler

__all__ = [
    'build_gradient_handler', 'build_from_config', 'build_from_registry'
]
[pipeline] refactor the pipeline module (#1087) * [pipeline] refactor the pipeline module * polish code 2022-06-10 03:27:38 +00:00			`from .builder import build_from_config, build_from_registry, build_gradient_handler`
Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> 2021-11-18 11:45:06 +00:00
			`__all__ = [`
[pipeline] refactor the pipeline module (#1087) * [pipeline] refactor the pipeline module * polish code 2022-06-10 03:27:38 +00:00			`'build_gradient_handler', 'build_from_config', 'build_from_registry'`
Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> 2021-11-18 11:45:06 +00:00			`]`