github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/nn/lr_scheduler/__init__.py

24 lines
673 B
Python
Raw Normal View History

Migrated project
2021-10-28 16:21:23 +00:00
from .cosine import CosineAnnealingLR, CosineAnnealingWarmupLR, FlatAnnealingLR, FlatAnnealingWarmupLR
Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com>
2021-11-18 11:45:06 +00:00
from .linear import LinearWarmupLR
Migrated project
2021-10-28 16:21:23 +00:00
from .multistep import MultiStepLR, MultiStepWarmupLR
from .onecycle import OneCycleLR
from .poly import PolynomialLR, PolynomialWarmupLR
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
from .torch import ExponentialLR, LambdaLR, MultiplicativeLR, StepLR
Migrated project
2021-10-28 16:21:23 +00:00
__all__ = [
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
"CosineAnnealingLR",
"CosineAnnealingWarmupLR",
"FlatAnnealingLR",
"FlatAnnealingWarmupLR",
"LinearWarmupLR",
"MultiStepLR",
"MultiStepWarmupLR",
"OneCycleLR",
"PolynomialLR",
"PolynomialWarmupLR",
"LambdaLR",
"MultiplicativeLR",
"StepLR",
"ExponentialLR",
Migrated project
2021-10-28 16:21:23 +00:00
]