ColossalAI/examples/language/roberta/configs/colossalai_ddp.py

from colossalai.nn.optimizer import FusedAdam

try:
    from colossalai.zero.shard_utils import TensorShardStrategy
except ImportError:
    # colossalai > 0.2.8
    from colossalai.zero.legacy import TensorShardStrategy

clip_grad_norm = 1.0
add RoBERTa (#1980) * update roberta * update roberta & readme * update roberta & readme * update roberta & readme 2022-11-18 06:04:49 +00:00			`from colossalai.nn.optimizer import FusedAdam`

[example] update examples related to zero/gemini (#3431) * [zero] update legacy import * [zero] update examples * [example] fix opt tutorial * [example] fix opt tutorial * [example] fix opt tutorial * [example] fix opt tutorial * [example] fix import 2023-04-04 09:32:51 +00:00			`try:`
			`from colossalai.zero.shard_utils import TensorShardStrategy`
			`except ImportError:`
			`# colossalai > 0.2.8`
			`from colossalai.zero.legacy import TensorShardStrategy`

add RoBERTa (#1980) * update roberta * update roberta & readme * update roberta & readme * update roberta & readme 2022-11-18 06:04:49 +00:00			`clip_grad_norm = 1.0`