github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/booster/plugin/__init__.py

16 lines
529 B
Python
Raw Normal View History

[booster] implement Gemini plugin (#3352) * [booster] add gemini plugin * [booster] update docstr * [booster] gemini plugin add coloparam convertor * [booster] fix coloparam convertor * [booster] fix gemini plugin device * [booster] add gemini plugin test * [booster] gemini plugin ignore sync bn * [booster] skip some model * [booster] skip some model * [booster] modify test world size * [booster] modify test world size * [booster] skip test
2023-03-31 08:06:13 +00:00
from .gemini_plugin import GeminiPlugin
[plugin] add 3d parallel plugin (#4295) * [amp] add mixed precision optimizer * [plugin] add 3d parallel plugin * [booster] support pipeline * [plugin] 3d parallel plugin support clip grad norm * [shardformer] fix sharder and add plugin test * [plugin] rename 3d parallel plugin * [ci] support testmon core pkg change detection (#4305) * [hotfix] debug testmon * [hotfix] fix llama * [hotfix] fix p2p bugs * [hotfix] fix requirements
2023-07-25 16:53:57 +00:00
from .hybrid_parallel_plugin import HybridParallelPlugin
[booster] add low level zero plugin (#3594) * [booster] add low level zero plugin * [booster] fix gemini plugin test * [booster] fix precision * [booster] add low level zero plugin test * [test] fix booster plugin test oom * [test] fix booster plugin test oom * [test] fix googlenet and inception output trans * [test] fix diffuser clip vision model * [test] fix torchaudio_wav2vec2_base * [test] fix low level zero plugin test
2023-04-26 06:37:25 +00:00
from .low_level_zero_plugin import LowLevelZeroPlugin
[booster] added the plugin base and torch ddp plugin (#3180) * [booster] added the plugin base and torch ddp plugin * polish code * polish code * polish code
2023-03-21 09:39:30 +00:00
from .plugin_base import Plugin
from .torch_ddp_plugin import TorchDDPPlugin
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
__all__ = ["Plugin", "TorchDDPPlugin", "GeminiPlugin", "LowLevelZeroPlugin", "HybridParallelPlugin"]
[booster] support torch fsdp plugin in booster (#3697) Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>
2023-05-15 04:14:38 +00:00
import torch
from packaging import version
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
if version.parse(torch.__version__) >= version.parse("1.12.0"):
[booster] support torch fsdp plugin in booster (#3697) Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>
2023-05-15 04:14:38 +00:00
from .torch_fsdp_plugin import TorchFSDPPlugin
[misc] update pre-commit and run all files (#4752) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format
2023-09-19 06:20:26 +00:00
__all__.append("TorchFSDPPlugin")