ColossalAI/colossalai
eric8607242 c3d5fa3bac
[shardformer] Support customized policy for llamav2 based model with HybridParallelPlugin (#4624)
* Enable policy assignment in HybridPlugin and enable llama policy for llamav2

* Remove Policy from Plugin

* revert changes of plugin

HybridParallelModule

* revert changes in plugin

* upgrade transformers

* revert transformers version

---------

Co-authored-by: flybird11111 <1829166702@qq.com>
2023-09-07 10:15:13 +08:00
..
_C [setup] support pre-build and jit-build of cuda kernels (#2374) 2023-01-06 20:50:26 +08:00
_analyzer [example] add train resnet/vit with booster example (#3694) 2023-05-08 10:42:30 +08:00
amp [pipeline] support fp32 for HybridPlugin/merge shardformer test and pipeline test into one file (#4354) 2023-08-15 23:25:14 +08:00
auto_parallel [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00
autochunk fix typo colossalai/auto_parallel autochunk fx/passes etc. (#3808) 2023-05-24 09:01:50 +08:00
booster Merge branch 'main' into feature/shardformer 2023-09-05 21:54:08 +08:00
checkpoint_io Merge branch 'main' into feature/shardformer 2023-09-04 23:43:13 +08:00
cli [example] add llama2 example (#4527) 2023-08-28 17:59:11 +08:00
cluster [shardformer] support interleaved pipeline (#4448) 2023-08-16 19:29:03 +08:00
communication [NFC] fix: format (#4270) 2023-07-26 14:12:57 +08:00
context [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00
device [format] applied code formatting on changed files in pull request 4152 (#4157) 2023-07-04 16:07:47 +08:00
fx [nfc] fix typo colossalai/cli fx kernel (#3847) 2023-06-02 15:02:45 +08:00
interface [zero] hotfix master param sync (#4618) 2023-09-05 15:04:02 +08:00
kernel [example] add llama2 example (#4527) 2023-08-28 17:59:11 +08:00
lazy [shardformer] support lazy init (#4202) 2023-08-15 23:25:14 +08:00
legacy [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00
logging [logger] hotfix, missing _FORMAT (#2231) 2022-12-29 22:59:39 +08:00
nn [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00
pipeline [shardformer] update bert finetune example with HybridParallelPlugin (#4584) 2023-09-04 21:46:29 +08:00
shardformer [shardformer] Support customized policy for llamav2 based model with HybridParallelPlugin (#4624) 2023-09-07 10:15:13 +08:00
tensor [gemini] improve compatibility and add static placement policy (#4479) 2023-08-24 09:29:25 +08:00
testing Next commit [checkpointio] Unsharded Optimizer Checkpoint for Gemini Plugin (#4141) 2023-07-07 16:33:06 +08:00
utils [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00
zero Merge branch 'main' into feature/shardformer 2023-09-05 21:54:08 +08:00
__init__.py [setup] supported conda-installed torch (#2048) 2022-11-30 16:45:15 +08:00
constants.py updated tp layers 2022-11-02 12:19:38 +08:00
core.py [Tensor] distributed view supports inter-process hybrid parallel (#1169) 2022-06-27 09:45:26 +08:00
global_variables.py [NFC] polish colossalai/global_variables.py code style (#3259) 2023-03-29 15:22:21 +08:00
initialize.py [legacy] move builder and registry to legacy (#4603) 2023-09-05 21:53:10 +08:00