Jiarui Fang
060b917daf
[refactor] remove gpc dependency in colotensor's _ops ( #1189 )
2 years ago
Frank Lee
abf6a262dc
[fx] added module patch for pooling layers ( #1197 )
2 years ago
github-actions[bot]
23442a5bc1
Automated submodule synchronization ( #1194 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
YuliangLiu0306
63d2a93878
[context]support arbitary module materialization. ( #1193 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [context]support arbitary module materialization.
* [test]add numerical check for lazy init context.
2 years ago
Jiarui Fang
a444633d13
warmup ratio configration ( #1192 )
2 years ago
ver217
dba7e0cfb4
make AutoPlacementPolicy configurable ( #1191 )
2 years ago
YuliangLiu0306
2053e138a2
[context]use meta tensor to init model lazily. ( #1187 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [context]use meta tensor to init model lazily.
* polish
* make module with device kwargs bypass the normal init.
* change unit test to adapt updated context.
2 years ago
Frank Lee
2c8c05675d
[fx] patched conv and normalization ( #1188 )
2 years ago
github-actions[bot]
6f0733a1ef
Automated submodule synchronization ( #1159 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Frank Lee
6d86f1bc91
[fx] supported data-dependent control flow in model tracing ( #1185 )
...
* [fx] supported data-dependent control flow in model tracing
* polish code
2 years ago
Jiarui Fang
c463f8adf9
[tensor] remove gpc in tensor tests ( #1186 )
2 years ago
Jiarui Fang
372f791444
[refactor] move chunk and chunkmgr to directory gemini ( #1182 )
2 years ago
ver217
6b2f2ab9bb
[ddp] ColoDDP uses bucket all-reduce ( #1177 )
...
* add reducer
* update colo ddp with reducer
* polish unit test
* polish unit test
2 years ago
Jiarui Fang
7487215b95
[ColoTensor] add independent process group ( #1179 )
2 years ago
YuliangLiu0306
26ba87272d
[hotfix]fixed p2p process send stuck ( #1181 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [hotfix]fixed p2p process send stuck
2 years ago
Jiarui Fang
1b657f9ce1
[tensor] revert local view back ( #1178 )
2 years ago
Jiarui Fang
0dd4e2bbfb
[Tensor] rename some APIs in TensorSpec and Polish view unittest ( #1176 )
2 years ago
Ziyue Jiang
dd0420909f
[Tensor] rename parallel_action ( #1174 )
...
* rename parallel_action
* polish
2 years ago
YuliangLiu0306
e27645376d
[hotfix]different overflow status lead to communication stuck. ( #1175 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [hotfix]fix some bugs caused by refactored schedule.
* [hotfix]different overflow statu llead to communication stuck.
2 years ago
Jiarui Fang
aa7bef73d4
[Tensor] distributed view supports inter-process hybrid parallel ( #1169 )
2 years ago
ver217
9e1daa63d2
[zero] sharded optim supports loading local state dict ( #1170 )
...
* sharded optim supports loading local state dict
* polish code
* add unit test
2 years ago
ver217
561e90493f
[zero] zero optim supports loading local state dict ( #1171 )
...
* zero optim supports loading local state dict
* polish code
* add unit test
2 years ago
Jiarui Fang
4b9bba8116
[ColoTensor] rename APIs and add output_replicate to ComputeSpec ( #1168 )
2 years ago
Jiarui Fang
f4ef224358
[Tensor] remove ParallelAction, use ComputeSpec instread ( #1166 )
2 years ago
Jiarui Fang
177c374401
remove gather out in parallel action ( #1163 )
2 years ago
Frank Lee
51f1ec96b0
[workflow] polish readme and dockerfile ( #1165 )
...
* [workflow] polish readme and dockerfile
* polish
2 years ago
Frank Lee
ca73028a3a
[workflow] auto-publish docker image upon release ( #1164 )
2 years ago
ver217
634eecb98e
mark sanity_check of dist_spec_mgr as staticmethod ( #1161 )
2 years ago
Ziyue Jiang
955ac912de
remove log ( #1160 )
2 years ago
ver217
4e67b2a890
fix chunk move device ( #1158 )
2 years ago
Jiarui Fang
07f9c781f9
[graph] improve the graph building. ( #1157 )
2 years ago
ver217
22717a856f
[tensor] add embedding bag op ( #1156 )
2 years ago
ver217
ae86151968
[tensor] add more element-wise ops ( #1155 )
...
* add more element-wise ops
* update test_op
* polish unit test
2 years ago
github-actions[bot]
e8c34eedfd
Automated submodule synchronization ( #1129 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Frank Lee
d415d73286
[workflow] fixed release post workflow ( #1154 )
2 years ago
ver217
54aabb8da4
[gemini] refactor gemini mgr ( #1151 )
...
* refactor gemini mgr
* udpate __init__
2 years ago
Frank Lee
f8eec98ff5
[tensor] fixed non-serializable colo parameter during model checkpointing ( #1153 )
2 years ago
ver217
ffa025e120
[tensor] dist spec s2s uses all-to-all ( #1136 )
...
* dist spec s2s uses all-to-all
* update unit test
* add sanity check
* polish unitest test with titans
* add sanity check for DistMgr
* add sanity check
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>
2 years ago
Frank Lee
c77da0dc81
[workflow] fixed format error in yaml file ( #1145 )
2 years ago
Jiarui Fang
ff644ee5e4
polish unitest test with titans ( #1152 )
2 years ago
YuliangLiu0306
f1f51990b9
[hotfix]fix some bugs caused by refactored schedule. ( #1148 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [hotfix]fix some bugs caused by refactored schedule.
2 years ago
Jiarui Fang
8cdce0399c
[ColoTensor] improves init functions. ( #1150 )
2 years ago
ver217
8106d7b8c7
[ddp] refactor ColoDDP and ZeroDDP ( #1146 )
...
* ColoDDP supports overwriting default process group
* rename ColoDDPV2 to ZeroDDP
* add docstr for ZeroDDP
* polish docstr
2 years ago
Frank Lee
0e4e62d30d
[tensor] added __repr__ to spec ( #1147 )
2 years ago
YuliangLiu0306
70dd88e2ee
[pipeline]add customized policy ( #1139 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [pipeline]add customized policy
2 years ago
Frank Lee
d1918304bb
[workflow] added workflow to auto draft the release post ( #1144 )
2 years ago
YuliangLiu0306
18091581c0
[pipeline]support more flexible pipeline ( #1138 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [pipeline]support more flexible pipeline
2 years ago
ver217
ccf3c58c89
embedding op use gather_out ( #1143 )
2 years ago
Frank Lee
e61dc31b05
[ci] added scripts to auto-generate release post text ( #1142 )
...
* [ci] added scripts to auto-generate release post text
* polish code
2 years ago
ver217
6690a61b4d
[hotfix] prevent nested ZeRO ( #1140 )
2 years ago