Jiarui Fang
52736205d9
[checkpoint] make unitest faster ( #1217 )
2 years ago
Jiarui Fang
f38006ea83
[checkpoint] checkpoint for ColoTensor Model ( #1196 )
2 years ago
XYE
291e22aac6
[fx] temporarily used ( #1215 )
2 years ago
Jiarui Fang
ae7d3f4927
[refactor] move process group from _DistSpec to ColoTensor. ( #1203 )
2 years ago
Frank Lee
5da87ce35d
[fx] added testing for all albert variants ( #1211 )
2 years ago
Frank Lee
2d13a45a3b
[fx] added testing for all gpt variants ( #1210 )
...
* [fx] added testing for all gpt variants
* polish code
* polish code
2 years ago
YuliangLiu0306
189946c5c4
[fx]add uniform policy ( #1208 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [fx]add uniform policy
2 years ago
Frank Lee
426a279ce7
[fx] added testing for all bert variants ( #1207 )
...
* [fx] added testing for all bert variants
* polish code
2 years ago
Jiarui Fang
b5f25eb32a
[Tensor] add cpu group to ddp ( #1200 )
2 years ago
Frank Lee
f7878f465c
[fx] supported model tracing for huggingface bert ( #1201 )
...
* [fx] supported model tracing for huggingface bert
* polish test
2 years ago
Jiarui Fang
060b917daf
[refactor] remove gpc dependency in colotensor's _ops ( #1189 )
2 years ago
Frank Lee
abf6a262dc
[fx] added module patch for pooling layers ( #1197 )
2 years ago
github-actions[bot]
23442a5bc1
Automated submodule synchronization ( #1194 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
YuliangLiu0306
63d2a93878
[context]support arbitary module materialization. ( #1193 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [context]support arbitary module materialization.
* [test]add numerical check for lazy init context.
2 years ago
Jiarui Fang
a444633d13
warmup ratio configration ( #1192 )
2 years ago
ver217
dba7e0cfb4
make AutoPlacementPolicy configurable ( #1191 )
2 years ago
YuliangLiu0306
2053e138a2
[context]use meta tensor to init model lazily. ( #1187 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [context]use meta tensor to init model lazily.
* polish
* make module with device kwargs bypass the normal init.
* change unit test to adapt updated context.
2 years ago
Frank Lee
2c8c05675d
[fx] patched conv and normalization ( #1188 )
2 years ago
github-actions[bot]
6f0733a1ef
Automated submodule synchronization ( #1159 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Frank Lee
6d86f1bc91
[fx] supported data-dependent control flow in model tracing ( #1185 )
...
* [fx] supported data-dependent control flow in model tracing
* polish code
2 years ago
Jiarui Fang
c463f8adf9
[tensor] remove gpc in tensor tests ( #1186 )
2 years ago
Jiarui Fang
372f791444
[refactor] move chunk and chunkmgr to directory gemini ( #1182 )
2 years ago
ver217
6b2f2ab9bb
[ddp] ColoDDP uses bucket all-reduce ( #1177 )
...
* add reducer
* update colo ddp with reducer
* polish unit test
* polish unit test
2 years ago
Jiarui Fang
7487215b95
[ColoTensor] add independent process group ( #1179 )
2 years ago
YuliangLiu0306
26ba87272d
[hotfix]fixed p2p process send stuck ( #1181 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [hotfix]fixed p2p process send stuck
2 years ago
Jiarui Fang
1b657f9ce1
[tensor] revert local view back ( #1178 )
2 years ago
Jiarui Fang
0dd4e2bbfb
[Tensor] rename some APIs in TensorSpec and Polish view unittest ( #1176 )
2 years ago
Ziyue Jiang
dd0420909f
[Tensor] rename parallel_action ( #1174 )
...
* rename parallel_action
* polish
2 years ago
YuliangLiu0306
e27645376d
[hotfix]different overflow status lead to communication stuck. ( #1175 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [hotfix]fix some bugs caused by refactored schedule.
* [hotfix]different overflow statu llead to communication stuck.
2 years ago
Jiarui Fang
aa7bef73d4
[Tensor] distributed view supports inter-process hybrid parallel ( #1169 )
2 years ago
ver217
9e1daa63d2
[zero] sharded optim supports loading local state dict ( #1170 )
...
* sharded optim supports loading local state dict
* polish code
* add unit test
2 years ago
ver217
561e90493f
[zero] zero optim supports loading local state dict ( #1171 )
...
* zero optim supports loading local state dict
* polish code
* add unit test
2 years ago
Jiarui Fang
4b9bba8116
[ColoTensor] rename APIs and add output_replicate to ComputeSpec ( #1168 )
2 years ago
Jiarui Fang
f4ef224358
[Tensor] remove ParallelAction, use ComputeSpec instread ( #1166 )
2 years ago
Jiarui Fang
177c374401
remove gather out in parallel action ( #1163 )
2 years ago
Frank Lee
51f1ec96b0
[workflow] polish readme and dockerfile ( #1165 )
...
* [workflow] polish readme and dockerfile
* polish
2 years ago
Frank Lee
ca73028a3a
[workflow] auto-publish docker image upon release ( #1164 )
2 years ago
ver217
634eecb98e
mark sanity_check of dist_spec_mgr as staticmethod ( #1161 )
2 years ago
Ziyue Jiang
955ac912de
remove log ( #1160 )
2 years ago
ver217
4e67b2a890
fix chunk move device ( #1158 )
2 years ago
Jiarui Fang
07f9c781f9
[graph] improve the graph building. ( #1157 )
2 years ago
ver217
22717a856f
[tensor] add embedding bag op ( #1156 )
2 years ago
ver217
ae86151968
[tensor] add more element-wise ops ( #1155 )
...
* add more element-wise ops
* update test_op
* polish unit test
2 years ago
github-actions[bot]
e8c34eedfd
Automated submodule synchronization ( #1129 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Frank Lee
d415d73286
[workflow] fixed release post workflow ( #1154 )
2 years ago
ver217
54aabb8da4
[gemini] refactor gemini mgr ( #1151 )
...
* refactor gemini mgr
* udpate __init__
2 years ago
Frank Lee
f8eec98ff5
[tensor] fixed non-serializable colo parameter during model checkpointing ( #1153 )
2 years ago
ver217
ffa025e120
[tensor] dist spec s2s uses all-to-all ( #1136 )
...
* dist spec s2s uses all-to-all
* update unit test
* add sanity check
* polish unitest test with titans
* add sanity check for DistMgr
* add sanity check
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>
2 years ago
Frank Lee
c77da0dc81
[workflow] fixed format error in yaml file ( #1145 )
2 years ago
Jiarui Fang
ff644ee5e4
polish unitest test with titans ( #1152 )
2 years ago