Super Daniel
be229217ce
[fx] add torchaudio test ( #1369 )
...
* [fx]add torchaudio test
* [fx]add torchaudio test
* [fx] add torchaudio test
* [fx] add torchaudio test
* [fx] add torchaudio test
* [fx] add torchaudio test
* [fx] add torchaudio test
* [fx] add torchaudio test and test patches
* Delete ~
* [fx] add patches and patches test
* [fx] add patches and patches test
* [fx] fix patches
* [fx] fix rnn patches
* [fx] fix rnn patches
* [fx] fix rnn patches
* [fx] fix rnn patches
* [fx] merge upstream
* [fx] fix import errors
2 years ago
github-actions[bot]
fb6f085907
Automated submodule synchronization ( #1372 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Boyuan Yao
bb640ec728
[fx] Add colotracer compatibility test on torchrec ( #1370 )
2 years ago
ver217
c415240db6
[nvme] CPUAdam and HybridAdam support NVMe offload ( #1360 )
...
* impl nvme optimizer
* update cpu adam
* add unit test
* update hybrid adam
* update docstr
* add TODOs
* update CI
* fix CI
* fix CI
* fix CI path
* fix CI path
* fix CI path
* fix install tensornvme
* fix CI
* fix CI path
* fix CI env variables
* test CI
* test CI
* fix CI
* fix nvme optim __del__
* fix adam __del__
* fix nvme optim
* fix CI env variables
* fix nvme optim import
* test CI
* test CI
* fix CI
2 years ago
HELSON
8463290642
[checkpoint] use args, kwargs in save_checkpoint, load_checkpoint ( #1368 )
2 years ago
github-actions[bot]
c491c2a948
Automated submodule synchronization ( #1364 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
YuliangLiu0306
5542816690
[fx]add gpt2 passes for pipeline performance test ( #1366 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [fx]add gpt2 passes for pipeline performance test
2 years ago
HELSON
87775a0682
[colotensor] use cpu memory to store state_dict ( #1367 )
2 years ago
HELSON
943a96323e
[hotfix] fix no optimizer in save/load ( #1363 )
2 years ago
Frank Lee
cd063ac37f
[fx] added activation checkpoint codegen support for torch < 1.12 ( #1359 )
2 years ago
HELSON
4417804129
[unit test] add megatron init test in zero_optim ( #1358 )
2 years ago
HELSON
7a065dc9f6
[hotfix] fix megatron_init in test_gpt2.py ( #1357 )
2 years ago
Frank Lee
644582eee9
[fx] added activation checkpoint codegen ( #1355 )
2 years ago
ver217
38fd8844c0
[docker] add tensornvme in docker ( #1354 )
...
* add tensornvme in docker
* fix dockerfile
* fix dockerfile
2 years ago
ver217
6b43c789fd
fix zero optim backward_by_grad and save/load ( #1353 )
2 years ago
ver217
d068af81a3
[doc] update rst and docstring ( #1351 )
...
* update rst
* add zero docstr
* fix docstr
* remove fx.tracer.meta_patch
* fix docstr
* fix docstr
* update fx rst
* fix fx docstr
* remove useless rst
2 years ago
Frank Lee
274c1a3b5f
[fx] fixed apex normalization patch exception ( #1352 )
2 years ago
ver217
ce470ba37e
[checkpoint] sharded optim save/load grad scaler ( #1350 )
2 years ago
Frank Lee
05fae1fd56
[fx] added activation checkpointing annotation ( #1349 )
...
* [fx] added activation checkpointing annotation
* polish code
* polish code
2 years ago
YuliangLiu0306
051592c64e
[fx] update MetaInforProp pass to process more complex node.meta ( #1344 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [fx] update MetaInforProp pass to process more complex node.meta
2 years ago
HELSON
7a8702c06d
[colotensor] add Tensor.view op and its unit test ( #1343 )
...
[colotensor] add megatron initialization for gpt2
2 years ago
github-actions[bot]
6160a1d6a7
Automated submodule synchronization ( #1348 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
binmakeswell
92b0b139eb
[NFC] add OPT ( #1345 )
2 years ago
YuliangLiu0306
942c8cd1fb
[fx] refactor tracer to trace complete graph ( #1342 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [fx] refactor tracer to trace complete graph
* add comments and solve conflicts.
2 years ago
Frank Lee
2cc1175c76
[fx] tested the complete workflow for auto-parallel ( #1336 )
...
* [fx] tested the complete workflow for auto-parallel
* polish code
* polish code
* polish code
2 years ago
YuliangLiu0306
4631fef8a0
[fx]refactor tracer ( #1335 )
2 years ago
HELSON
bf5066fba7
[refactor] refactor ColoTensor's unit tests ( #1340 )
2 years ago
HELSON
f92c100ddd
[checkpoint] use gather_tensor in checkpoint and update its unit test ( #1339 )
2 years ago
Frank Lee
f3ce7b8336
[fx] recovered skipped pipeline tests ( #1338 )
2 years ago
ver217
0c51ff2c13
[hotfix] ZeroDDP use new process group ( #1333 )
...
* process group supports getting ranks in group
* chunk mgr receives a process group
* update unit test
* fix unit tests
2 years ago
Frank Lee
11d1436a67
[workflow] update docker build workflow to use proxy ( #1334 )
2 years ago
Frank Lee
75abc75c15
[fx] fixed compatiblity issue with torch 1.10 ( #1331 )
2 years ago
Frank Lee
069d6fdc84
[workflow] update 8-gpu test to use torch 1.11 ( #1332 )
2 years ago
fastalgo
7857fd7616
Update README.md
2 years ago
Frank Lee
169954f87e
[test] removed outdated unit test for meta context ( #1329 )
2 years ago
ver217
7a05367101
[hotfix] shared model returns cpu state_dict ( #1328 )
2 years ago
Frank Lee
b2475d8c5c
[fx] fixed unit tests for torch 1.12 ( #1327 )
2 years ago
HELSON
d49708ae43
[hotfix] fix ddp for unit test test_gpt2 ( #1326 )
2 years ago
Frank Lee
250be4d31e
[utils] integrated colotensor with lazy init context ( #1324 )
...
* [utils] integrated colotensor with lazy init context
* polish code
* polish code
* polish code
2 years ago
Frank Lee
659a740738
[workflow] roll back to use torch 1.11 for unit testing ( #1325 )
2 years ago
Frank Lee
4d5dbf48a6
[workflow] fixed trigger condition for 8-gpu unit test ( #1323 )
2 years ago
YuliangLiu0306
e8acf55e8b
[fx] add balanced policy v2 ( #1251 )
...
* [CLI] add CLI launcher
* Revert "[CLI] add CLI launcher"
This reverts commit df7e6506d4
.
* [fx] add balanced policy v2
* add unittest
2 years ago
XYE
ca2d3f284f
[fx] Add unit test and fix bugs for transform_mlp_pass ( #1299 )
...
* add test and fix bugs
* add functions back
* add comments
2 years ago
HELSON
1b41686461
[hotfix] fix unit test test_module_spec ( #1321 )
2 years ago
Jiarui Fang
9e4c6449b0
[checkpoint] add ColoOptimizer checkpointing ( #1316 )
2 years ago
Frank Lee
7c2634f4b3
[workflow] updated release bdist workflow ( #1318 )
...
* [workflow] updated release bdist workflow
* polish workflow
* polish workflow
2 years ago
github-actions[bot]
869cf3d3b8
Automated submodule synchronization ( #1319 )
...
Co-authored-by: github-actions <github-actions@github.com>
2 years ago
Frank Lee
efdc240f1f
[workflow] disable SHM for compatibility CI on rtx3080 ( #1315 )
2 years ago
ver217
7c70bfbefa
[hotfix] fix PipelineSharedModuleGradientHandler ( #1314 )
2 years ago
Jiarui Fang
85f933b58b
[Optimizer] Remove useless ColoOptimizer ( #1312 )
2 years ago