1e88811c7a
[autoparallel] move ckpt solvers to autoparallel folder / refactor code ( #1764 )
...
* [autoparallel] first move.
* [autoparallel] add solver rotor.
* [autoparallel] add ckpt solvers.
* [autoparallel] modify codegen.
* [fx] fix annotation in test.
* [fx] remove check.
* [autoparallel] polish docstring.
* [fx] refactor MetaTensor.
2022-11-01 10:43:15 +08:00
5d0fdb9cb4
[fx] fix offload codegen test ( #1648 )
...
* [fx] fix offload codegen test
* [fx] modify typing
2022-09-27 10:25:27 +08:00
f921733621
[autoparallel] Add pofo sequence annotation ( #1637 )
...
* [autoparallel] annotate pofo sequence
* [autoparallel] remove unused print
* [autoparallel] fix some code
2022-09-24 01:52:57 +08:00
d6b01feb66
[fx] Modify offload codegen ( #1618 )
...
* [fx] modify offload codegen
* [fx] remove repeated hook definitions
* [fx] modify offload test
2022-09-23 11:04:52 +08:00
a7cda6f57d
[fx] Add offload codegen ( #1598 )
...
* [fx] add input activation offload to codegen
* [fx] modify unit test
* [fx] remove two skips in torch11
* [fx] use all_input_nodes instead of _input_nodes
2022-09-14 15:49:06 +08:00
f3687e4ee2
[fx] Add nested checkpoint in activation checkpoint codegen ( #1585 )
...
* [fx] add nested activation_checkpoint codegen
* undo algorithms commits
* solver
* undo some commits
* [fx] torch11 add nested activation checkpoint codegen
* remove some imports
* [fx] add some comments in activation codegen
* [fx] codegen instance error fix
2022-09-12 20:00:48 +08:00
56159049e8
[fx] Modify solver linearize and add corresponding test ( #1531 )
...
* [fx] modify solver linearize and add test
* [fx] add torch11 test of linearize but skip it
* [fx] remove some unused imports
2022-09-02 10:24:41 +08:00
4acc58ee20
[fx] Fix activation codegen dealing with checkpointing first op ( #1510 )
2022-08-27 19:39:21 +08:00
1f2e547f7a
[fx] Fix ckpt functions' definitions in forward ( #1476 )
...
* [fx] fix defining ckpt functions inside forward
* [fx] Modify activation checkpoint codegen and add ColoGraphModule
* [fx] some modification
* some modifications
* some modifications
* some modifications
* some modifications
* some code modifications
2022-08-22 16:59:54 +08:00
092b9c8f49
[fx] Add use_reentrant=False to checkpoint in codegen ( #1463 )
...
* [utils] Add use_reetrant=False into colossalai checkpoint
* [utils] add some annotation in utils.activaion_checkpoint
* [test] add reset_seed at the beginning of tests in test_actiavion_checkpointing.py
* [test] modify test_activation_checkpoint.py
* [test] modify test for reentrant=False
* [fx] Add use_reentrant=False of checkpoint into codegen
2022-08-17 10:34:50 +08:00
b1553fdf96
[NFC] global vars should be upper case ( #1456 )
2022-08-15 09:50:29 +08:00
5774fe0270
[fx] Use colossalai checkpoint and add offload recognition in codegen ( #1439 )
...
* [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen
* [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen
* Modification of test and add TODO in codegen
* [fx] Modification of colossal ckpt usage
* [fx] add gpc.destroy() to test_codegen
2022-08-12 12:23:30 +08:00
89e60d1505
[fx] fixed indentation error in checkpointing codegen ( #1385 )
2022-07-30 00:27:12 +08:00
cd063ac37f
[fx] added activation checkpoint codegen support for torch < 1.12 ( #1359 )
2022-07-25 23:35:31 +08:00
644582eee9
[fx] added activation checkpoint codegen ( #1355 )
2022-07-25 09:39:10 +08:00
Super Daniel