ColossalAI

Commit Graph

Author	SHA1	Message	Date
Super Daniel	4f59693207	[fx] provide a stable but not accurate enough version of profiler. (#1547 ) * [fx] compute memory stat and flop count for MetaInfoProp. * [fx] modify node attribute. * [fx] modify ckpt_chen. * [fx] fix compatibility. * [fx] fix import error. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip if torch 1.11.0. * [fx] recover MetaInfoProp support for PyTorch 1.11. * [fx] provide a stable but not accurate enough version of profiler. * [fx] provide a stable but not accurate enough version of profiler. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix import error.	2 years ago
Boyuan Yao	56159049e8	[fx] Modify solver linearize and add corresponding test (#1531 ) * [fx] modify solver linearize and add test * [fx] add torch11 test of linearize but skip it * [fx] remove some unused imports	2 years ago
Super Daniel	7dc53237c3	[fx] add test for meta tensor. (#1527 ) * [fx] add test for meta tensor. * [fx] add test for meta tensor. * [fx] add test for meta tensor. * [fx] add test for meta tensor. * [fx] fix error.	2 years ago
YuliangLiu0306	4b3d6caeb3	[fx]patch nn.functional convolution (#1528 )	2 years ago
Boyuan Yao	b231430bcb	[fx] Fix wrong index in annotation and minimal flops in ckpt solver (#1521 ) * [fx] fix wrong variable name in solver rotor * [fx] fix wrong variable name in solver rotor * [fx] fix the discretize bug * [fx] fix the first op in activation checkpoint codegen * [fx] fix some bugs of ckpt solver * [fx] modify test_ckpt_torchvision * [fx] set sequence to __sequence__ attr of GraphModule * [fx] docstring modification * [fx] remove performance test	2 years ago
Boyuan Yao	4acc58ee20	[fx] Fix activation codegen dealing with checkpointing first op (#1510 )	2 years ago
Boyuan Yao	de1e716dc4	[fx] Add activation checkpoint solver rotor (#1496 ) * [fx] fix defining ckpt functions inside forward * [fx] Modify activation checkpoint codegen and add ColoGraphModule * [fx] some modification * some modifications * some modifications * some modifications * some modifications * some code modifications * [automatic_parallel] ckpt solver rotor * [fx] add ckpt_solver_rotor * [fx] modification * code refactor * code refactor	2 years ago
Frank Lee	3da68d6b1b	[fx] fixed adapative pooling size concatenation error (#1489 )	2 years ago
Super Daniel	32efe8e740	[fx] add profiler for fx nodes. (#1480 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] add rules to linearize computation graphs for searching. (#2) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] fix test and algorithm bugs in activation checkpointing. * [fx] polish ckpt_test. * [fx] add rules to linearize computation graphs for searching. * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] fix inconsistencies. * [fx] fix MetaInfoProp. * [fx] fix MetaInfoProp. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] add profiler for fx nodes. * [fx] fix error in tests. * [fx] unfix bug. * [fx] unfix bug.	2 years ago
Boyuan Yao	1f2e547f7a	[fx] Fix ckpt functions' definitions in forward (#1476 ) * [fx] fix defining ckpt functions inside forward * [fx] Modify activation checkpoint codegen and add ColoGraphModule * [fx] some modification * some modifications * some modifications * some modifications * some modifications * some code modifications	2 years ago
Super Daniel	e7383f578b	[fx] add rules to linearize computation graphs for searching. (#1461 ) * [fx] polish ckpt_test. * [fx] add rules to linearize computation graphs for searching. * [fx] remove chen_sqrt for sake of simplicity * [fx] fix inconsistencies.	2 years ago
Boyuan Yao	092b9c8f49	[fx] Add use_reentrant=False to checkpoint in codegen (#1463 ) * [utils] Add use_reetrant=False into colossalai checkpoint * [utils] add some annotation in utils.activaion_checkpoint * [test] add reset_seed at the beginning of tests in test_actiavion_checkpointing.py * [test] modify test_activation_checkpoint.py * [test] modify test for reentrant=False * [fx] Add use_reentrant=False of checkpoint into codegen	2 years ago
Super Daniel	0dbd61c29b	[fx] fix test and algorithm bugs in activation checkpointing. (#1451 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * [fx] polish ckpt_test. * [fx] polish ckpt_test. * [fx] polish ckpt_test.	2 years ago
Frank Lee	5a52e21fe3	[test] fixed the activation codegen test (#1447 ) * [test] fixed the activation codegen test * polish code	2 years ago
Boyuan Yao	5774fe0270	[fx] Use colossalai checkpoint and add offload recognition in codegen (#1439 ) * [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen * [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen * Modification of test and add TODO in codegen * [fx] Modification of colossal ckpt usage * [fx] add gpc.destroy() to test_codegen	2 years ago
Super Daniel	d40a9392ba	[fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174 . (#1446 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * mend * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions.	2 years ago
Super Daniel	3b26516c69	[fx] add vanilla activation checkpoint search with test on resnet and densenet (#1433 ) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen.	2 years ago
Super Daniel	f20cb4e893	[fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages (#1425 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages	2 years ago
Frank Lee	adf5054ff8	[fx] fixed torchaudio conformer tracing (#1392 )	2 years ago
Frank Lee	7d6293927f	[fx] patched torch.max and data movement operator (#1391 ) * [fx] patched torch.max and data movement operator * polish code	2 years ago
Super Daniel	be229217ce	[fx] add torchaudio test (#1369 ) * [fx]add torchaudio test * [fx]add torchaudio test * [fx] add torchaudio test * [fx] add torchaudio test * [fx] add torchaudio test * [fx] add torchaudio test * [fx] add torchaudio test * [fx] add torchaudio test and test patches * Delete ~ * [fx] add patches and patches test * [fx] add patches and patches test * [fx] fix patches * [fx] fix rnn patches * [fx] fix rnn patches * [fx] fix rnn patches * [fx] fix rnn patches * [fx] merge upstream * [fx] fix import errors	2 years ago
Boyuan Yao	bb640ec728	[fx] Add colotracer compatibility test on torchrec (#1370 )	2 years ago
Frank Lee	cd063ac37f	[fx] added activation checkpoint codegen support for torch < 1.12 (#1359 )	2 years ago
Frank Lee	644582eee9	[fx] added activation checkpoint codegen (#1355 )	2 years ago
Frank Lee	05fae1fd56	[fx] added activation checkpointing annotation (#1349 ) * [fx] added activation checkpointing annotation * polish code * polish code	2 years ago
YuliangLiu0306	942c8cd1fb	[fx] refactor tracer to trace complete graph (#1342 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * [fx] refactor tracer to trace complete graph * add comments and solve conflicts.	2 years ago
Frank Lee	2cc1175c76	[fx] tested the complete workflow for auto-parallel (#1336 ) * [fx] tested the complete workflow for auto-parallel * polish code * polish code * polish code	2 years ago
YuliangLiu0306	4631fef8a0	[fx]refactor tracer (#1335 )	2 years ago
Frank Lee	f3ce7b8336	[fx] recovered skipped pipeline tests (#1338 )	2 years ago
Frank Lee	75abc75c15	[fx] fixed compatiblity issue with torch 1.10 (#1331 )	2 years ago
Frank Lee	b2475d8c5c	[fx] fixed unit tests for torch 1.12 (#1327 )	2 years ago
YuliangLiu0306	e8acf55e8b	[fx] add balanced policy v2 (#1251 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * [fx] add balanced policy v2 * add unittest	2 years ago
XYE	ca2d3f284f	[fx] Add unit test and fix bugs for transform_mlp_pass (#1299 ) * add test and fix bugs * add functions back * add comments	2 years ago
Frank Lee	4f4d8c3656	[fx] added apex normalization to patched modules (#1300 ) * [fx] added apex normalization to patched modules * remove unused imports	2 years ago
YuliangLiu0306	93a75433df	[hotfix] skip some unittest due to CI environment. (#1301 )	2 years ago
Frank Lee	7e8114a8dd	[hotfix] skipped unsafe test cases (#1282 )	2 years ago
YuliangLiu0306	01ea68b2e6	[tests] remove T5 test skip decorator (#1271 )	2 years ago
Jiarui Fang	ca9d5ee91c	[hotfix] torchvison fx unittests miss import pytest (#1277 )	2 years ago
Frank Lee	4a09fc0947	[fx] fixed tracing with apex-based T5 model (#1252 ) * [fx] fixed tracing with apex-based T5 model * polish code * polish code	2 years ago
YuliangLiu0306	97d713855a	[fx] methods to get fx graph property. (#1246 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * manipulation * [fx]add graph manipulation methods. * [fx]methods to get fx graph property. * add unit test * add docstring to explain top node and leaf node in this context	2 years ago
YuliangLiu0306	30b4fc0eb0	[fx]add split module pass and unit test from pipeline passes (#1242 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * [fx]add split module pass and unit test from pipeline passes * fix MNASNet bug * polish	2 years ago
Frank Lee	5581170890	[fx] fixed huggingface OPT and T5 results misalignment (#1227 )	2 years ago
YuliangLiu0306	2b7dca44b5	[fx]get communication size between partitions (#1224 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * [fx]get communication size between partitions. * polish	2 years ago
Frank Lee	84f2298a96	[fx] added patches for tracing swin transformer (#1228 )	2 years ago
Frank Lee	37fcf96b7f	[fx] fixed timm tracing result misalignment (#1225 )	2 years ago
Frank Lee	b6cb5a47ad	[fx] added timm model tracing testing (#1221 )	2 years ago
Frank Lee	11973d892d	[fx] added torchvision model tracing testing (#1216 ) * [fx] added torchvision model tracing testing * remove unused imports	2 years ago
Jiarui Fang	ae7d3f4927	[refactor] move process group from _DistSpec to ColoTensor. (#1203 )	2 years ago
Frank Lee	5da87ce35d	[fx] added testing for all albert variants (#1211 )	2 years ago
Frank Lee	2d13a45a3b	[fx] added testing for all gpt variants (#1210 ) * [fx] added testing for all gpt variants * polish code * polish code	2 years ago
YuliangLiu0306	189946c5c4	[fx]add uniform policy (#1208 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * [fx]add uniform policy	2 years ago
Frank Lee	426a279ce7	[fx] added testing for all bert variants (#1207 ) * [fx] added testing for all bert variants * polish code	2 years ago
Frank Lee	f7878f465c	[fx] supported model tracing for huggingface bert (#1201 ) * [fx] supported model tracing for huggingface bert * polish test	2 years ago
Frank Lee	abf6a262dc	[fx] added module patch for pooling layers (#1197 )	2 years ago
Frank Lee	2c8c05675d	[fx] patched conv and normalization (#1188 )	2 years ago
Frank Lee	6d86f1bc91	[fx] supported data-dependent control flow in model tracing (#1185 ) * [fx] supported data-dependent control flow in model tracing * polish code	2 years ago
YuliangLiu0306	fcf55777dd	[fx]add autoparallel passes (#1121 ) * [CLI] add CLI launcher * Revert "[CLI] add CLI launcher" This reverts commit `df7e6506d4`. * feature/add autoparallel passes	2 years ago
Frank Lee	16302a5359	[fx] added unit test for coloproxy (#1119 ) * [fx] added unit test for coloproxy * polish code * polish code	2 years ago

1 2 3

108 Commits (f2e8b9ef9ff3032513732a699d766bcde1a3506e)