ColossalAI

Commit Graph

Author	SHA1	Message	Date
Geng Zhang	0aad53c62b	[FCE] update interface for frequency statistics in FreqCacheEmbedding (#1462 )	2 years ago
Frank Lee	ede326298b	[autoparallel] integrate auto parallel with torch fx (#1479 )	2 years ago
github-actions[bot]	8fb09a950a	Automated submodule synchronization (#1478 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
fastalgo	0f438d15ee	Update README.md	2 years ago
Sze-qq	1750d6f573	[doc] update readme with the new xTrimoMultimer project (#1477 ) * update xTrimoMultimer project * update xTrimoMultimer project * latest update of xTrimoMultimer project info	2 years ago
Boyuan Yao	1f2e547f7a	[fx] Fix ckpt functions' definitions in forward (#1476 ) * [fx] fix defining ckpt functions inside forward * [fx] Modify activation checkpoint codegen and add ColoGraphModule * [fx] some modification * some modifications * some modifications * some modifications * some modifications * some code modifications	2 years ago
Kirigaya Kazuto	bb5f5289e0	[pipeline/rpc] implement a demo for PP with cuda rpc framework (#1470 ) * support p2p communication with any type of object \| pass test * reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) \| pass test * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [pipeline/rpc] implement a demo for PP with cuda rpc framework * Delete p2p_v2.py * Delete _pipeline_schedule_v2.py * Delete test_object_list_p2p_v2.py * Delete test_boardcast_send_recv_v2.py * Delete test_cifar_with_data_pipeline_tensor_v2.py	2 years ago
Frank Lee	628c7e3fc8	[autoparallel] added dot handler (#1475 )	2 years ago
github-actions[bot]	d08566fb61	Automated submodule synchronization (#1472 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Frank Lee	9dae9bb2bc	[autoparallel] introduced baseclass for op handler and reduced code redundancy (#1471 ) * [autoparallel] introduced baseclass for op handler and reduced code redundancy * polish code	2 years ago
Frank Lee	3a54e1c9b7	[autoparallel] standardize the code structure (#1469 )	2 years ago
YuliangLiu0306	26a37b5cd5	[autoparallel] Add conv handler to generate strategies and costs info for conv (#1467 )	2 years ago
Jiarui Fang	1b491ad7de	[doc] update docstring in ProcessGroup (#1468 )	2 years ago
YuliangLiu0306	b73fb7a077	[tensor] support runtime ShardingSpec apply (#1453 ) * [tensor] support runtime ShardingSpec apply * polish code * polish code	2 years ago
github-actions[bot]	177d3f5718	Automated submodule synchronization (#1465 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Super Daniel	bbc58d881b	[fx] fix MetaInfoProp for incorrect calculations and add detections for inplace op. (#1466 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] add rules to linearize computation graphs for searching. (#2) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] fix test and algorithm bugs in activation checkpointing. * [fx] polish ckpt_test. * [fx] add rules to linearize computation graphs for searching. * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] remove chen_sqrt for sake of simplicity * [fx] fix inconsistencies. * [fx] fix MetaInfoProp. * [fx] fix MetaInfoProp. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands. * [fx] consider MetaInfoProp for inplace operands.	2 years ago
Super Daniel	e7383f578b	[fx] add rules to linearize computation graphs for searching. (#1461 ) * [fx] polish ckpt_test. * [fx] add rules to linearize computation graphs for searching. * [fx] remove chen_sqrt for sake of simplicity * [fx] fix inconsistencies.	2 years ago
github-actions[bot]	a7a3d55114	Automated submodule synchronization (#1452 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Boyuan Yao	092b9c8f49	[fx] Add use_reentrant=False to checkpoint in codegen (#1463 ) * [utils] Add use_reetrant=False into colossalai checkpoint * [utils] add some annotation in utils.activaion_checkpoint * [test] add reset_seed at the beginning of tests in test_actiavion_checkpointing.py * [test] modify test_activation_checkpoint.py * [test] modify test for reentrant=False * [fx] Add use_reentrant=False of checkpoint into codegen	2 years ago
Boyuan Yao	47fd8e4a02	[utils] Add use_reetrant=False in utils.activation_checkpoint (#1460 ) * [utils] Add use_reetrant=False into colossalai checkpoint * [utils] add some annotation in utils.activaion_checkpoint * [test] add reset_seed at the beginning of tests in test_actiavion_checkpointing.py * [test] modify test_activation_checkpoint.py * [test] modify test for reentrant=False	2 years ago
Jiarui Fang	36824a304c	[Doc] add more doc for ColoTensor. (#1458 )	2 years ago
Jiarui Fang	a1476ea882	[NFC] polish doc style for ColoTensor (#1457 )	2 years ago
Super Daniel	0dbd61c29b	[fx] fix test and algorithm bugs in activation checkpointing. (#1451 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] merge development into main (#1) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions. * [fx] simplify test for ckpt. * [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * mend [fx] fix test and algorithm bugs in activation checkpointing. * [fx] polish ckpt_test. * [fx] polish ckpt_test. * [fx] polish ckpt_test.	2 years ago
Jiarui Fang	b1553fdf96	[NFC] global vars should be upper case (#1456 )	2 years ago
ver217	367c615818	fix nvme docstring (#1450 )	2 years ago
Frank Lee	6474e31556	[workflow] added TensorNVMe to compatibility test (#1449 )	2 years ago
Geng Zhang	9f3eed66eb	[FAW] reorganize the inheritance struct of FreqCacheEmbedding (#1448 )	2 years ago
Frank Lee	5a52e21fe3	[test] fixed the activation codegen test (#1447 ) * [test] fixed the activation codegen test * polish code	2 years ago
YuliangLiu0306	0f3042363c	[tensor] shape consistency generate transform path and communication cost (#1435 ) * [tensor] shape consistency output transform path and communication cost * polish code	2 years ago
Boyuan Yao	5774fe0270	[fx] Use colossalai checkpoint and add offload recognition in codegen (#1439 ) * [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen * [fx] Use colossalai.utils.checkpoint to replace torch.utils.checkpoint for offload activation and add offload annotation recognition in codegen * Modification of test and add TODO in codegen * [fx] Modification of colossal ckpt usage * [fx] add gpc.destroy() to test_codegen	2 years ago
Kirigaya Kazuto	e9460b45c8	[engin/schedule] use p2p_v2 to recontruct pipeline_schedule (#1408 ) * support p2p communication with any type of object \| pass test * reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) \| pass test * [communication] add p2p_v2.py to support communication with List[Any] * Delete _pipeline_schedule_v2.py * Delete test_cifar_with_data_pipeline_tensor_v2.py * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * Delete p2p_v2.py * Delete test_boardcast_send_recv_v2.py * Delete test_object_list_p2p_v2.py * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [communication] remove print code * [communication] remove print code * [engin/schedule] shorten the running time of testing file to prevent cancelling in CI	2 years ago
Frank Lee	ae1b58cd16	[tensor] added linear implementation for the new sharding spec (#1416 ) * [tensor] added linear implementation for the new sharding spec * polish code	2 years ago
Super Daniel	d40a9392ba	[fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174 . (#1446 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * mend * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen. * [fx] fix the false interpretation of algorithm 3 in https://arxiv.org/abs/1604.06174. * [fx] fix lowercase naming conventions.	2 years ago
ver217	821c6172e2	[utils] Impl clip_grad_norm for ColoTensor and ZeroOptimizer (#1442 )	2 years ago
Frank Lee	74bee5f7e8	[release] update version.txt (#1444 )	2 years ago
HELSON	b80340168e	[zero] add chunk_managerV2 for all-gather chunk (#1441 )	2 years ago
Super Daniel	3b26516c69	[fx] add vanilla activation checkpoint search with test on resnet and densenet (#1433 ) * [fx] activation checkpointing using Chen strategies. * [fx] add test for ckpt_solver_chen * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add vanilla activation checkpoint search with test on resnet and densenet * [fx] add a namespace code for solver_chen.	2 years ago
Jiarui Fang	30b4dd17c0	[FAW] export FAW in _ops (#1438 )	2 years ago
HELSON	9056677b13	[zero] add chunk size searching algorithm for parameters in different groups (#1436 )	2 years ago
Jiarui Fang	c9427a323f	hotfix #1434 (#1437 )	2 years ago
HELSON	039b7ed3bc	[polish] add update directory in gemini; rename AgChunk to ChunkV2 (#1432 )	2 years ago
Super Daniel	f20cb4e893	[fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages (#1425 ) * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages * [fx] modify the calculation of node_size in MetaInfoProp for activation checkpointing usages	2 years ago
Jiarui Fang	89c434a0a6	[polish] add test_ops directory (#1431 )	2 years ago
Jiarui Fang	10b3df65c8	[FAW] move coloparam setting in test code. (#1429 )	2 years ago
Jiarui Fang	cb98cf5558	[FAW] parallel FreqAwareEmbedding (#1424 )	2 years ago
HELSON	0d212183c4	[zero] add has_inf_or_nan in AgChunk; enhance the unit test of AgChunk (#1426 )	2 years ago
YuliangLiu0306	33f0744d51	[tensor] add shape consistency feature to support auto spec transform (#1418 ) * [tensor] add shape consistency feature to supportauto sharding spec transform. * [tensor] remove unused argument in simulator, add doc string for target pair.	2 years ago
HELSON	4fb3c52cf0	[zero] add unit test for AgChunk's append, close, access (#1423 )	2 years ago
HELSON	c577ed016e	[zero] add AgChunk (#1417 )	2 years ago
Jiarui Fang	d209aff684	Add FreqAwareEmbeddingBag (#1421 )	2 years ago

1 2 3 4 5 ...

1031 Commits (d8a5aded19c6564500d7bbb8c4091cd06e6da003) All Branches Search

1031 Commits (d8a5aded19c6564500d7bbb8c4091cd06e6da003)

All Branches