ColossalAI

Commit Graph

Author	SHA1	Message	Date
YuliangLiu0306	2c4c7b3618	[autoparallel] add getattr handler (#1767 ) * [autoparallel] add getattr haandler * polish code * add extra processes for Parameters * add unit test for param resharding cost * add docstring and polish test	2 years ago
YuliangLiu0306	e859380bf7	[fx] support module with bias addition (#1780 ) * [autoparallel] refactor tracer to fix bias addition issue * [fx] support module with bias addition * create bias_addition_module * refactor file structure * polish code * fix unit test	2 years ago
Super Daniel	1e88811c7a	[autoparallel] move ckpt solvers to autoparallel folder / refactor code (#1764 ) * [autoparallel] first move. * [autoparallel] add solver rotor. * [autoparallel] add ckpt solvers. * [autoparallel] modify codegen. * [fx] fix annotation in test. * [fx] remove check. * [autoparallel] polish docstring. * [fx] refactor MetaTensor.	2 years ago
Super Daniel	0584654c79	[fx] refactor memory utils and extend shard utils. (#1754 ) * [fx] change memory.py to memory_utils.py. * [fx] add shard utils. * [fx] fix import. * [fx] check code style. * [fx] add comment. * [autoparallel] first move. * [fx] add time computations.	2 years ago
YuliangLiu0306	314d8c497f	[autoparallel] refactor the runtime apply pass and add docstring to passes (#1757 ) * [autoparallel] refactor the runtime apply pass and add doc string to passes * fix unit test * polish	2 years ago
YuliangLiu0306	d2fc067231	[autoparallel] fix param hook issue in transform pass (#1755 )	2 years ago
Frank Lee	262652c8bc	[autoparallel] added addbmm handler (#1751 )	2 years ago
YuliangLiu0306	980ed21723	[autoparallel] shard param and buffer as expected (#1753 ) * [autoparallel] shard param and buffer as expected * fix unit test issue	2 years ago
YuliangLiu0306	a4ce180e85	[autoparallel] add sequential order to communication actions (#1735 )	2 years ago
Frank Lee	993b8875b6	[autoparallel] handled illegal sharding strategy in shape consistency (#1744 ) * [autoparallel] handled illegal sharding strategy in shape consistency * polish code	2 years ago
Super Daniel	30874f1692	[fx/profiler] debug the fx.profiler / add an example test script for fx.profiler (#1730 ) * [fx/profiler] add test. * [fx] fix file names. * [fx] add docstring and comment. * [fx] polish profiler.py. * [fx] fix import errors. * [fx] fix profiler. * [fx] fix names.	2 years ago
YuliangLiu0306	51b89d2202	[autoparallel] runtime_backward_apply (#1720 )	2 years ago
Super Daniel	393f594051	[fx/meta/rpc] move _meta_registration.py to fx folder / register fx functions with compatibility checks / remove color debug (#1710 ) * [fx] move meta registration * [fx] fix tests. * [fx] fix test. * [fx] fix. * [meta] refactor meta registration.py. * [fx] add compatibility descriptions. * [fx] polish import. * [fx] add a decorator. * [fx] fix tests. * [fx] remove print. * [fx] edit raise error. * [fx] edit raise error. * [fx] add type hint. * [fx] fix import in experimental. * [rpc] remove color debug. * [meta] fix naming.	2 years ago
YuliangLiu0306	845ff4a47a	[autoparallel] resnet block runtime apply (#1709 ) * [autoparallel] resnet block runtime apply * seperate buffer and parameter in MemoryCost * polish code * add comments and todos * fix test issue	2 years ago
YuliangLiu0306	451cd72dea	[autoparallel] adapt runtime passes (#1703 ) * [autoparallel] adapt runtime passes v2 * polish code	2 years ago
Boyuan Yao	31d2f03d27	[autoparallel] fix C version rotor inconsistency (#1691 )	2 years ago
Super Daniel	3dd6994427	[fx/profiler] assigned UUID to each unrecorded tensor/ improved performance on GPT-2 (#1679 ) * [fx/profiler] modify data_ptr into uuid for all tensors. * [fx] modify uuid. * [fx/profiler] tune performance on GPT-2. * [fx] updates. * [fx] debug. * [fx] debug. * [fx] cuda.	2 years ago
Boyuan Yao	b1be5b88bd	[autoparallel] fix insecure subprocess (#1680 ) * [autoparallel] fix insecure subprocess * [fx] fix insecure subprocess	2 years ago
Boyuan Yao	d8420f81a4	[hotfix] fix wrong type name in profiler (#1678 )	2 years ago
Boyuan Yao	132b4306b7	[fx] Add concrete info prop (#1677 ) * [fx] concreteinfoprop * [fx] add concreteinfoprop * [fx] modify docstring of ConcreteInfoProp * [fx] fix device error * [fx] modify parameter calculation * [fx] modify parameters calculation	2 years ago
Boyuan Yao	1df98d5b66	[autoparallel] add rotor C version (#1658 ) * [autoparallel] add rotor c version * [fx] remove metainfoprop in rotor solver * [autoparallel] modify C code format * [autoparallel] remove build.py * [autoparallel] fix C extension build * [autoparallel] add C solver consistency test * [autoparallel] remove some unused imports * [autoparallel] refactor rotor solver code * [autoparallel] replace print with colossalai logger * [autoparallel] ranks fixed	2 years ago
Super Daniel	6135e178b3	[fx] refactor code for profiler / enable fake tensor movement. (#1646 ) * [fx/profiling] provide summary for MetaInfoProp. * [fx/profiler] provide a table of summary. * [fx/profiler] provide a table of summary. * [fx/profiler] provide a table of summary. * [fx/profiler] provide a table of summary. * [fx] optimize table repr. * [fx] optimize table repr. * [fx] refactor code for profiler. * [fx] add docstring. * [fx] add docstring. * [fx] skip test. * [fx] redo. * [fx] redo. * [fx] fix import error for torch11. * [fx] fix import error for torch11.	2 years ago
Boyuan Yao	5d0fdb9cb4	[fx] fix offload codegen test (#1648 ) * [fx] fix offload codegen test * [fx] modify typing	2 years ago
Boyuan Yao	f921733621	[autoparallel] Add pofo sequence annotation (#1637 ) * [autoparallel] annotate pofo sequence * [autoparallel] remove unused print * [autoparallel] fix some code	2 years ago
Super Daniel	04bbabeea8	[fx/profiler] provide a table of summary. (#1634 ) * [fx/profiling] provide summary for MetaInfoProp. * [fx/profiler] provide a table of summary. * [fx] optimize table repr.	2 years ago
Boyuan Yao	d6b01feb66	[fx] Modify offload codegen (#1618 ) * [fx] modify offload codegen * [fx] remove repeated hook definitions * [fx] modify offload test	2 years ago
Super Daniel	d967779a32	[fx/profiler] tuned the calculation of memory estimation (#1619 ) * [fx] tuned the meta info and rotor solver. * [fx] remove import. * [fx] remove import. * [fx] remove import. * [fx] tune the meta calculations. * [fx] polish comments. * [fx] remove assertions. * [fx] modify test cases. * [fx] modify test cases. * [fx] optimize import. * [fx	2 years ago
YuliangLiu0306	7d1bb71d5d	[fx] PoC of runtime shape consistency application (#1607 ) * [fx] PoC of runtime shape consistency application * polish code	2 years ago
Boyuan Yao	933b6c6367	[fx] Add pofo solver (#1608 ) * [fx] add pofo algorithm * [fx] Add pofo solver * [fx] code refactor * [fx] fix test_linearize import	2 years ago
Super Daniel	cd5cf2bcc9	[fx/tuning] tune performance on rotor with meta info. (#1599 )	2 years ago
Boyuan Yao	a7cda6f57d	[fx] Add offload codegen (#1598 ) * [fx] add input activation offload to codegen * [fx] modify unit test * [fx] remove two skips in torch11 * [fx] use all_input_nodes instead of _input_nodes	2 years ago
Super Daniel	c8e9b2ad78	[hotfix/rotor] fix variable names (#1597 ) * [fx] add some comment and docstrings. * [fx] add dataflow analysis for an autograd graph. * add intepretation for graph analysis. * [fx] before doing save_tensor_hooks. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] a very accurate version on GPT-2. * [fx] refactor code. * [fx] remove redundant inplace=True. * [fx] refactor code. * [fx] refactor code. * [fx] refactor code. * [fx] dive into backward memory. * [fx] fix variable names in ckpt_solvers and unskip tests. * [fx] commit my changes. * [fx] restore skips. * [fx] restore skips. * [fx] chaange stage into phase. * [fx] chaange stage into phase. * [fx] chaange stage into phase.	2 years ago
Super Daniel	5c494d4540	[fx] provide an accurate estimation of memory. (#1587 ) * [fx] add some comment and docstrings. * [fx] add dataflow analysis for an autograd graph. * add intepretation for graph analysis. * [fx] before doing save_tensor_hooks. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] a very accurate version on GPT-2. * [fx] refactor code. * [fx] remove redundant inplace=True. * [fx] refactor code. * [fx] refactor code. * [fx] refactor code. * [fx] dive into backward memory.	2 years ago
Boyuan Yao	49ccf8b5f8	[fx] Improve linearize and rotor solver (#1586 ) * [fx] add nested activation_checkpoint codegen * undo algorithms commits * solver * undo some commits * [fx] torch11 add nested activation checkpoint codegen * remove some imports * [fx] add some comments in activation codegen * [fx] codegen instance error fix * [fx] imporve linearize and rotor solver * [fx] some comments and format modification	2 years ago
Boyuan Yao	f3687e4ee2	[fx] Add nested checkpoint in activation checkpoint codegen (#1585 ) * [fx] add nested activation_checkpoint codegen * undo algorithms commits * solver * undo some commits * [fx] torch11 add nested activation checkpoint codegen * remove some imports * [fx] add some comments in activation codegen * [fx] codegen instance error fix	2 years ago
Xue Fuzhao	e070ca45c6	[NFC] polish colossalai/fx/tracer/meta_patch/patched_module/convolution.py code style (#1563 )	2 years ago
Super Daniel	4f59693207	[fx] provide a stable but not accurate enough version of profiler. (#1547 ) * [fx] compute memory stat and flop count for MetaInfoProp. * [fx] modify node attribute. * [fx] modify ckpt_chen. * [fx] fix compatibility. * [fx] fix import error. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip test for MetaInfoProp. * [fx] skip if torch 1.11.0. * [fx] recover MetaInfoProp support for PyTorch 1.11. * [fx] provide a stable but not accurate enough version of profiler. * [fx] provide a stable but not accurate enough version of profiler. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix compatibility in tests. * [fx] fix import error.	2 years ago
Super Daniel	d8a5aded19	[hotfix] change namespace for meta_trace. (#1541 )	2 years ago
Boyuan Yao	46c6cc79a9	[fx] Add common node in model linearize (#1542 ) * [fx] Add common node into linearize * [fx] Add common node to solver	2 years ago
Super Daniel	70129603aa	[fx] support meta tracing for aten level computation graphs like functorch. (#1536 ) * [fx] support meta tracing for aten level computation graphs like functorch. * [fx] support meta tracing for aten level computation graphs like functorch. * [fx] remove redundant import. * [fx] add docstring.	2 years ago
Boyuan Yao	56159049e8	[fx] Modify solver linearize and add corresponding test (#1531 ) * [fx] modify solver linearize and add test * [fx] add torch11 test of linearize but skip it * [fx] remove some unused imports	2 years ago
YuliangLiu0306	4b3d6caeb3	[fx]patch nn.functional convolution (#1528 )	2 years ago
Super Daniel	112a1f0a8f	[hotfix] avoid conflict of meta registry with torch 1.13.0. (#1530 ) * [hotfix] avoid conflict of meta registry with torch 1.13.0. * [hotfix] avoid conflict of meta registry with torch 1.13.0.	2 years ago
Boyuan Yao	b231430bcb	[fx] Fix wrong index in annotation and minimal flops in ckpt solver (#1521 ) * [fx] fix wrong variable name in solver rotor * [fx] fix wrong variable name in solver rotor * [fx] fix the discretize bug * [fx] fix the first op in activation checkpoint codegen * [fx] fix some bugs of ckpt solver * [fx] modify test_ckpt_torchvision * [fx] set sequence to __sequence__ attr of GraphModule * [fx] docstring modification * [fx] remove performance test	2 years ago
Super Daniel	5cc849f6ce	[fx] hack __torch_dispatch__ for meta tensor and autograd. (#1515 ) * [fx] hack __torch_dispatch__ for meta tensor and autograd. * [fx] hack __torch_dispatch__ for meta tensor and autograd. * [fx] hack __torch_dispatch__ for meta tensor and autograd. * [fx] hack __torch_dispatch__ for meta tensor and autograd. * [fx] hack __torch_dispatch__ for meta tensor and autograd. * [fx] add bad case detections. * [fx] add bad case detections. * [fx] rename MetaTensor attributes. * [fx] fix unexpected error. * [fx] fix unexpected error. * [fx] fix unexpected error. * [fx] fix unexpected error. * [fx] fix unexpected error. * [fx] add register backward for native_batch_norm_backward. * [fx] add more meta backend support for nn.Modules. * [fx] add meta backend to support timm and torchvision models. * [fx] add meta hardswish for timm models.	2 years ago
Frank Lee	a0436a62ee	[autoparallel] added liveness analysis (#1516 ) * [autoparallel] added liveness analysis * remove memory cost	2 years ago
Super Daniel	ea1a95b8b9	[hotfix] fix coloproxy typos. (#1519 )	2 years ago
Boyuan Yao	4acc58ee20	[fx] Fix activation codegen dealing with checkpointing first op (#1510 )	2 years ago
Boyuan Yao	ac3a453a50	[fx] fix the discretize bug (#1506 ) * [fx] fix wrong variable name in solver rotor * [fx] fix wrong variable name in solver rotor * code modification * [fx] fix the discretize bug	2 years ago
Boyuan Yao	31fffd3fc5	[fx] fix wrong variable name in solver rotor (#1502 ) * [fx] fix wrong variable name in solver rotor * [fx] fix wrong variable name in solver rotor * code modification	2 years ago

1 2 3 4

161 Commits (5f8c0a0ac3b52a71b664c3e36dd1a8cef40f428d)