ColossalAI

Commit Graph

Author	SHA1	Message	Date
HELSON	a1ce02d740	[zero] test gradient accumulation (#1964 ) * [zero] fix memory leak for zero2 * [zero] test gradient accumulation * [zero] remove grad clip test	2 years ago
Ziyue Jiang	b0936e4a44	[rpc] split with dag (#2028 ) * add DAG to split_module * add comment * add test case for DAG * remove print * add DAG middleware in scheduler * add test case for scheduler * remove break * recover old lifecycle Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
Jiarui Fang	96134e7be3	[hotfix] add bert test for gemini fwd bwd (#2035 )	2 years ago
YuliangLiu0306	0dbcd4a6f5	[autoparallel] add split handler (#2032 ) * [autoparallel] add split handler * add numerical test and runtime passes	2 years ago
Jiarui Fang	28aa9a4294	[Gemini] more rigorous unit tests for run_fwd_bwd (#2034 )	2 years ago
YuliangLiu0306	81330b0352	[autoparallel] add experimental permute handler (#2029 )	2 years ago
Zihao	95c4532fff	[Gemini] paramWrapper paramTracerHook unitest (#2030 )	2 years ago
Jiarui Fang	8daf1b4db1	[Gemini] patch for supporting orch.add_ function for ColoTensor (#2003 )	2 years ago
Ziyue Jiang	632753abbc	[fx]Split partition with DAG information (#2025 ) * add DAG to split_module * add comment * add test case for DAG * remove print Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
YuliangLiu0306	ea0f6b8df9	[autoparallel] add runtime pass and numerical test for view handler (#2018 )	2 years ago
Jiarui Fang	2e9cbfca12	[Gemini] add unitests to check gemini correctness (#2015 )	2 years ago
Jiarui Fang	0b0d8f9e17	[hotfix] revert bug PRs (#2016 )	2 years ago
Zihao	0160a62a3c	[Gemini] param_tracer_wrapper and test case (#2009 )	2 years ago
YuliangLiu0306	1438993113	[autoparallel] add experimental view handler (#2011 ) * [autoparallel] add experimental view handler * polish * polish * polish code * rename variables	2 years ago
Genghan Zhang	d655eea515	[autoparallel] mix gather (#1977 ) * Add mix-gather * Add comments * Add comments * Polish comments * Change the global rank assumption * Add tests * Add two-step tests * Fix 10 and 01 * Skip test becasue the number of GPUs	2 years ago
Jiarui Fang	3d907faede	[Gemini] add an inline_op_module to common test models and polish unitests. (#2004 )	2 years ago
Boyuan Yao	6cd784ffee	[autoparallel] Add metainfo support for F.linear (#1987 ) * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test * [autoparallel] add batchnorm metainfo class * [autoparallel] fix batchnorm unit test function declaration * [fx] restore profiler * [fx] add relu metainfo class * [fx] restore profiler * [autoparallel] modify metainfo input * [autoparallel] add pooling metainfo * [autoparallel] add F.linear metainfo generator	2 years ago
YuliangLiu0306	35e6b9ec82	[autoparallel] adapt handlers with attention block (#1990 ) * [autoparallel] adapt handlers with attention block * polish	2 years ago
Jiarui Fang	5bec3b2168	[Gemini] open grad checkpoint when model building (#1984 )	2 years ago
Boyuan Yao	c26f21d365	[autoparallel] add pooling metainfo (#1968 ) * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test * [autoparallel] add batchnorm metainfo class * [autoparallel] fix batchnorm unit test function declaration * [fx] restore profiler * [fx] add relu metainfo class * [fx] restore profiler * [autoparallel] modify metainfo input * [autoparallel] add pooling metainfo	2 years ago
Jiarui Fang	3712ac7f90	[Gemini] add bert for MemtracerWrapper unintests (#1982 )	2 years ago
Jiarui Fang	e481489aa6	[Gemini] MemtracerWrapper unittests (#1981 )	2 years ago
YuliangLiu0306	0da1d00399	[autoparallel] support distributed dataloader option (#1906 ) * [autoparallel] support distributed dataloader option * update output handler to support ddp dataloader * poish code	2 years ago
Genghan Zhang	6630d45546	[autoparallel] Add alpha beta (#1973 ) * Add alpha beta * Fix test * Fix test	2 years ago
ver217	f8a7148dec	[kernel] move all symlinks of kernel to `colossalai._C` (#1971 )	2 years ago
Boyuan Yao	7c7921f71b	[autoparallel] add torch.nn.ReLU metainfo (#1868 ) * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test * [autoparallel] add batchnorm metainfo class * [autoparallel] fix batchnorm unit test function declaration * [fx] restore profiler * [fx] add relu metainfo class * [fx] restore profiler * [autoparallel] modify metainfo input	2 years ago
YuliangLiu0306	fea3cb661c	[autoparallel] support addmm in tracer and solver (#1961 ) * [fx] patch addmm * [autoparallel] support addmm in tracer and solver	2 years ago
Jiarui Fang	f7e276fa71	[Gemini] add GeminiAdamOptimizer (#1960 )	2 years ago
HELSON	7066dfbf82	[zero] fix memory leak for zero2 (#1955 )	2 years ago
Jiarui Fang	52c6ad26e0	[ColoTensor] reconfig ColoInitContext, decouple default_pg and default_dist_spec. (#1953 )	2 years ago
zbian	6877121377	updated flash attention api	2 years ago
Jiarui Fang	9f4fb3f28a	[ColoTensor] ColoInitContext initialize parameters in shard mode. (#1937 )	2 years ago
HELSON	6e51d296f0	[zero] migrate zero1&2 (#1878 ) * add zero1&2 optimizer * rename test ditectory * rename test files * change tolerance in test	2 years ago
Jiarui Fang	51597f6a28	[hotfix] pass test_complete_workflow (#1877 )	2 years ago
Jiarui Fang	986f8cbaa7	[inference] overlap comm and compute in Linear1D_Row when stream_chunk_num > 1 (#1876 )	2 years ago
YuliangLiu0306	1b494ad73c	[autoparallel] fix linear logical convert issue (#1857 )	2 years ago
Jiarui Fang	c2947dadf1	[inference] streaming Linear 1D Row inference (#1874 )	2 years ago
xcnick	a141681260	[amp] add torch amp test (#1860 )	2 years ago
Frank Lee	e6ec99d389	[utils] fixed lazy init context (#1867 )	2 years ago
Jiarui Fang	3ce4463fe6	[utils] remove lazy_memory_allocate from ColoInitContext (#1844 )	2 years ago
YuliangLiu0306	f6032ddb17	[autoparallel] fix bias addition module (#1800 )	2 years ago
ver217	99870726b1	[CheckpointIO] a uniform checkpoint I/O module (#1689 )	2 years ago
Boyuan Yao	629172b319	[autoparallel] add batch norm metainfo (#1815 ) * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test * [autoparallel] add batchnorm metainfo class * [autoparallel] fix batchnorm unit test function declaration * [fx] restore profiler	2 years ago
Super Daniel	441d584e4a	[fx] add a symbolic_trace api. (#1812 ) * [fx] add a symbolic_trace api. * [fx] fix import errors.	2 years ago
Jiarui Fang	6fa71d65d3	[fx] skip diffusers unitest if it is not installed (#1799 )	2 years ago
oahzxl	9639ea88fc	[kernel] more flexible flashatt interface (#1804 )	2 years ago
Boyuan Yao	327d07c44a	[autoparallel] add conv metainfo class for auto parallel (#1796 ) * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test	2 years ago
oahzxl	501a9e9cd2	[hotfix] polish flash attention (#1802 )	2 years ago
Jiarui Fang	c248800359	[kernel] skip tests of flash_attn and triton when they are not available (#1798 )	2 years ago
YuliangLiu0306	e34e850a4c	[autoparallel]add essential CommActions for broadcast oprands (#1793 )	2 years ago

... 3 4 5 6 7 ...

768 Commits (788e07dbc5dc5acaf34e24d98238780ecf134ef2)