647 Commits (4befaabace567589251a0c5ba2916a7fc891bcfa)

Author	SHA1	Message	Date
HELSON	ea13a201bb	[polish] polish code for get_static_torch_model (#2405) ... * [gemini] polish code * [testing] remove code * [gemini] make more robust	2 years ago
Jiarui Fang	4e96039649	[device] find best logical mesh	2 years ago
Frank Lee	40d376c566	[setup] support pre-build and jit-build of cuda kernels (#2374) ... * [setup] support pre-build and jit-build of cuda kernels * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
HELSON	48d33b1b17	[gemini] add get static torch model (#2356)	2 years ago
YuliangLiu0306	b5a3a4a65f	[device] find best logical mesh	2 years ago
YuliangLiu0306	9c9246c0d9	[device] alpha beta profiler (#2311) ... * [device] alpha beta profiler * add usage * fix variable name	2 years ago
Jiarui Fang	db6eea3583	[builder] reconfig op_builder for pypi install (#2314)	2 years ago
HELSON	5d3a2be3af	[amp] add gradient clipping for unit tests (#2283) ... * [amp] add gradient clipping in unit tests * fix bugs	2 years ago
zbian	e94c79f15b	improved allgather & reducescatter for 3d	2 years ago
YuliangLiu0306	fb87322773	[autoparallel] fix spelling error (#2270)	2 years ago
YuliangLiu0306	8897b8f753	[autoparallel] autoparallel initialize (#2238)	2 years ago
YuliangLiu0306	3b1b91eaf4	[autoparallel] record parameter attribute in colotracer (#2217) ... * [autoparallel] record parameter attribute in collotracer * [autoparallel] fix construct_meta_info bug	2 years ago
Boyuan Yao	24246f7aa5	[autoparallel] Attach input, buffer and output tensor to MetaInfo class (#2162) ... * [fx] metainfo class for auto parallel * [fx] add unit test for linear metainfo * [fx] fix bwd param for linear * [fx] modify unit test * [fx] modify unit test * [fx] modify import * [fx] modify import * [fx] modify import * [fx] move meta profiler to auto parallel * [fx] add conv metainfo class * [fx] restore profiler * [fx] restore meta profiler * [autoparallel] modify unit test * [fx] modify unit test * [autoparallel] add batchnorm metainfo class * [autoparallel] fix batchnorm unit test function declaration * [fx] restore profiler * [fx] add relu metainfo class * [fx] restore profiler * [autoparallel] modify metainfo input * [autoparallel] add pooling metainfo * [autoparallel] add F.linear metainfo generator * [autoparallel] add binary elementwise metainfo * [fx] recover profiler * [autoparallel] fix forward memory calculation * [autoparallel] modify constants.py * [autoparallel] remove redundant print * [autoparallel] add F.conv metainfo * [autoparallel] linear fix * [autoparallel] memory estimation for communication actions * [autoparallel] fix docstring * [autoparallel] fix variables name * [autoparallel] attach tensor to metainfo class * [autoparallel] fix dangerous try except * [autoparallel] attach memory cost to shape consistency node * [autoparallel] attach shape consistency node's metainfo to the node * [autoparallel] remove todo in shape consistency memory estimation * [autoparallel] fix the annotation	2 years ago
YuliangLiu0306	78509124d3	[autoparallel] update getitem handler (#2207)	2 years ago
YuliangLiu0306	4851f2d607	[autoparallel] update_getattr_handler (#2193)	2 years ago
YuliangLiu0306	f10ce01e31	[autoparallel] add gpt2 performance test code (#2194)	2 years ago
HELSON	a3100bd50d	[testing] add beit model for unit testings (#2196) ... * [testing] add beit model * [beit] fix bugs * [beit] fix bugs * [testing] fix bugs	2 years ago
HELSON	2458659919	[zero] fix error for BEiT models (#2169) ... * [zero] fix error for BEiT models * [ColoParameter] add unpack operation for tuple arguments * fix bugs * fix chunkv2 unit testing * add assertion for gradient state	2 years ago
Jiarui Fang	355ffb386e	[builder] unified cpu_optim fused_optim inferface (#2190)	2 years ago
Jiarui Fang	9587b080ba	[builder] use runtime builder for fused_optim (#2189)	2 years ago
Jiarui Fang	bc0e271e71	[buider] use builder() for cpu adam and fused optim in setup.py (#2187)	2 years ago
Jiarui Fang	d42afd30f8	[builder] runtime adam and fused_optim builder (#2184)	2 years ago
YuliangLiu0306	550f8f8905	[autoparallel] integrate_gpt_related_tests (#2134) ... * [autoparallel] integrate_gpt_related_tests * polish code * polish code * add GPT2Model into runtime test	2 years ago
Jiarui Fang	27327a4c90	[example] add palm pytorch version (#2172)	2 years ago
Jiarui Fang	b87496a66b	[hotfix] fix auto policy of test_sharded_optim_v2 (#2157)	2 years ago
YuliangLiu0306	16335cb537	[hotfix] fix aten default bug (#2158)	2 years ago
Jiarui Fang	2827f41898	[Gemini] GeminiDPP convert to PyTorch Module. (#2151)	2 years ago
アマデウス	077a66dd81	updated attention kernel (#2133)	2 years ago
YuliangLiu0306	536560ccc0	[autoparallel] implement softmax handler (#2132)	2 years ago
Jiarui Fang	c89c66a858	[Gemini] update API of the chunkmemstatscollector. (#2129)	2 years ago
Jiarui Fang	2938edf446	[Gemini] update the non model data record method in runtime memory tracer (#2128)	2 years ago
Jiarui Fang	deee317b0f	[Gemini] test step-tensor mapping using repeated_computed_layers.py (#2127)	2 years ago
Jiarui Fang	8fac837679	[Gemini] update non model data calculation method (#2126)	2 years ago
Jiarui Fang	5efda69735	[Gemini] hotfix the unittest bugs (#2125)	2 years ago
Jiarui Fang	05bb28aacf	[Gemini] mapping of preop timestep and param (#2124)	2 years ago
YuliangLiu0306	cd0af9f7f6	[autoparallel] gpt2lp runtimee test (#2113)	2 years ago
Jiarui Fang	9214d1fe28	[Gemini] chunk init using runtime visited param order (#2115)	2 years ago
HELSON	e7d3afc9cc	[optimizer] add div_scale for optimizers (#2117) ... * [optimizer] add div_scale for optimizers * [zero] use div_scale in zero optimizer * fix testing error	2 years ago
Jiarui Fang	e5aa8333e4	[NFC] update chunk manager API (#2119)	2 years ago
Jiarui Fang	e99edfcb51	[NFC] polish comments for Chunk class (#2116)	2 years ago
Ziyue Jiang	09d69e1c25	[PP Middleware] Add bwd and step for PP middleware (#2111) ... * add bwd and step for PP middleware * pre-commit Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
HELSON	63fbba3c19	[zero] add L2 gradient clipping for ZeRO (#2112) ... * [zero] add L2 gradient clipping * [testing] add MlpModel * [zero] add unit test for grad clipping * fix atol	2 years ago
Jiarui Fang	70a8556946	[gemini] get the param visited order during runtime (#2108)	2 years ago
YuliangLiu0306	d87baa85d9	[autoparallel] support linear function bias addition (#2104)	2 years ago
YuliangLiu0306	0fecbb9e20	[autoparallel] support addbmm computation (#2102)	2 years ago
YuliangLiu0306	d3d4630495	[autoparallel] add sum handler (#2101)	2 years ago
Ziyue Jiang	e4705ba4e2	[Pipeline Middleware] fix data race in Pipeline Scheduler for DAG (#2087) ... * add DAG test case * fix datarace by adjusting theposition of lock * polish code * fix pytest for middleware * remove test Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
YuliangLiu0306	b175e6d58e	[autoparallel] add bias addtion function class (#2098) ... * [autoparallel] add bias addtion function class * polish code * polish	2 years ago
YuliangLiu0306	3af7e65dea	[autoparallel] complete gpt related module search (#2097)	2 years ago
Jiarui Fang	85efb7ac2e	[Gemini] gemini use the runtime memory tracer (RMT) (#2099)	2 years ago