70 Commits (e5b1a0c9bee8ba6cc1fe5af99afe725aec2b6509)

Author	SHA1	Message	Date
Jiarui Fang	b5f25eb32a	[Tensor] add cpu group to ddp (#1200)	2022-07-05 14:58:28 +08:00
Jiarui Fang	060b917daf	[refactor] remove gpc dependency in colotensor's _ops (#1189)	2022-07-04 18:54:37 +08:00
Jiarui Fang	372f791444	[refactor] move chunk and chunkmgr to directory gemini (#1182)	2022-06-29 13:31:02 +08:00
ver217	6b2f2ab9bb	[ddp] ColoDDP uses bucket all-reduce (#1177) ... * add reducer * update colo ddp with reducer * polish unit test * polish unit test	2022-06-29 10:34:13 +08:00
Ziyue Jiang	dd0420909f	[Tensor] rename parallel_action (#1174) ... * rename parallel_action * polish	2022-06-27 10:04:45 +08:00
Jiarui Fang	4b9bba8116	[ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168)	2022-06-24 13:08:54 +08:00
Jiarui Fang	f4ef224358	[Tensor] remove ParallelAction, use ComputeSpec instread (#1166)	2022-06-23 17:34:59 +08:00
ver217	54aabb8da4	[gemini] refactor gemini mgr (#1151) ... * refactor gemini mgr * udpate __init__	2022-06-22 11:54:36 +08:00
ver217	8106d7b8c7	[ddp] refactor ColoDDP and ZeroDDP (#1146) ... * ColoDDP supports overwriting default process group * rename ColoDDPV2 to ZeroDDP * add docstr for ZeroDDP * polish docstr	2022-06-21 16:35:23 +08:00
Frank Lee	15aab1476e	[zero] avoid zero hook spam by changing log to debug level (#1137)	2022-06-21 10:44:01 +08:00
ver217	d26902645e	[ddp] add save/load state dict for ColoDDP (#1127) ... * add save/load state dict for ColoDDP * add unit test * refactor unit test folder * polish unit test * rename unit test	2022-06-20 10:51:47 +08:00
ver217	f0a954f16d	[ddp] add set_params_to_ignore for ColoDDP (#1122) ... * add set_params_to_ignore for ColoDDP * polish code * fix zero hook v2 * add unit test * polish docstr	2022-06-16 12:54:46 +08:00
ver217	e127b4375b	cast colo ddp v2 inputs/outputs (#1120)	2022-06-15 15:57:04 +08:00
ver217	7d14b473f0	[gemini] gemini mgr supports "cpu" placement policy (#1118) ... * update gemini mgr * update chunk * add docstr * polish placement policy * update test chunk * update test zero * polish unit test * remove useless unit test	2022-06-15 15:05:19 +08:00
ver217	895c1c5ee7	[tensor] refactor param op hook (#1097) ... * refactor param op hook * add docstr * fix bug	2022-06-13 16:11:53 +08:00
Frank Lee	cb18922c47	[doc] added documentation to chunk and chunk manager (#1094) ... * [doc] added documentation to chunk and chunk manager * polish code * polish code * polish code	2022-06-10 15:33:06 +08:00
ver217	1f894e033f	[gemini] zero supports gemini (#1093) ... * add placement policy * add gemini mgr * update mem stats collector * update zero * update zero optim * fix bugs * zero optim monitor os * polish unit test * polish unit test * add assert	2022-06-10 14:48:28 +08:00
ver217	be01db37c8	[tensor] refactor chunk mgr and impl MemStatsCollectorV2 (#1077) ... * polish chunk manager * polish unit test * impl add_extern_static_tensor for chunk mgr * add mem stats collector v2 * polish code * polish unit test * polish code * polish get chunks	2022-06-09 20:56:34 +08:00
Ziyue Jiang	4fc748f69b	[Tensor] fix optimizer for CPU parallel (#1069)	2022-06-06 17:36:11 +08:00
Jiarui Fang	49832b2344	[refactory] add nn.parallel module (#1068)	2022-06-06 15:34:41 +08:00