35 Commits (3abf98a6337ae39f11b3c259a0af8d40477fe7f7)

Author	SHA1	Message	Date
BigOneLiXiaoMing	0c4c9aa6e0	[NFC] polish colossalai/nn/_ops/embedding.py code style (#1561)	2022-09-08 22:11:04 +08:00
yuxuan-lou	413f9c19f4	[NFC] polish colossalai/nn/_ops/layernorm.py code style (#1555)	2022-09-08 22:11:04 +08:00
Maruyama_Aya	bd2d789832	[NFC] polish colossalai/nn/_ops/embedding_bag.py code style (#1552)	2022-09-08 22:11:04 +08:00
Jiarui Fang	87134524fd	[embedding] tablewise sharding polish (#1535)	2022-09-02 11:09:37 +08:00
Frank Lee	ae1b58cd16	[tensor] added linear implementation for the new sharding spec (#1416) ... * [tensor] added linear implementation for the new sharding spec * polish code	2022-08-12 11:33:09 +08:00
Jiarui Fang	30b4dd17c0	[FAW] export FAW in _ops (#1438)	2022-08-11 13:43:24 +08:00
Jiarui Fang	c9427a323f	hotfix #1434 (#1437)	2022-08-11 13:14:25 +08:00
Jiarui Fang	10b3df65c8	[FAW] move coloparam setting in test code. (#1429)	2022-08-10 14:31:53 +08:00
Jiarui Fang	cb98cf5558	[FAW] parallel FreqAwareEmbedding (#1424)	2022-08-10 13:44:30 +08:00
Jiarui Fang	d209aff684	Add FreqAwareEmbeddingBag (#1421)	2022-08-09 16:26:12 +08:00
Jiarui Fang	504419d261	[FAW] add cache manager for the cached embedding (#1419)	2022-08-09 15:17:17 +08:00
HELSON	7a8702c06d	[colotensor] add Tensor.view op and its unit test (#1343) ... [colotensor] add megatron initialization for gpt2	2022-07-21 10:53:15 +08:00
HELSON	260a55804a	[hotfix] fix shape error in backward when using ColoTensor (#1298)	2022-07-13 23:06:12 +08:00
HELSON	abba4d84e1	[hotfix] fix bert model test in unitests (#1272)	2022-07-12 23:26:45 +08:00
Jiarui Fang	1aad903c15	[tensor] redistribute among different process groups (#1247) ... * make it faster * [tensor] rename convert_to_dist -> redistribute * [tensor] ShardSpec and ReplicaSpec * [tensor] redistribute among diff pgs * polish code	2022-07-12 10:24:05 +08:00
Jiarui Fang	9bcd2fd4af	[tensor] a shorter shard and replicate spec (#1245)	2022-07-11 15:51:48 +08:00
Jiarui Fang	2699dfbbfd	[rename] convert_to_dist -> redistribute (#1243)	2022-07-11 13:05:44 +08:00
Jiarui Fang	4a76084dc9	[tensor] add zero_like colo op, important for Optimizer (#1236)	2022-07-08 14:55:27 +08:00
Jiarui Fang	3b500984b1	[tensor] fix some unittests (#1234)	2022-07-08 14:18:30 +08:00
HELSON	0453776def	[tensor] fix a assertion in colo_tensor cross_entropy (#1232)	2022-07-08 11:18:00 +08:00
HELSON	42ab36b762	[tensor] add unitest for colo_tensor 1DTP cross_entropy (#1230)	2022-07-07 19:17:23 +08:00
Jiarui Fang	a98319f023	[tensor] torch function return colotensor (#1229)	2022-07-07 18:09:18 +08:00
Jiarui Fang	ae7d3f4927	[refactor] move process group from _DistSpec to ColoTensor. (#1203)	2022-07-06 16:15:16 +08:00
Jiarui Fang	060b917daf	[refactor] remove gpc dependency in colotensor's _ops (#1189)	2022-07-04 18:54:37 +08:00
Jiarui Fang	1b657f9ce1	[tensor] revert local view back (#1178)	2022-06-27 18:38:34 +08:00
Jiarui Fang	0dd4e2bbfb	[Tensor] rename some APIs in TensorSpec and Polish view unittest (#1176)	2022-06-27 15:56:11 +08:00
Jiarui Fang	aa7bef73d4	[Tensor] distributed view supports inter-process hybrid parallel (#1169)	2022-06-27 09:45:26 +08:00
Jiarui Fang	4b9bba8116	[ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168)	2022-06-24 13:08:54 +08:00
Jiarui Fang	f4ef224358	[Tensor] remove ParallelAction, use ComputeSpec instread (#1166)	2022-06-23 17:34:59 +08:00
Jiarui Fang	177c374401	remove gather out in parallel action (#1163)	2022-06-23 16:35:05 +08:00
Jiarui Fang	07f9c781f9	[graph] improve the graph building. (#1157)	2022-06-22 16:47:20 +08:00
ver217	22717a856f	[tensor] add embedding bag op (#1156)	2022-06-22 15:54:03 +08:00
ver217	ae86151968	[tensor] add more element-wise ops (#1155) ... * add more element-wise ops * update test_op * polish unit test	2022-06-22 15:16:47 +08:00
ver217	ccf3c58c89	embedding op use gather_out (#1143)	2022-06-21 13:21:20 +08:00
Jiarui Fang	a00644079e	reorgnize colotensor directory (#1062) ... * reorgnize colotensor directory * polish code	2022-06-03 18:04:22 +08:00