ColossalAI

Commit Graph

Author	SHA1	Message	Date
YuliangLiu0306	2731531bc2	[autoparallel] integrate device mesh initialization into autoparallelize (#2393 ) * [autoparallel] integrate device mesh initialization into autoparallelize * add megatron solution * update gpt autoparallel examples with latest api * adapt beta value to fit the current computation cost	2023-01-11 14:03:49 +08:00
YuliangLiu0306	b5a3a4a65f	[device] find best logical mesh	2023-01-05 17:21:29 +08:00
YuliangLiu0306	9c9246c0d9	[device] alpha beta profiler (#2311 ) * [device] alpha beta profiler * add usage * fix variable name	2023-01-05 16:39:55 +08:00
YuliangLiu0306	677e1e20d4	[device] update flatten device mesh usage (#2079 )	2022-12-05 16:16:07 +08:00
Genghan Zhang	d655eea515	[autoparallel] mix gather (#1977 ) * Add mix-gather * Add comments * Add comments * Polish comments * Change the global rank assumption * Add tests * Add two-step tests * Fix 10 and 01 * Skip test becasue the number of GPUs	2022-11-23 21:49:17 +08:00
Genghan Zhang	6630d45546	[autoparallel] Add alpha beta (#1973 ) * Add alpha beta * Fix test * Fix test	2022-11-17 16:01:14 +08:00
YuliangLiu0306	b4cc59b61e	[autoparallel] add numerical test for node strategies (#1760 ) * [autoparallel] add numerical test for node strategies * polish code * polish code	2022-10-27 10:42:54 +08:00
YuliangLiu0306	4b03c25f85	[tensor]add 1D device mesh (#1492 )	2022-08-25 16:48:12 +08:00
YuliangLiu0306	b73fb7a077	[tensor] support runtime ShardingSpec apply (#1453 ) * [tensor] support runtime ShardingSpec apply * polish code * polish code	2022-08-19 13:39:51 +08:00
YuliangLiu0306	0442f940f0	[device] add DeviceMesh class to support logical device layout (#1394 ) * [device] add DeviceMesh class to support logical device layout * polish code * add doc string	2022-08-02 19:23:48 +08:00

10 Commits (8b7495dd541ea12e1af84b3a3a0e24abc1e847d1)