ColossalAI

Commit Graph

Author	SHA1	Message	Date
ver217	62b0a8d644	[zero] sharded optim support hybrid cpu adam (#486 ) * sharded optim support hybrid cpu adam * update unit test * polish docstring	2022-03-22 14:56:59 +08:00
Jiarui Fang	b334822163	[zero] polish sharded param name (#484 ) * [zero] polish sharded param name * polish code * polish * polish code * polish * polsih * polish	2022-03-22 14:36:16 +08:00
ver217	9506a8beb2	use double buffer to handle grad	2022-03-16 14:24:09 +08:00
Jiarui Fang	b5f43acee3	[zero] find miss code (#378 )	2022-03-11 15:50:28 +08:00
jiaruifang	d9217e1960	Revert "[zero] bucketized tensor cpu gpu copy (#368 )" This reverts commit `bef05489b6`.	2022-03-11 15:50:28 +08:00
Jiarui Fang	00670c870e	[zero] bucketized tensor cpu gpu copy (#368 )	2022-03-11 15:50:28 +08:00
Jiarui Fang	44e4891f57	[zero] able to place params on cpu after zero init context (#365 ) * place params on cpu after zero init context * polish code	2022-03-11 15:50:28 +08:00
Jiarui Fang	ea2872073f	[zero] global model data memory tracer (#360 )	2022-03-11 15:50:28 +08:00
ver217	1388671699	[zero] Update sharded model v2 using sharded param v2 (#323 )	2022-03-11 15:50:28 +08:00
Jiarui Fang	11bddb6e55	[zero] update zero context init with the updated test utils (#327 )	2022-03-11 15:50:28 +08:00
Jiarui Fang	de0468c7a8	[zero] zero init context (#321 ) * add zero init context * add more flags for zero init context fix bug of repeated converting param to ShardedParamV2 * polish code	2022-03-11 15:50:28 +08:00
Jiarui Fang	90d3aef62c	[zero] yet an improved sharded param (#311 )	2022-03-11 15:50:28 +08:00
Jiarui Fang	c9e7d9582d	[zero] polish shard strategy (#310 ) * init shard param from shape tuple * add more unitest for shard param * add set_payload method for ShardedParam * [zero] add shareded tensor class * polish code * add shard stratgy * move shard and gather logic to shard strategy from shard tensor. * polish code	2022-03-11 15:50:28 +08:00
ver217	36f9a74ab2	fix sharded param hook and unit test	2022-03-11 15:50:28 +08:00
Jiarui Fang	80364c7686	[zero] sharded tensor (#305 ) * init shard param from shape tuple * add more unitest for shard param * add set_payload method for ShardedParam * [zero] add shareded tensor class * polish code	2022-03-11 15:50:28 +08:00
Jiarui Fang	e17e92c54d	Polish sharded parameter (#297 ) * init shard param from shape tuple * add more unitest for shard param * add more unittests to shareded param	2022-03-11 15:50:28 +08:00

16 Commits (f24b5ed20135e3b8c53ef07fc0c60e26104b7e5a)