7 Commits (fae6c92eadafc7c700af5dd5bec9553d03c81550)

Author	SHA1	Message	Date
Hongxin Liu	d921ce8391	[shardformer] support inplace sharding (#4251) ... * [shardformer] embedding support inplace sharding * [shardformer] linear support inplace sharding * [shardformer] layernorm support inplace sharding * [shardformer] qkv support inplace sharding * [test] update shardformer layer test * [shardformer] fix shared param sharding * [shardformer] fix bert policy * [shardformer] fix bloom policy * [shardformer] fix llama policy * [shardformer] fix opt policy * [shardformer] fix t5 policy * [shardformer] fix fused qkv linear * [shardformer] fix bugs * force sync * [test] fix bugs * [test] fix transformer version	1 year ago
Hongxin Liu	890774b2fb	[shardformer] support lazy init (#4202) ... * [shardformer] support lazy init * [shardformer] linear support lazy init * [shardformer] embedding support lazy init * [shardformer] norm support lazy init * [shardformer] fused linear support lazy init * [test] update shardformer test layer * [test] shardformer with lazy init fit ddp * [lazy] hotfix deepcopy of param * [shardformer] fix bert policy and update test * [shardformer] fix bloom policy and update test * [shardformer] fix opt policy and update test * [shardformer] fix t5 policy and update test * [shardformer] fix gpt2 policy and update test * [shardformer] fix llama policy and update test	1 year ago
Jianghai	f3bcc292c8	[pipeline] move bert related pipeline components to shardformer (#4187) ... * move bert related pipeline components to shardformer * fix bugs * revision * fix bert model tests * fix bert_lm_head model tests * fix tests * fix tests * done checks * skip bloom	1 year ago
github-actions[bot]	c77b3b19be	[format] applied code formatting on changed files in pull request 4152 (#4157) ... Co-authored-by: github-actions <github-actions@github.com>	1 year ago
Kun Lin	8af29ee47a	[shardformer] support vision transformer (#4096) ... * first v of vit shardformer * keep vit * update * vit shard add vitattention vitlayer * update num head shard para * finish test for vit * add new_model_class & postprocess * add vit readme * delete old files & fix the conflict * fix sth	1 year ago
Frank Lee	d33a44e8c3	[shardformer] refactored layernorm (#4086)	1 year ago
FoolPlayer	92f6791095	[shardformer] Add layernorm (#4072) ... * add layernorm to bert * add layernorm test * add layernorm test with load state dict * add use_mixedfusedLN in shard config * refactor policy to support fused_layernorm	1 year ago