ColossalAI

Commit Graph

Author	SHA1	Message	Date
Kun Lin	8af29ee47a	[shardformer] support vision transformer (#4096 ) * first v of vit shardformer * keep vit * update * vit shard add vitattention vitlayer * update num head shard para * finish test for vit * add new_model_class & postprocess * add vit readme * delete old files & fix the conflict * fix sth	2023-07-04 16:05:01 +08:00
Frank Lee	d33a44e8c3	[shardformer] refactored layernorm (#4086 )	2023-07-04 16:05:01 +08:00
FoolPlayer	92f6791095	[shardformer] Add layernorm (#4072 ) * add layernorm to bert * add layernorm test * add layernorm test with load state dict * add use_mixedfusedLN in shard config * refactor policy to support fused_layernorm	2023-07-04 16:05:01 +08:00

Author

SHA1

Message

Date

Kun Lin

8af29ee47a

[shardformer] support vision transformer (#4096 )

* first v of vit shardformer

* keep vit

* update

* vit shard add vitattention vitlayer

* update num head shard para

* finish test for vit

* add new_model_class & postprocess

* add vit readme

* delete old files & fix the conflict

* fix sth

2023-07-04 16:05:01 +08:00

Frank Lee

d33a44e8c3

[shardformer] refactored layernorm (#4086 )

2023-07-04 16:05:01 +08:00

FoolPlayer

92f6791095

[shardformer] Add layernorm (#4072 )

* add layernorm to bert

* add layernorm test

* add layernorm test with load state dict

* add use_mixedfusedLN in shard config

* refactor policy to support fused_layernorm

2023-07-04 16:05:01 +08:00

3 Commits (44a190e6ac71976304838febcffe82fb61941926)