ColossalAI

Commit Graph

Author	SHA1	Message	Date
アマデウス	b8899e0905	[TP] allow layernorm without bias (#750 )	2022-04-14 11:43:56 +08:00
HELSON	dceae85195	Added MoE parallel (#127 )	2022-01-07 15:08:36 +08:00
アマデウス	0fedef4f3c	Layer integration (#83 ) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	2021-12-27 15:04:32 +08:00

Author

SHA1

Message

Date

アマデウス

b8899e0905

[TP] allow layernorm without bias (#750 )

2022-04-14 11:43:56 +08:00

HELSON

dceae85195

Added MoE parallel (#127 )

2022-01-07 15:08:36 +08:00

アマデウス

0fedef4f3c

Layer integration (#83 )

* integrated parallel layers for ease of building models

* integrated 2.5d layers

* cleaned codes and unit tests

* added log metric by step hook; updated imagenet benchmark; fixed some bugs

* reworked initialization; cleaned codes

Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>

2021-12-27 15:04:32 +08:00

3 Commits (4d8a574cd3beb18e497d854e599f2899bc8afa70)