ColossalAI

Commit Graph

Author	SHA1	Message	Date
Baizhou Zhang	d99b2c961a	[hotfix] fix grad accumulation plus clipping for gemini (#5002 )	2023-11-02 17:59:10 +08:00
Hongxin Liu	b8e770c832	[test] merge old components to test to model zoo (#4945 ) * [test] add custom models in model zoo * [test] update legacy test * [test] update model zoo * [test] update gemini test * [test] remove components to test	2023-10-20 10:35:08 +08:00
Baizhou Zhang	21ba89cab6	[gemini] support gradient accumulation (#4869 ) * add test * fix no_sync bug in low level zero plugin * fix test * add argument for grad accum * add grad accum in backward hook for gemini * finish implementation, rewrite tests * fix test * skip stuck model in low level zero test * update doc * optimize communication & fix gradient checkpoint * modify doc * cleaning codes * update cpu adam fp16 case	2023-10-17 14:07:21 +08:00

Author

SHA1

Message

Date

Baizhou Zhang

d99b2c961a

[hotfix] fix grad accumulation plus clipping for gemini (#5002 )

2023-11-02 17:59:10 +08:00

Hongxin Liu

b8e770c832

[test] merge old components to test to model zoo (#4945 )

* [test] add custom models in model zoo

* [test] update legacy test

* [test] update model zoo

* [test] update gemini test

* [test] remove components to test

2023-10-20 10:35:08 +08:00

Baizhou Zhang

21ba89cab6

[gemini] support gradient accumulation (#4869 )

* add test

* fix no_sync bug in low level zero plugin

* fix test

* add argument for grad accum

* add grad accum in backward hook for gemini

* finish implementation, rewrite tests

* fix test

* skip stuck model in low level zero test

* update doc

* optimize communication & fix gradient checkpoint

* modify doc

* cleaning codes

* update cpu adam fp16 case

2023-10-17 14:07:21 +08:00

3 Commits (8d56c9c3894b882c94810a575bb3eb13fe7e2a98)