github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
3,444 Commits
37 Branches
48 Tags
232 MiB
Commit Graph

6 Commits (d9d5e7ea1f8f92cbefd6ebb1ceefc554e5ba0fd6)

Author SHA1 Message Date
Hongxin Liu 7f8b16635b
[misc] refactor launch API and tensor constructor (#5666) 
* [misc] remove config arg from initialize

* [misc] remove old tensor contrusctor

* [plugin] add npu support for ddp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [devops] fix doc test ci

* [test] fix test launch

* [doc] update launch doc

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-04-29 10:40:11 +08:00
Baizhou Zhang 21ba89cab6
[gemini] support gradient accumulation (#4869) 
* add test

* fix no_sync bug in low level zero plugin

* fix test

* add argument for grad accum

* add grad accum in backward hook for gemini

* finish implementation, rewrite tests

* fix test

* skip stuck model in low level zero test

* update doc

* optimize communication & fix gradient checkpoint

* modify doc

* cleaning codes

* update cpu adam fp16 case
 2023-10-17 14:07:21 +08:00
Hongxin Liu 66f3926019
[doc] clean up outdated docs (#4765) 
* [doc] clean up outdated docs

* [doc] fix linking

* [doc] fix linking
 2023-09-21 11:36:20 +08:00
flybird1111 f40b718959
[doc] Fix gradient accumulation doc. (#4349) 
* [doc] fix gradient accumulation doc

* [doc] fix gradient accumulation doc
 2023-08-04 17:24:35 +08:00
jiangmingyan 278fcbc444 [doc]fix 2023-05-23 17:53:11 +08:00
jiangmingyan ef02d7ef6d
[doc] update gradient accumulation (#3771) 
* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, add sidebars

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, fix

* [doc]update gradient accumulation, resolve comments

* [doc]update gradient accumulation, resolve comments

* fix
 2023-05-23 10:52:30 +08:00