769cddcb2c
fix typo docs/ ( #4033 )
2023-06-28 15:30:30 +08:00
4da324cd60
[hotfix]fix argument naming in docs and examples ( #4083 )
2023-06-26 23:50:04 +08:00
ddcf58cacf
Revert "[sync] sync feature/shardformer with develop"
2023-06-09 09:41:27 +08:00
24651fdd4f
Merge pull request #3931 from FrankLeeeee/sync/develop-to-shardformer
...
[sync] sync feature/shardformer with develop
2023-06-09 09:34:00 +08:00
33eef714db
fix typo examples and docs ( #3932 )
2023-06-08 16:09:32 +08:00
12c90db3f3
[doc] add lazy init tutorial ( #3922 )
...
* [doc] add lazy init en doc
* [doc] add lazy init zh doc
* [doc] add lazy init doc in sidebar
* [doc] add lazy init doc test
* [doc] fix lazy init doc link
2023-06-07 17:59:58 +08:00
c1535ccbba
[doc] fix docs about booster api usage ( #3898 )
2023-06-06 13:36:11 +08:00
07cb21142f
[doc]update moe chinese document. ( #3890 )
...
* [doc]update-moe
* [doc]update-moe
* [doc]update-moe
* [doc]update-moe
* [doc]update-moe
2023-06-05 15:57:54 +08:00
281b33f362
[doc] update document of zero with chunk. ( #3855 )
...
* [doc] fix title of mixed precision
* [doc]update document of zero with chunk
* [doc] update document of zero with chunk, fix
* [doc] update document of zero with chunk, fix
* [doc] update document of zero with chunk, fix
* [doc] update document of zero with chunk, add doc test
* [doc] update document of zero with chunk, add doc test
* [doc] update document of zero with chunk, fix installation
* [doc] update document of zero with chunk, fix zero with chunk doc
* [doc] update document of zero with chunk, fix zero with chunk doc
2023-05-30 18:41:56 +08:00
b0474878bf
[doc] update nvme offload documents. ( #3850 )
2023-05-26 01:22:01 +08:00
a64df3fa97
[doc] update document of gemini instruction. ( #3842 )
...
* [doc] update meet_gemini.md
* [doc] update meet_gemini.md
* [doc] fix parentheses
* [doc] fix parentheses
* [doc] fix doc test
* [doc] fix doc test
* [doc] fix doc
2023-05-25 14:58:01 +08:00
54e97ed7ea
[workflow] supported test on CUDA 10.2 ( #3841 )
2023-05-25 14:14:34 +08:00
3229f93e30
[booster] add warning for torch fsdp plugin doc ( #3833 )
2023-05-25 14:00:02 +08:00
518b31c059
[docs] change placememt_policy to placement_policy ( #3829 )
...
* fix typo colossalai/autochunk auto_parallel amp
* fix typo colossalai/auto_parallel nn utils etc.
* fix typo colossalai/auto_parallel autochunk fx/passes etc.
* fix typo docs/
* change placememt_policy to placement_policy in docs/ and examples/
2023-05-24 14:51:49 +08:00
e90fdb1000
fix typo docs/
2023-05-24 13:57:43 +08:00
725365f297
Merge pull request #3810 from jiangmingyan/amp
...
[doc] update amp document
2023-05-23 18:58:16 +08:00
278fcbc444
[doc]fix
2023-05-23 17:53:11 +08:00
8aa1fb2c7f
[doc]fix
2023-05-23 17:50:30 +08:00
19d153057e
[doc] add warning about fsdp plugin ( #3813 )
2023-05-23 17:16:10 +08:00
c425a69d52
[doc] add removed change of config.py
2023-05-23 16:42:36 +08:00
75272ef37b
[doc] add removed warning
2023-05-23 16:34:30 +08:00
a520610bd9
[doc] update amp document
2023-05-23 16:20:29 +08:00
1167bf5b10
[doc] update amp document
2023-05-23 16:20:17 +08:00
8c62e50dbb
[doc] update amp document
2023-05-23 16:20:01 +08:00
ef02d7ef6d
[doc] update gradient accumulation ( #3771 )
...
* [doc]update gradient accumulation
* [doc]update gradient accumulation
* [doc]update gradient accumulation
* [doc]update gradient accumulation
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, add sidebars
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, fix
* [doc]update gradient accumulation, resolve comments
* [doc]update gradient accumulation, resolve comments
* fix
2023-05-23 10:52:30 +08:00
62c7e67f9f
[format] applied code formatting on changed files in pull request 3786 ( #3787 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-05-22 14:42:09 +08:00
fe1561a884
[doc] update gradient cliping document ( #3778 )
...
* [doc] update gradient clipping document
* [doc] update gradient clipping document
* [doc] update gradient clipping document
* [doc] update gradient clipping document
* [doc] update gradient clipping document
* [doc] update gradient clipping document
* [doc] update gradient clipping doc, fix sidebars.json
* [doc] update gradient clipping doc, fix doc test
2023-05-22 14:13:15 +08:00
d9393b85f1
[doc] add deprecated warning on doc Basics section ( #3754 )
...
* Update colotensor_concept.md
* Update configure_parallelization.md
* Update define_your_config.md
* Update engine_trainer.md
* Update initialize_features.md
* Update model_checkpoint.md
* Update colotensor_concept.md
* Update configure_parallelization.md
* Update define_your_config.md
* Update engine_trainer.md
* Update initialize_features.md
* Update model_checkpoint.md
2023-05-22 11:12:53 +08:00
72688adb2f
[doc] add booster docstring and fix autodoc ( #3789 )
...
* [doc] add docstr for booster methods
* [doc] fix autodoc
2023-05-22 10:56:47 +08:00
60e6a154bc
[doc] add tutorial for booster checkpoint ( #3785 )
...
* [doc] add checkpoint related docstr for booster
* [doc] add en checkpoint doc
* [doc] add zh checkpoint doc
* [doc] add booster checkpoint doc in sidebar
* [doc] add cuation about ckpt for plugins
* [doc] add doctest placeholder
* [doc] add doctest placeholder
* [doc] add doctest placeholder
2023-05-19 18:05:08 +08:00
ad2cf58f50
[chat] add performance and tutorial ( #3786 )
2023-05-19 18:03:56 +08:00
21e29e2212
[doc] add tutorial for booster plugins ( #3758 )
...
* [doc] add en booster plugins doc
* [doc] add booster plugins doc in sidebar
* [doc] add zh booster plugins doc
* [doc] fix zh booster plugin translation
* [doc] reoganize tutorials order of basic section
* [devops] force sync to test ci
2023-05-19 12:12:42 +08:00
5ce6c9d86f
[doc] add tutorial for cluster utils ( #3763 )
...
* [doc] add en cluster utils doc
* [doc] add zh cluster utils doc
* [doc] add cluster utils doc in sidebar
2023-05-19 12:12:20 +08:00
48bd056761
[doc] update hybrid parallelism doc ( #3770 )
2023-05-18 14:16:13 +08:00
d449525acf
[doc] update booster tutorials ( #3718 )
...
* [booster] update booster tutorials#3717
* [booster] update booster tutorials#3717, fix
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, update setup doc
* [booster] update booster tutorials#3717, rename colossalai booster.md
* [booster] update booster tutorials#3717, rename colossalai booster.md
* [booster] update booster tutorials#3717, rename colossalai booster.md
* [booster] update booster tutorials#3717, fix
* [booster] update booster tutorials#3717, fix
* [booster] update tutorials#3717, update booster api doc
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, modify file
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3717, fix reference link
* [booster] update tutorials#3713
* [booster] update tutorials#3713, modify file
2023-05-18 11:41:56 +08:00
5dd573c6b6
[devops] fix ci for document check ( #3751 )
...
* [doc] add test info
* [devops] update doc check ci
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] add debug info
* [devops] remove debug info and update invalid doc
* [devops] add essential comments
2023-05-17 11:24:22 +08:00
b9a8dff7e5
[doc] Fix typo under colossalai and doc( #3618 )
...
* Fixed several spelling errors under colossalai
* Fix the spelling error in colossalai and docs directory
* Cautious Changed the spelling error under the example folder
* Update runtime_preparation_pass.py
revert autograft to autograd
* Update search_chunk.py
utile to until
* Update check_installation.py
change misteach to mismatch in line 91
* Update 1D_tensor_parallel.md
revert to perceptron
* Update 2D_tensor_parallel.md
revert to perceptron in line 73
* Update 2p5D_tensor_parallel.md
revert to perceptron in line 71
* Update 3D_tensor_parallel.md
revert to perceptron in line 80
* Update README.md
revert to resnet in line 42
* Update reorder_graph.py
revert to indice in line 7
* Update p2p.py
revert to megatron in line 94
* Update initialize.py
revert to torchrun in line 198
* Update routers.py
change to detailed in line 63
* Update routers.py
change to detailed in line 146
* Update README.md
revert random number in line 402
2023-04-26 11:38:43 +08:00
9edeadfb24
[doc] Update 1D_tensor_parallel.md ( #3573 )
...
Display format optimization , same as fix#3562
Simultaneous modification of en version
2023-04-17 12:19:53 +08:00
1c7734bc94
[doc] Update 1D_tensor_parallel.md ( #3563 )
...
Display format optimization, fix bug#3562
Specific changes
1. "This is called a column-parallel fashion" Translate to Chinese
2. use the ```math code block syntax to display a math expression as a block, No modification of formula content
Please check that the math formula is displayed correctly
If OK, I will change the format of the English version of the formula in parallel
2023-04-14 22:12:32 +08:00
a3ac48ef3d
[doc] Update README-zh-Hans.md ( #3541 )
...
Fixing document link errors using absolute paths
2023-04-12 23:09:30 +08:00
0c0455700f
[doc] add requirement and highlight application ( #3516 )
...
* [doc] add requirement and highlight application
* [doc] link example and application
2023-04-10 17:37:16 +08:00
4e9989344d
[doc] updated contributor list ( #3474 )
2023-04-06 17:47:59 +08:00
80eba05b0a
[test] refactor tests with spawn ( #3452 )
...
* [test] added spawn decorator
* polish code
* polish code
* polish code
* polish code
* polish code
* polish code
2023-04-06 14:51:35 +08:00
26b7aac0be
[zero] reorganize zero/gemini folder structure ( #3424 )
...
* [zero] refactor low-level zero folder structure
* [zero] fix legacy zero import path
* [zero] fix legacy zero import path
* [zero] remove useless import
* [zero] refactor gemini folder structure
* [zero] refactor gemini folder structure
* [zero] refactor legacy zero import path
* [zero] refactor gemini folder structure
* [zero] refactor gemini folder structure
* [zero] refactor gemini folder structure
* [zero] refactor legacy zero import path
* [zero] fix test import path
* [zero] fix test
* [zero] fix circular import
* [zero] update import
2023-04-04 13:48:16 +08:00
15a74da79c
[doc] add Intel cooperation news ( #3333 )
...
* [doc] add Intel cooperation news
* [doc] add Intel cooperation news
2023-03-30 11:45:01 +08:00
31c78f2be3
[doc] add ColossalChat news ( #3304 )
...
* [doc] add ColossalChat news
* [doc] add ColossalChat news
2023-03-29 09:27:55 +08:00
682af61396
[doc] add ColossalChat ( #3297 )
...
* [doc] add ColossalChat
2023-03-29 02:35:10 +08:00
20d1c99444
[refactor] update docs ( #3174 )
...
* refactor: README-zh-Hans
* refactor: REFERENCE
* docs: update paths in README
2023-03-20 10:52:01 +08:00
3213347b49
[doc] fixed typos in docs/README.md ( #3082 )
2023-03-10 10:32:14 +08:00
416a50dbd7
[doc] moved doc test command to bottom ( #3075 )
2023-03-09 18:10:45 +08:00
ea0b52c12e
[doc] specified operating system requirement ( #3019 )
...
* [doc] specified operating system requirement
* polish code
2023-03-07 18:04:10 +08:00
378d827c6b
[doc] update nvme offload doc ( #3014 )
...
* [doc] update nvme offload doc
* [doc] add doc testing cmd and requirements
* [doc] add api reference
* [doc] add dependencies
2023-03-07 17:49:01 +08:00
8fedc8766a
[workflow] supported conda package installation in doc test ( #3028 )
...
* [workflow] supported conda package installation in doc test
* polish code
* polish code
* polish code
* polish code
* polish code
* polish code
2023-03-07 14:21:26 +08:00
e0a1c1321c
[doc] added reference to related works ( #2994 )
...
* [doc] added reference to related works
* polish code
2023-03-04 17:32:22 +08:00
dca98937f8
[format] applied code formatting on changed files in pull request 2933 ( #2939 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-02-28 15:41:52 +08:00
8264cd7ef1
[doc] add env scope ( #2933 )
2023-02-28 15:39:51 +08:00
b8804aa60c
[doc] added readme for documentation ( #2935 )
2023-02-28 14:04:52 +08:00
9e3b8b7aff
[doc] removed read-the-docs ( #2932 )
2023-02-28 11:28:24 +08:00
77b88a3849
[workflow] added auto doc test on PR ( #2929 )
...
* [workflow] added auto doc test on PR
* [workflow] added doc test workflow
* polish code
* polish code
* polish code
* polish code
* polish code
* polish code
* polish code
2023-02-28 11:10:38 +08:00
0afb55fc5b
[doc] add os scope, update tutorial install and tips ( #2914 )
2023-02-27 14:59:27 +08:00
cf6409dd40
Hotfix/auto parallel zh doc ( #2820 )
...
* [hotfix] fix autoparallel zh docs
* polish
* polish
2023-02-19 15:57:14 +08:00
2059fdd6b0
[hotfix] add copyright for solver and device mesh ( #2803 )
...
* [hotfix] add copyright for solver and device mesh
* add readme
* add alpa license
* polish
2023-02-18 21:14:38 +08:00
e376954305
[doc] add opt service doc ( #2747 )
2023-02-16 15:45:26 +08:00
5479fdd5b8
[doc] updated documentation version list ( #2730 )
2023-02-15 17:39:50 +08:00
2045d45ab7
[doc] updated documentation version list ( #2715 )
2023-02-15 11:24:18 +08:00
0966008839
[dooc] fixed the sidebar itemm key ( #2672 )
2023-02-13 10:45:16 +08:00
6d60634433
[doc] added documentation sidebar translation ( #2670 )
2023-02-13 10:10:12 +08:00
81ea66d25d
[release] v0.2.3 ( #2669 )
...
* [release] v0.2.3
* polish code
2023-02-13 09:51:25 +08:00
8de85051b3
[Docs] layout converting management ( #2665 )
2023-02-10 18:38:32 +08:00
b673e5f78b
[release] v0.2.2 ( #2661 )
2023-02-10 11:01:24 +08:00
cd4f02bed8
[doc] fixed compatiblity with docusaurus ( #2657 )
2023-02-09 17:06:29 +08:00
a4ae43f071
[doc] added docusaurus-based version control ( #2656 )
2023-02-09 16:38:49 +08:00
85b2303b55
[doc] migrate the markdown files ( #2652 )
2023-02-09 14:21:38 +08:00
d3480396f8
[doc] updated the sphinx theme ( #2635 )
2023-02-08 13:48:08 +08:00
a01278e810
Update requirements.txt
2022-11-18 18:57:18 +08:00
cc0ed7cf33
[Gemini] ZeROHookV2 -> GeminiZeROHook ( #1972 )
2022-11-17 14:43:49 +08:00
63f250bbd4
fix file name ( #1759 )
...
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
2022-10-25 16:48:48 +08:00
d068af81a3
[doc] update rst and docstring ( #1351 )
...
* update rst
* add zero docstr
* fix docstr
* remove fx.tracer.meta_patch
* fix docstr
* fix docstr
* update fx rst
* fix fx docstr
* remove useless rst
2022-07-21 15:54:53 +08:00
4165eabb1e
[hotfix] remove potiential circle import ( #1307 )
...
* make it faster
* [hotfix] remove circle import
2022-07-14 13:44:26 +08:00
4d9332b4c5
[refactor] moving memtracer to gemini ( #801 )
2022-04-19 10:13:08 +08:00
f69507dd22
update rst ( #615 )
2022-04-01 15:46:38 +08:00
2c45efc398
html refactor ( #555 )
2022-03-31 11:36:56 +08:00
c44d797072
[docs] updatad docs of hybrid adam and cpu adam ( #552 )
2022-03-30 18:14:59 +08:00
ffca99d187
[doc] update apidoc ( #530 )
2022-03-25 18:29:43 +08:00
9caa8b6481
docs get correct release version ( #489 )
2022-03-22 14:24:41 +08:00
7e30068a22
[doc] update rst ( #470 )
...
* update rst
* remove empty rst
2022-03-21 10:52:45 +08:00
ce7b2c9ae3
update README and images path ( #384 )
2022-03-11 15:50:28 +08:00
08eccfe681
add community group and update issue template( #271 )
2022-03-11 15:50:28 +08:00
3312d716a0
update experimental visualization ( #253 )
2022-03-11 15:50:28 +08:00
753035edd3
add Chinese README
2022-03-11 15:50:28 +08:00
6fb550acdb
update logo
2022-01-21 12:31:07 +08:00
1949d3a889
update doc requirements and rtd conf ( #165 )
2022-01-19 19:46:43 +08:00
be85a0f366
removed tutorial markdown and refreshed rst files for consistency
2022-01-19 17:01:37 +08:00
17ce8569a8
add logo at homepage, add forum in issue template ( #161 )
2022-01-19 14:29:31 +08:00
9473a1b9c8
AMP docstring/markdown update ( #160 )
2022-01-18 18:33:36 +08:00
96780e6ee4
Optimize pipeline schedule ( #94 )
...
* add pipeline shared module wrapper and update load batch
* added model parallel process group for amp and clip grad (#86 )
* added model parallel process group for amp and clip grad
* update amp and clip with model parallel process group
* remove pipeline_prev/next group (#88 )
* micro batch offload
* optimize pipeline gpu memory usage
* pipeline can receive tensor shape (#93 )
* optimize pipeline gpu memory usage
* fix grad accumulation step counter
* rename classes and functions
Co-authored-by: Frank Lee <somerlee.9@gmail.com>
2021-12-30 15:56:46 +08:00
8f02a88db2
add interleaved pipeline, fix naive amp and update pipeline model initializer ( #80 )
2021-12-20 23:26:19 +08:00
35813ed3c4
update examples and sphnix docs for the new api ( #63 )
2021-12-13 22:07:01 +08:00
7d3711058f
fix zero3 fp16 and add zero3 model context ( #62 )
2021-12-10 17:48:50 +08:00
9a0466534c
update markdown docs (english) ( #60 )
2021-12-10 14:37:33 +08:00
digger yu