192 Commits (main)

Author	SHA1	Message	Date
flybird11111	6a21f96a87	[doc] update advanced tutorials, training gpt with hybrid parallelism (#4866) ... * [doc]update advanced tutorials, training gpt with hybrid parallelism * [doc]update advanced tutorials, training gpt with hybrid parallelism * update vit tutorials * update vit tutorials * update vit tutorials * update vit tutorials * update en/train_vit_with_hybrid_parallel.py * fix * resolve comments * fix	1 year ago
Zhongkai Zhao	db40e086c8	[test] modify model supporting part of low_level_zero plugin (including correspoding docs)	1 year ago
binmakeswell	822051d888	[doc] update slack link (#4823)	1 year ago
Hongxin Liu	da15fdb9ca	[doc] add lazy init docs (#4808)	1 year ago
Baizhou Zhang	64a08b2dc3	[checkpointio] support unsharded checkpointIO for hybrid parallel (#4774) ... * support unsharded saving/loading for model * support optimizer unsharded saving * update doc * support unsharded loading for optimizer * small fix	1 year ago
Baizhou Zhang	a2db75546d	[doc] polish shardformer doc (#4779) ... * fix example format in docstring * polish shardformer doc	1 year ago
binmakeswell	d512a4d38d	[doc] add llama2 domain-specific solution news (#4789) ... * [doc] add llama2 domain-specific solution news	1 year ago
Baizhou Zhang	493a5efeab	[doc] add shardformer doc to sidebar (#4768)	1 year ago
Hongxin Liu	66f3926019	[doc] clean up outdated docs (#4765) ... * [doc] clean up outdated docs * [doc] fix linking * [doc] fix linking	1 year ago
Pengtai Xu	4d7537ba25	[doc] put native colossalai plugins first in description section	1 year ago
Pengtai Xu	e10d9f087e	[doc] add model examples for each plugin	1 year ago
Pengtai Xu	a04337bfc3	[doc] put individual plugin explanation in front	1 year ago
Pengtai Xu	10513f203c	[doc] explain suitable use case for each plugin	1 year ago
Hongxin Liu	b5f9e37c70	[legacy] clean up legacy code (#4743) ... * [legacy] remove outdated codes of pipeline (#4692) * [legacy] remove cli of benchmark and update optim (#4690) * [legacy] remove cli of benchmark and update optim * [doc] fix cli doc test * [legacy] fix engine clip grad norm * [legacy] remove outdated colo tensor (#4694) * [legacy] remove outdated colo tensor * [test] fix test import * [legacy] move outdated zero to legacy (#4696) * [legacy] clean up utils (#4700) * [legacy] clean up utils * [example] update examples * [legacy] clean up amp * [legacy] fix amp module * [legacy] clean up gpc (#4742) * [legacy] clean up context * [legacy] clean core, constants and global vars * [legacy] refactor initialize * [example] fix examples ci * [example] fix examples ci * [legacy] fix tests * [example] fix gpt example * [example] fix examples ci * [devops] fix ci installation * [example] fix examples ci	1 year ago
Baizhou Zhang	d151dcab74	[doc] explaination of loading large pretrained models (#4741)	1 year ago
Baizhou Zhang	451c3465fb	[doc] polish shardformer doc (#4735) ... * arrange position of chapters * fix typos in seq parallel doc	1 year ago
Bin Jia	6a03c933a0	[shardformer] update seq parallel document (#4730) ... * update doc of seq parallel * fix typo	1 year ago
flybird11111	46162632e5	[shardformer] update pipeline parallel document (#4725) ... * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document * [shardformer] update pipeline parallel document	1 year ago
Baizhou Zhang	50e5602c2d	[doc] add shardformer support matrix/update tensor parallel documents (#4728) ... * add compatibility matrix for shardformer doc * update tp doc	1 year ago
github-actions[bot]	8c2dda7410	[format] applied code formatting on changed files in pull request 4726 (#4727) ... Co-authored-by: github-actions <github-actions@github.com>	1 year ago
Baizhou Zhang	f911d5b09d	[doc] Add user document for Shardformer (#4702) ... * create shardformer doc files * add docstring for seq-parallel * update ShardConfig docstring * add links to llama example * add outdated massage * finish introduction & supporting information * finish 'how shardformer works' * finish shardformer.md English doc * fix doctest fail * add Chinese document	1 year ago
binmakeswell	ce97790ed7	[doc] fix llama2 code link (#4726) ... * [doc] fix llama2 code link * [doc] fix llama2 code link * [doc] fix llama2 code link	1 year ago
Baizhou Zhang	1d454733c4	[doc] Update booster user documents. (#4669) ... * update booster_api.md * update booster_checkpoint.md * update booster_plugins.md * move transformers importing inside function * fix Dict typing * fix autodoc bug * small fix	1 year ago
Hongxin Liu	554aa9592e	[legacy] move communication and nn to legacy and refactor logger (#4671) ... * [legacy] move communication to legacy (#4640) * [legacy] refactor logger and clean up legacy codes (#4654) * [legacy] make logger independent to gpc * [legacy] make optim independent to registry * [legacy] move test engine to legacy * [legacy] move nn to legacy (#4656) * [legacy] move nn to legacy * [checkpointio] fix save hf config * [test] remove useledd rpc pp test * [legacy] fix nn init * [example] skip tutorial hybriad parallel example * [devops] test doc check * [devops] test doc check	1 year ago
Hongxin Liu	ac178ca5c1	[legacy] move builder and registry to legacy (#4603)	1 year ago
Hongxin Liu	8accecd55b	[legacy] move engine to legacy (#4560) ... * [legacy] move engine to legacy * [example] fix seq parallel example * [example] fix seq parallel example * [test] test gemini pluging hang * [test] test gemini pluging hang * [test] test gemini pluging hang * [test] test gemini pluging hang * [test] test gemini pluging hang * [example] update seq parallel requirements	1 year ago
Hongxin Liu	89fe027787	[legacy] move trainer to legacy (#4545) ... * [legacy] move trainer to legacy * [doc] update docs related to trainer * [test] ignore legacy test	1 year ago
binmakeswell	7a978eb3d0	[DOC] hotfix/llama2news (#4595) ... * [doc] add llama2 news * [doc] add llama2 news * [doc] add llama2 news	1 year ago
Hongxin Liu	27061426f7	[gemini] improve compatibility and add static placement policy (#4479) ... * [gemini] remove distributed-related part from colotensor (#4379) * [gemini] remove process group dependency * [gemini] remove tp part from colo tensor * [gemini] patch inplace op * [gemini] fix param op hook and update tests * [test] remove useless tests * [test] remove useless tests * [misc] fix requirements * [test] fix model zoo * [test] fix model zoo * [test] fix model zoo * [test] fix model zoo * [test] fix model zoo * [misc] update requirements * [gemini] refactor gemini optimizer and gemini ddp (#4398) * [gemini] update optimizer interface * [gemini] renaming gemini optimizer * [gemini] refactor gemini ddp class * [example] update gemini related example * [example] update gemini related example * [plugin] fix gemini plugin args * [test] update gemini ckpt tests * [gemini] fix checkpoint io * [example] fix opt example requirements * [example] fix opt example * [example] fix opt example * [example] fix opt example * [gemini] add static placement policy (#4443) * [gemini] add static placement policy * [gemini] fix param offload * [test] update gemini tests * [plugin] update gemini plugin * [plugin] update gemini plugin docstr * [misc] fix flash attn requirement * [test] fix gemini checkpoint io test * [example] update resnet example result (#4457) * [example] update bert example result (#4458) * [doc] update gemini doc (#4468) * [example] update gemini related examples (#4473) * [example] update gpt example * [example] update dreambooth example * [example] update vit * [example] update opt * [example] update palm * [example] update vit and opt benchmark * [hotfix] fix bert in model zoo (#4480) * [hotfix] fix bert in model zoo * [test] remove chatglm gemini test * [test] remove sam gemini test * [test] remove vit gemini test * [hotfix] fix opt tutorial example (#4497) * [hotfix] fix opt tutorial example * [hotfix] fix opt tutorial example	1 year ago
binmakeswell	089c365fa0	[doc] add Series A Funding and NeurIPS news (#4377) ... * [doc] add Series A Funding and NeurIPS news * [kernal] fix mha kernal * [CI] skip moe * [CI] fix requirements	1 year ago
flybird1111	f40b718959	[doc] Fix gradient accumulation doc. (#4349) ... * [doc] fix gradient accumulation doc * [doc] fix gradient accumulation doc	1 year ago
Baizhou Zhang	c6f6005990	[checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin (#4302) ... * sharded optimizer checkpoint for gemini plugin * modify test to reduce testing time * update doc * fix bug when keep_gatherd is true under GeminiPlugin	1 year ago
binmakeswell	7ff11b5537	[example] add llama pretraining (#4257)	1 year ago
Jianghai	711e2b4c00	[doc] update and revise some typos and errs in docs (#4107) ... * fix some typos and problems in doc * fix some typos and problems in doc * add doc test	1 year ago
digger yu	769cddcb2c	fix typo docs/ (#4033)	1 year ago
Baizhou Zhang	4da324cd60	[hotfix]fix argument naming in docs and examples (#4083)	1 year ago
Frank Lee	ddcf58cacf	Revert "[sync] sync feature/shardformer with develop"	1 year ago
FoolPlayer	24651fdd4f	Merge pull request #3931 from FrankLeeeee/sync/develop-to-shardformer ... [sync] sync feature/shardformer with develop	1 year ago
digger yu	33eef714db	fix typo examples and docs (#3932)	1 year ago
Hongxin Liu	12c90db3f3	[doc] add lazy init tutorial (#3922) ... * [doc] add lazy init en doc * [doc] add lazy init zh doc * [doc] add lazy init doc in sidebar * [doc] add lazy init doc test * [doc] fix lazy init doc link	1 year ago
Baizhou Zhang	c1535ccbba	[doc] fix docs about booster api usage (#3898)	1 year ago
jiangmingyan	07cb21142f	[doc]update moe chinese document. (#3890) ... * [doc]update-moe * [doc]update-moe * [doc]update-moe * [doc]update-moe * [doc]update-moe	1 year ago
jiangmingyan	281b33f362	[doc] update document of zero with chunk. (#3855) ... * [doc] fix title of mixed precision * [doc]update document of zero with chunk * [doc] update document of zero with chunk, fix * [doc] update document of zero with chunk, fix * [doc] update document of zero with chunk, fix * [doc] update document of zero with chunk, add doc test * [doc] update document of zero with chunk, add doc test * [doc] update document of zero with chunk, fix installation * [doc] update document of zero with chunk, fix zero with chunk doc * [doc] update document of zero with chunk, fix zero with chunk doc	2 years ago
jiangmingyan	b0474878bf	[doc] update nvme offload documents. (#3850)	2 years ago
jiangmingyan	a64df3fa97	[doc] update document of gemini instruction. (#3842) ... * [doc] update meet_gemini.md * [doc] update meet_gemini.md * [doc] fix parentheses * [doc] fix parentheses * [doc] fix doc test * [doc] fix doc test * [doc] fix doc	2 years ago
Frank Lee	54e97ed7ea	[workflow] supported test on CUDA 10.2 (#3841)	2 years ago
wukong1992	3229f93e30	[booster] add warning for torch fsdp plugin doc (#3833)	2 years ago
digger yu	518b31c059	[docs] change placememt_policy to placement_policy (#3829) ... * fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc. * fix typo colossalai/auto_parallel autochunk fx/passes etc. * fix typo docs/ * change placememt_policy to placement_policy in docs/ and examples/	2 years ago
digger yu	e90fdb1000	fix typo docs/	2 years ago
jiangmingyan	725365f297	Merge pull request #3810 from jiangmingyan/amp ... [doc] update amp document	2 years ago
jiangmingyan	278fcbc444	[doc]fix	2 years ago
jiangmingyan	8aa1fb2c7f	[doc]fix	2 years ago
Hongxin Liu	19d153057e	[doc] add warning about fsdp plugin (#3813)	2 years ago
jiangmingyan	c425a69d52	[doc] add removed change of config.py	2 years ago
jiangmingyan	75272ef37b	[doc] add removed warning	2 years ago
Mingyan Jiang	a520610bd9	[doc] update amp document	2 years ago
Mingyan Jiang	1167bf5b10	[doc] update amp document	2 years ago
Mingyan Jiang	8c62e50dbb	[doc] update amp document	2 years ago
jiangmingyan	ef02d7ef6d	[doc] update gradient accumulation (#3771) ... * [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, add sidebars * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, resolve comments * [doc]update gradient accumulation, resolve comments * fix	2 years ago
github-actions[bot]	62c7e67f9f	[format] applied code formatting on changed files in pull request 3786 (#3787) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
jiangmingyan	fe1561a884	[doc] update gradient cliping document (#3778) ... * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping doc, fix sidebars.json * [doc] update gradient clipping doc, fix doc test	2 years ago
Yanjia0	d9393b85f1	[doc] add deprecated warning on doc Basics section (#3754) ... * Update colotensor_concept.md * Update configure_parallelization.md * Update define_your_config.md * Update engine_trainer.md * Update initialize_features.md * Update model_checkpoint.md * Update colotensor_concept.md * Update configure_parallelization.md * Update define_your_config.md * Update engine_trainer.md * Update initialize_features.md * Update model_checkpoint.md	2 years ago
Hongxin Liu	72688adb2f	[doc] add booster docstring and fix autodoc (#3789) ... * [doc] add docstr for booster methods * [doc] fix autodoc	2 years ago
Hongxin Liu	60e6a154bc	[doc] add tutorial for booster checkpoint (#3785) ... * [doc] add checkpoint related docstr for booster * [doc] add en checkpoint doc * [doc] add zh checkpoint doc * [doc] add booster checkpoint doc in sidebar * [doc] add cuation about ckpt for plugins * [doc] add doctest placeholder * [doc] add doctest placeholder * [doc] add doctest placeholder	2 years ago
binmakeswell	ad2cf58f50	[chat] add performance and tutorial (#3786)	2 years ago
Hongxin Liu	21e29e2212	[doc] add tutorial for booster plugins (#3758) ... * [doc] add en booster plugins doc * [doc] add booster plugins doc in sidebar * [doc] add zh booster plugins doc * [doc] fix zh booster plugin translation * [doc] reoganize tutorials order of basic section * [devops] force sync to test ci	2 years ago
Hongxin Liu	5ce6c9d86f	[doc] add tutorial for cluster utils (#3763) ... * [doc] add en cluster utils doc * [doc] add zh cluster utils doc * [doc] add cluster utils doc in sidebar	2 years ago
jiangmingyan	48bd056761	[doc] update hybrid parallelism doc (#3770)	2 years ago
jiangmingyan	d449525acf	[doc] update booster tutorials (#3718) ... * [booster] update booster tutorials#3717 * [booster] update booster tutorials#3717, fix * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, fix * [booster] update booster tutorials#3717, fix * [booster] update tutorials#3717, update booster api doc * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3713 * [booster] update tutorials#3713, modify file	2 years ago
Hongxin Liu	5dd573c6b6	[devops] fix ci for document check (#3751) ... * [doc] add test info * [devops] update doc check ci * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] remove debug info and update invalid doc * [devops] add essential comments	2 years ago
digger-yu	b9a8dff7e5	[doc] Fix typo under colossalai and doc(#3618) ... * Fixed several spelling errors under colossalai * Fix the spelling error in colossalai and docs directory * Cautious Changed the spelling error under the example folder * Update runtime_preparation_pass.py revert autograft to autograd * Update search_chunk.py utile to until * Update check_installation.py change misteach to mismatch in line 91 * Update 1D_tensor_parallel.md revert to perceptron * Update 2D_tensor_parallel.md revert to perceptron in line 73 * Update 2p5D_tensor_parallel.md revert to perceptron in line 71 * Update 3D_tensor_parallel.md revert to perceptron in line 80 * Update README.md revert to resnet in line 42 * Update reorder_graph.py revert to indice in line 7 * Update p2p.py revert to megatron in line 94 * Update initialize.py revert to torchrun in line 198 * Update routers.py change to detailed in line 63 * Update routers.py change to detailed in line 146 * Update README.md revert random number in line 402	2 years ago
digger-yu	9edeadfb24	[doc] Update 1D_tensor_parallel.md (#3573) ... Display format optimization , same as fix#3562 Simultaneous modification of en version	2 years ago
digger-yu	1c7734bc94	[doc] Update 1D_tensor_parallel.md (#3563) ... Display format optimization, fix bug#3562 Specific changes 1. "This is called a column-parallel fashion" Translate to Chinese 2. use the ```math code block syntax to display a math expression as a block, No modification of formula content Please check that the math formula is displayed correctly If OK, I will change the format of the English version of the formula in parallel	2 years ago
digger-yu	a3ac48ef3d	[doc] Update README-zh-Hans.md (#3541) ... Fixing document link errors using absolute paths	2 years ago
binmakeswell	0c0455700f	[doc] add requirement and highlight application (#3516) ... * [doc] add requirement and highlight application * [doc] link example and application	2 years ago
Frank Lee	4e9989344d	[doc] updated contributor list (#3474)	2 years ago
Frank Lee	80eba05b0a	[test] refactor tests with spawn (#3452) ... * [test] added spawn decorator * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
ver217	26b7aac0be	[zero] reorganize zero/gemini folder structure (#3424) ... * [zero] refactor low-level zero folder structure * [zero] fix legacy zero import path * [zero] fix legacy zero import path * [zero] remove useless import * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] fix test import path * [zero] fix test * [zero] fix circular import * [zero] update import	2 years ago
binmakeswell	15a74da79c	[doc] add Intel cooperation news (#3333) ... * [doc] add Intel cooperation news * [doc] add Intel cooperation news	2 years ago
binmakeswell	31c78f2be3	[doc] add ColossalChat news (#3304) ... * [doc] add ColossalChat news * [doc] add ColossalChat news	2 years ago
binmakeswell	682af61396	[doc] add ColossalChat (#3297) ... * [doc] add ColossalChat	2 years ago
Saurav Maheshkar	20d1c99444	[refactor] update docs (#3174) ... * refactor: README-zh-Hans * refactor: REFERENCE * docs: update paths in README	2 years ago
Frank Lee	3213347b49	[doc] fixed typos in docs/README.md (#3082)	2 years ago
Frank Lee	416a50dbd7	[doc] moved doc test command to bottom (#3075)	2 years ago
Frank Lee	ea0b52c12e	[doc] specified operating system requirement (#3019) ... * [doc] specified operating system requirement * polish code	2 years ago
ver217	378d827c6b	[doc] update nvme offload doc (#3014) ... * [doc] update nvme offload doc * [doc] add doc testing cmd and requirements * [doc] add api reference * [doc] add dependencies	2 years ago
Frank Lee	8fedc8766a	[workflow] supported conda package installation in doc test (#3028) ... * [workflow] supported conda package installation in doc test * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
Frank Lee	e0a1c1321c	[doc] added reference to related works (#2994) ... * [doc] added reference to related works * polish code	2 years ago
github-actions[bot]	dca98937f8	[format] applied code formatting on changed files in pull request 2933 (#2939) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
binmakeswell	8264cd7ef1	[doc] add env scope (#2933)	2 years ago
Frank Lee	b8804aa60c	[doc] added readme for documentation (#2935)	2 years ago
Frank Lee	9e3b8b7aff	[doc] removed read-the-docs (#2932)	2 years ago
Frank Lee	77b88a3849	[workflow] added auto doc test on PR (#2929) ... * [workflow] added auto doc test on PR * [workflow] added doc test workflow * polish code * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
binmakeswell	0afb55fc5b	[doc] add os scope, update tutorial install and tips (#2914)	2 years ago
YuliangLiu0306	cf6409dd40	Hotfix/auto parallel zh doc (#2820) ... * [hotfix] fix autoparallel zh docs * polish * polish	2 years ago
YuliangLiu0306	2059fdd6b0	[hotfix] add copyright for solver and device mesh (#2803) ... * [hotfix] add copyright for solver and device mesh * add readme * add alpa license * polish	2 years ago
Frank Lee	e376954305	[doc] add opt service doc (#2747)	2 years ago
Frank Lee	5479fdd5b8	[doc] updated documentation version list (#2730)	2 years ago
Frank Lee	2045d45ab7	[doc] updated documentation version list (#2715)	2 years ago
Frank Lee	0966008839	[dooc] fixed the sidebar itemm key (#2672)	2 years ago