379 Commits (19e1a5cf16ead982eb8818cd69e41b06a5d23b20)

Author	SHA1	Message	Date
Jiarui Fang	7c31706227	[CI] add test_ci.sh for palm, opt and gpt (#2475)	2 years ago
Jiarui Fang	e4c38ba367	[example] stable diffusion add roadmap (#2482)	2 years ago
ver217	f525d1f528	[example] update gpt gemini example ci test (#2477)	2 years ago
Ziyue Jiang	fef5c949c3	polish pp middleware (#2476) ... Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
Frank Lee	8b7495dd54	[example] integrate seq-parallel tutorial with CI (#2463)	2 years ago
ver217	8e85d2440a	[example] update vit ci script (#2469) ... * [example] update vit ci script * [example] update requirements * [example] update requirements	2 years ago
Jiarui Fang	867c8c2d3a	[zero] low level optim supports ProcessGroup (#2464)	2 years ago
Frank Lee	e6943e2d11	[example] integrate autoparallel demo with CI (#2466) ... * [example] integrate autoparallel demo with CI * polish code * polish code * polish code * polish code	2 years ago
YuliangLiu0306	c20529fe78	[examples] update autoparallel tutorial demo (#2449) ... * [examples] update autoparallel tutorial demo * add test_ci.sh * polish * add conda yaml	2 years ago
Haofan Wang	cfd1d5ee49	[example] fixed seed error in train_dreambooth_colossalai.py (#2445)	2 years ago
Frank Lee	ac18a445fa	[example] updated large-batch optimizer tutorial (#2448) ... * [example] updated large-batch optimizer tutorial * polish code * polish code	2 years ago
Frank Lee	39163417a1	[example] updated the hybrid parallel tutorial (#2444) ... * [example] updated the hybrid parallel tutorial * polish code	2 years ago
YuliangLiu0306	2731531bc2	[autoparallel] integrate device mesh initialization into autoparallelize (#2393) ... * [autoparallel] integrate device mesh initialization into autoparallelize * add megatron solution * update gpt autoparallel examples with latest api * adapt beta value to fit the current computation cost	2 years ago
Frank Lee	a3e5496156	[example] improved the clarity yof the example readme (#2427) ... * [example] improved the clarity yof the example readme * polish workflow * polish workflow * polish workflow * polish workflow * polish workflow * polish workflow	2 years ago
Frank Lee	63be79d505	[example] removed duplicated stable diffusion example (#2424)	2 years ago
ZijianYY	fe0f7970a2	[examples] adding tflops to PaLM (#2365)	2 years ago
HELSON	d84e747975	[hotfix] add DISTPAN argument for benchmark (#2412) ... * change the benchmark config file * change config * revert config file * rename distpan to distplan	2 years ago
Frank Lee	8327932d2c	[workflow] refactored the example check workflow (#2411) ... * [workflow] refactored the example check workflow * polish code * polish code * polish code * polish code * polish code * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
HELSON	498b5ca993	[hotfix] fix gpt gemini example (#2404) ... * [hotfix] fix gpt gemini example * [example] add new assertions	2 years ago
jiaruifang	b2e0d502b8	[doc] hotfix #2377	2 years ago
Jiarui Fang	8f72b6f8fb	[hotfix] fix implement error in diffusers	2 years ago
1SAA	33f3023e19	[hotfix] fix implement error in diffusers	2 years ago
Jiarui Fang	12c8bf38d7	[Pipeline] Refine GPT PP Example	2 years ago
Ziyue Jiang	ad00894f7f	polish	2 years ago
Jiarui Fang	1aaeb596c6	[example] gpt, shard init on all processes (#2366)	2 years ago
Ziyue Jiang	3a15b20421	Move GPT PP Example	2 years ago
HELSON	48d33b1b17	[gemini] add get static torch model (#2356)	2 years ago
Fazzie-Maqianli	7a332b1734	Merge pull request #2338 from haofanwang/patch-1 ... Fix a typo in train_dreambooth_colossalai.py	2 years ago
YuliangLiu0306	8b1e0dfd80	[example] upload auto parallel gpt2 demo (#2354)	2 years ago
Jiarui Fang	00a9c781fd	[example] add google doc for benchmark results of GPT (#2355)	2 years ago
Jiarui Fang	509a87f3ff	[example] make gpt example directory more clear (#2353)	2 years ago
Ikko Eltociear Ashimine	5e4bced0a3	[NFC] Update roberta/README.md (#2350)	2 years ago
Jiarui Fang	35e22be2f6	[example] simplify opt example (#2344)	2 years ago
ziyuhuang123	7080a8edb0	[workflow]New version: Create workflow files for examples' auto check (#2298) ... * [workflows]bug_repair * [workflow]new_pr_fixing_bugs Co-authored-by: binmakeswell <binmakeswell@gmail.com>	2 years ago
binmakeswell	d7352bef2c	[example] add example requirement (#2345)	2 years ago
Haofan Wang	7ce965c7cc	Update requirement_colossalai.txt (#2348)	2 years ago
ZijianYY	f7fd592bf4	[examples]adding tp to PaLM (#2319)	2 years ago
Haofan Wang	9edd0aa75e	Update train_dreambooth_colossalai.py ... accelerator.num_processes -> gpc.get_world_size(ParallelMode.DATA)	2 years ago
Fazzie-Maqianli	89f26331e9	[example] diffusion update diffusion,Dreamblooth (#2329)	2 years ago
binmakeswell	e512ca9c24	[doc] update stable diffusion link (#2322) ... * [doc] update link	2 years ago
Fazzie-Maqianli	a9b27b9265	[exmaple] fix dreamblooth format (#2315)	2 years ago
Jiarui Fang	32253315b4	[example] update diffusion readme with official lightning (#2304)	2 years ago
HELSON	e00cedd181	[example] update gemini benchmark bash (#2306)	2 years ago
binmakeswell	c8144223b8	[doc] update diffusion doc (#2296)	2 years ago
ZijianYY	df1d6dc553	[examples] using args and combining two versions for PaLM (#2284)	2 years ago
Ziyue Jiang	ac863a01d6	[example] add benchmark (#2276) ... * add benchmark * merge common func * add total and avg tflops Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
BlueRum	1405b4381e	[example] fix save_load bug for dreambooth (#2280)	2 years ago
Jiarui Fang	879df8b943	[example] GPT polish readme (#2274)	2 years ago
Ziyue Jiang	9654df0e9a	Add GPT PP Example (#2272) ... Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>	2 years ago
YuliangLiu0306	4b29112ab2	[autoparallel] gpt2 autoparallel examples (#2267) ... * [autoparallel] gpt2 autoparallel examples * polish code * polish code	2 years ago
HELSON	09c0102fe6	[example] fix gpt example with 0.1.10 (#2265)	2 years ago
Fazzie-Maqianli	89f048a88a	[example] clear diffuser image (#2262)	2 years ago
Frank Lee	89542ceb44	[doc] updated the stable diffussion on docker usage (#2244) ... * [doc] updated the stable diffussion on docker usage * polish doc	2 years ago
Jiarui Fang	50cdf5430e	[example] diffusion install from docker (#2239) ... * [builder] builder for scaled_upper_triang_masked_softmax * add missing files * fix a bug * polish code * [example] diffusion install from docker	2 years ago
Jiarui Fang	db4cbdc7fb	[builder] builder for scaled_upper_triang_masked_softmax (#2234)	2 years ago
HELSON	31fe84237b	[example] fix benchmark.sh for gpt example (#2229)	2 years ago
Jiarui Fang	2cdecc9f38	[example] make palm + GeminiDPP work (#2227)	2 years ago
ZijianYY	63cc77173b	[example] Palm adding gemini, still has bugs (#2221)	2 years ago
HELSON	7010e18134	[example] update gpt example (#2225)	2 years ago
Jiarui Fang	49c601da21	[example] add benchmark.sh for gpt (#2226)	2 years ago
HELSON	3629e611cd	[example] update gpt benchmark (#2219)	2 years ago
ZijianYY	92de90dfb3	[examples] replace einsum with matmul (#2210)	2 years ago
Jiarui Fang	7675792100	[builder] raise Error when CUDA_HOME is not set (#2213)	2 years ago
HELSON	78a89d9b41	[diffusion] update readme (#2214)	2 years ago
Jiarui Fang	d96cc37e32	[example] update GPT example benchmark results (#2212)	2 years ago
Jiarui Fang	d5e3e3ec01	[example] update gpt example for larger model scale (#2211)	2 years ago
Jiarui Fang	29868a9ec1	[example] update gpt readme with performance (#2206)	2 years ago
BlueRum	6642cebdbe	[example] Change some training settings for diffusion (#2195)	2 years ago
ziyuhuang123	4363ff3e41	'[NFC] fix some typos' (#2175)	2 years ago
Fazzie-Maqianli	ce3c4eca7b	[example] support Dreamblooth (#2188)	2 years ago
BlueRum	1cf6d92d7c	[exmaple] diffuser, support quant inference for stable diffusion (#2186)	2 years ago
Jiarui Fang	65f56f49e8	[example] gpt demo more accuracy tflops (#2178)	2 years ago
ziyuhuang123	cf5028363c	'diffusion-typo-change'	2 years ago
Jiarui Fang	27327a4c90	[example] add palm pytorch version (#2172)	2 years ago
Jiarui Fang	a4b4bb01d6	[example] update vit readme (#2155)	2 years ago
Jiarui Fang	2cfe685b9f	[exmaple] add vit missing functions (#2154)	2 years ago
HELSON	a7d95b7024	[example] add zero1, zero2 example in GPT examples (#2146) ... * [example] add zero1 and zero2 for GPT * update readme in gpt example * polish code * change init value * update readme	2 years ago
Fazzie	cea4292ae5	support stable diffusion v2	2 years ago
ZijianYY	fa9d1aea71	[example] update GPT README (#2095)	2 years ago
YuliangLiu0306	edf4cd46c5	[examples] update autoparallel demo (#2061)	2 years ago
Super Daniel	2edbef13cc	[fx] add more meta_registry for MetaTensor execution. (#2000) ... * [sc] add examples for auto checkpoint. * merge upstream * [fx] add more meta_registry for MetaTensor execution.	2 years ago
Fazzie-Maqianli	b5dbb46172	[example] add diffusion inference (#1986)	2 years ago
mandoxzhang	52bd106627	add RoBERTa (#1980) ... * update roberta * update roberta & readme * update roberta & readme * update roberta & readme	2 years ago
Jiarui Fang	f7e276fa71	[Gemini] add GeminiAdamOptimizer (#1960)	2 years ago
Jiarui Fang	60abd86d6a	[example] enhance GPT demo (#1959) ... * [example] enhence GPT demo * Update README.md Co-authored-by: binmakeswell <binmakeswell@gmail.com>	2 years ago
Fazzie	a09f88ab07	update model download in README	2 years ago
Fazzie-Maqianli	6bdd0a90ca	update lightning version (#1954)	2 years ago
binmakeswell	9183e0dec5	[tutorial] polish all README (#1946)	2 years ago
Frank Lee	de56b563b9	[tutorial] added missing dummy dataloader (#1944)	2 years ago
Frank Lee	c6ea65011f	[tutorial] fixed pipeline bug for sequence parallel (#1943)	2 years ago
Jiarui Fang	cf68cc92ac	[example] add vit (#1942) ... * [ColoTensor] ColoInitContext initialize parameters in shard mode. * polish * [example] add vit	2 years ago
YuliangLiu0306	c7925c5d08	[sc demo] add requirements to spmd README (#1941)	2 years ago
Boyuan Yao	d5f5e06d82	[SC] remove redundant hands on (#1939) ... * [sc] SC tutorial for auto checkpoint * [sc] polish examples * [sc] polish readme * [sc] polish readme and help information * [sc] polish readme and help information * [sc] modify auto checkpoint benchmark * [sc] remove imgs * [sc] remove redundant handson	2 years ago
binmakeswell	41868f7605	[tutorial] polish README and OPT files (#1930) ... * [tutorial] polish README and OPT files * [tutorial] polish README and OPT files * [tutorial] polish README and OPT files	2 years ago
ver217	b0b7a786b7	[tutorial] add synthetic dataset for opt (#1924)	2 years ago
Frank Lee	0486048453	[tutorial] updated hybrid parallel readme (#1928) ... * [tutorial] updated hybrid parallel readme * polish code	2 years ago
Frank Lee	807cbdb87d	[tutorial] added synthetic data for sequence parallel (#1927) ... * [tutorial] added synthetic data for sequence parallel * polish code	2 years ago
Frank Lee	abf4c27f6a	[tutorial] removed huggingface model warning (#1925)	2 years ago
Frank Lee	d43a671ad6	Hotfix/tutorial readme index (#1922) ... * [tutorial] removed tutorial index in readme * [tutorial] removed tutorial index in readme	2 years ago
Boyuan Yao	24cbee0ebe	[tutorial] modify hands-on of auto activation checkpoint (#1920) ... * [sc] SC tutorial for auto checkpoint * [sc] polish examples * [sc] polish readme * [sc] polish readme and help information * [sc] polish readme and help information * [sc] modify auto checkpoint benchmark * [sc] remove imgs	2 years ago
Frank Lee	ff16773ded	[tutorial] added synthetic data for hybrid parallel (#1921) ... * [tutorial] added synthetic data for hybrid parallel * polish code	2 years ago
Frank Lee	3c42fdbedc	[tutorial] added synthetic data for hybrid parallel (#1919)	2 years ago
Frank Lee	1b0dd05940	[tutorial] added synthetic dataset for auto parallel demo (#1918)	2 years ago
Frank Lee	acd9abc5ca	[tutorial] updated auto parallel demo with latest data path (#1917)	2 years ago
Frank Lee	d53415bc10	[tutorial] added data script and updated readme (#1916)	2 years ago
binmakeswell	155e202318	[example] update auto_parallel img path (#1910)	2 years ago
Boyuan Yao	d5c5bc219e	[SC] add GPT example for auto checkpoint (#1889) ... * [sc] SC tutorial for auto checkpoint * [sc] polish examples * [sc] polish readme * [sc] polish readme and help information * [sc] polish readme and help information	2 years ago
binmakeswell	11ee8ae478	[tutorial] add cifar10 for diffusion (#1907)	2 years ago
Frank Lee	cb7ec714c8	[tutorial] removed duplicated tutorials (#1904)	2 years ago
Fazzie-Maqianli	351f0f64e6	[example] add cifar10 dadaset for diffusion (#1902) ... * add cifar10 dadasets * Update README.md Co-authored-by: binmakeswell <binmakeswell@gmail.com>	2 years ago
BoxiangW	ca6e75bc28	[tutorial] edited hands-on practices (#1899) ... * Add handson to ColossalAI. * Change names of handsons and edit sequence parallel example. * Edit wrong folder name * resolve conflict * delete readme	2 years ago
BoxiangW	d9bf83e084	Add handson to ColossalAI. (#1896) ... Co-authored-by: Boxiang Wang <boxiang.wang1@gmail.com>	2 years ago
Super Daniel	6d559ea614	[sc] add examples for auto checkpoint. (#1880)	2 years ago
HELSON	f9e7d179f2	[diffusion] fix package conflicts (#1875)	2 years ago
binmakeswell	610dda676c	[example] migrate diffusion and auto_parallel hands-on (#1871)	2 years ago
binmakeswell	50c4cb0167	[NFC] remove redundant dependency (#1869) ... * remove redundant config * remove redundant dependency	2 years ago
binmakeswell	fd8f0ca5a8	[example] initialize tutorial (#1865)	2 years ago
binmakeswell	e9635eb493	add explanation specified version	2 years ago
jiaruifang	27211d6267	[example] polish diffusion readme	2 years ago
binmakeswell	4ac7d3ec3b	[doc] polish diffusion README (#1840)	2 years ago
Jiarui Fang	f86a703bcf	[NFC] update gitignore remove DS_Store (#1830)	2 years ago
Jiarui Fang	a25f755331	[example] add TP to GPT example (#1828)	2 years ago
Fazzie-Maqianli	6e9730d7ab	[example] add stable diffuser (#1825)	2 years ago
Jiarui Fang	b1263d32ba	[example] simplify the GPT2 huggingface example (#1826)	2 years ago
Jiarui Fang	cd5a0d56fa	[Gemini] make gemini usage simple (#1821)	2 years ago
Maruyama_Aya	a7e8159da6	add ColoDiffusion codes: /ldm/module/, /ldm/data/, /scripts/test/	2 years ago
Jiarui Fang	350ccc0481	[example] opt does not depend on Titans (#1811)	2 years ago
Jiarui Fang	203ca57aed	[example] add GPT	2 years ago
Jiarui Fang	fd2c8d8156	[example] add opt model in lauguage (#1809)	2 years ago
Jiarui Fang	f5a92c288c	[example] add diffusion to example (#1805)	2 years ago
Jiarui Fang	a19eb80998	[embedding] updates some default parameters	2 years ago
github-actions[bot]	177d3f5718	Automated submodule synchronization (#1465) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	9b442ecdc3	Automated submodule synchronization (#1404) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	1e5eb0874c	Automated submodule synchronization (#1396) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	50dec605e1	Automated submodule synchronization (#1380) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	fb6f085907	Automated submodule synchronization (#1372) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	6160a1d6a7	Automated submodule synchronization (#1348) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	6f2f9eb214	Automated submodule synchronization (#1305) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	762905da68	Automated submodule synchronization (#1241) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	4951f7d80c	Automated submodule synchronization (#1204) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	23442a5bc1	Automated submodule synchronization (#1194) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	6f0733a1ef	Automated submodule synchronization (#1159) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	e8c34eedfd	Automated submodule synchronization (#1129) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	85b58093d2	Automated submodule synchronization (#1105) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
github-actions[bot]	e32470b6de	Automated submodule synchronization (#1049) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	4d8a574cd3	Automated submodule synchronization (#1034) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	9e3d602dba	Automated submodule synchronization (#1003) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	46bc95708f	Automated submodule synchronization (#960) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	7edb38193a	Automated submodule synchronization (#932) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	b61d64685f	Automated submodule synchronization (#929) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	1cf7fb3cd9	Automated submodule synchronization (#912) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	3b1f5f07ce	Automated submodule synchronization (#907) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	f271f34716	Automated submodule synchronization (#827) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	413ce30c45	Automated submodule synchronization (#819) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	9aae4197bb	Automated submodule synchronization (#810) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	6978980f6d	Automated submodule synchronization (#751) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	d878d843ad	Automated submodule synchronization (#695) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	d50cdabbc9	Automated submodule synchronization (#556) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	92f4224867	Automated submodule synchronization (#501)	3 years ago
github-actions[bot]	353566c198	Automated submodule synchronization (#483) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions[bot]	cfcc8271f3	[Bot] Automated submodule synchronization (#451) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
github-actions	6098bc4cce	Automated submodule synchronization	3 years ago
github-actions	b9f8521f8c	Automated submodule synchronization	3 years ago
github-actions[bot]	5420809f43	Automated submodule synchronization (#203) ... Co-authored-by: github-actions <github-actions@github.com>	3 years ago
Frank Lee	ca4ae52d6b	Set examples as submodule (#162) ... * remove examples folder * added examples as submodule * update .gitmodules	3 years ago
LuGY_mac	d143396cac	Added rand augment and update the dataloader	3 years ago
HELSON	1ff5be36c2	Added moe parallel example (#140)	3 years ago
ver217	f03bcb359b	update vit example for new API (#98) (#99)	3 years ago
アマデウス	0fedef4f3c	Layer integration (#83) ... * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	3 years ago
Xin Zhang	648f806315	add example of self-supervised SimCLR training - V2 (#50) ... * add example of self-supervised SimCLR training * simclr v2, replace nvidia dali dataloader * updated * sync to latest code writing style * sync to latest code writing style and modify README * detail README & standardize dataset path	3 years ago
Frank Lee	35813ed3c4	update examples and sphnix docs for the new api (#63)	3 years ago
Frank Lee	da01c234e1	Develop/experiments (#59) ... * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b7699. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> * Split conv2d, class token, positional embedding in 2d, Fix random number in ddp Fix convergence in cifar10, Imagenet1000 * Integrate 1d tensor parallel in Colossal-AI (#39) * fixed 1D and 2D convergence (#38) * optimized 2D operations * fixed 1D ViT convergence problem * Feature/ddp (#49) * remove redundancy func in setup (#19) (#20) * use env to control the language of doc (#24) (#25) * Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b7699. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> * add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29) * add explanation for ViT example (#35) (#36) * support torch ddp * fix loss accumulation * add log for ddp * change seed * modify timing hook Co-authored-by: Frank Lee <somerlee.9@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> * Feature/pipeline (#40) * remove redundancy func in setup (#19) (#20) * use env to control the language of doc (#24) (#25) * Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b7699. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> * add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29) * add explanation for ViT example (#35) (#36) * optimize communication of pipeline parallel * fix grad clip for pipeline Co-authored-by: Frank Lee <somerlee.9@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> * optimized 3d layer to fix slow computation ; tested imagenet performance with 3d; reworked lr_scheduler config definition; fixed launch args; fixed some printing issues; simplified apis of 3d layers (#51) * Update 2.5d layer code to get a similar accuracy on imagenet-1k dataset * update api for better usability (#58) update api for better usability Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> Co-authored-by: puck_WCR <46049915+WANG-CR@users.noreply.github.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> Co-authored-by: アマデウス <kurisusnowdeng@users.noreply.github.com> Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	3 years ago
ver217	eb2f8b1f6b	add how to build tfrecord dataset (#48)	3 years ago
ver217	4da256a584	add some details in vit-b16 example (#46)	3 years ago
ver217	e67dab92a9	add some details in vit-b16 example (#43) (#44)	3 years ago
binmakeswell	2528adc62f	add explanation for ViT example (#35) (#36)	3 years ago
ver217	dbe62c67b8	add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29)	3 years ago
Frank Lee	3defa32aee	Support TP-compatible Torch AMP and Update trainer API (#27) ... * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit 2e0b0b7699. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com>	3 years ago
zbian	404ecbdcc6	Migrated project	3 years ago