ColossalAI

Commit Graph

Author	SHA1	Message	Date
HELSON	0f8c7f9804	Fixed docstring in colossalai (#171 )	3 years ago
Frank Lee	e2089c5c15	adapted for sequence parallel (#163 )	3 years ago
Frank Lee	a2e649da39	update readme (#168 )	3 years ago
Frank Lee	9684bdce5c	fixed submodule url (#167 ) * added examples as submodule * update submodule url	3 years ago
BoxiangW	bd4840f1f1	Update workflow files and README.md (#166 )	3 years ago
ver217	1949d3a889	update doc requirements and rtd conf (#165 )	3 years ago
Frank Lee	be85a0f366	removed tutorial markdown and refreshed rst files for consistency	3 years ago
Frank Lee	ca4ae52d6b	Set examples as submodule (#162 ) * remove examples folder * added examples as submodule * update .gitmodules	3 years ago
binmakeswell	17ce8569a8	add logo at homepage, add forum in issue template (#161 )	3 years ago
puck_WCR	9473a1b9c8	AMP docstring/markdown update (#160 )	3 years ago
Frank Lee	2499faa2db	update benchmark commit id (#159 )	3 years ago
LuGY_mac	d143396cac	Added rand augment and update the dataloader	3 years ago
Frank Lee	c7b8ece736	set benchmarks as a git submodule (#156 ) * remove benchmark folder * added benchmark submodule * update .gitmodules	3 years ago
Frank Lee	f3802d6b06	fixed jit default setting (#154 )	3 years ago
Frank Lee	a1da3900c8	added docker documentation (#152 )	3 years ago
ver217	7bf1e98b97	pipeline last stage supports multi output (#151 )	3 years ago
HELSON	1ff5be36c2	Added moe parallel example (#140 )	3 years ago
ver217	f68eddfb3d	refactor kernel (#142 )	3 years ago
BoxiangW	4a3d3446b0	Update layer integration documentations (#108 ) Update the documentations of layer integration Update _log_hook.py Update _operation.py	3 years ago
binmakeswell	3a61d785b5	add doc issue template (#133 )	3 years ago
ver217	9ef05ed1fc	try import deepspeed when using zero (#130 )	3 years ago
ver217	b7975d2bcd	add workflow for deploying doc (#129 )	3 years ago
HELSON	dceae85195	Added MoE parallel (#127 )	3 years ago
Frank Lee	42741dd4a3	added docker image (#126 )	3 years ago
ver217	293fb40c42	add scatter/gather optim for pipeline (#123 )	3 years ago
Frank Lee	404e6f88ed	Hotfix/gitact (#125 ) * enable CI after PR sync * Fixed github action	3 years ago
binmakeswell	43e7d54643	fix issue template (#118 )	3 years ago
Jiarui Fang	2c0c85d3d3	fix a bug in timer (#114 )	3 years ago
ver217	7904baf6e1	fix layers/schedule for hybrid parallelization (#111 ) (#112 )	3 years ago
ver217	f03bcb359b	update vit example for new API (#98 ) (#99 )	3 years ago
Frank Lee	d09a79bad5	enable CI after PR sync (#97 )	3 years ago
ver217	a951bc6089	update default logger (#100 ) (#101 )	3 years ago
ver217	96780e6ee4	Optimize pipeline schedule (#94 ) * add pipeline shared module wrapper and update load batch * added model parallel process group for amp and clip grad (#86) * added model parallel process group for amp and clip grad * update amp and clip with model parallel process group * remove pipeline_prev/next group (#88) * micro batch offload * optimize pipeline gpu memory usage * pipeline can receive tensor shape (#93) * optimize pipeline gpu memory usage * fix grad accumulation step counter * rename classes and functions Co-authored-by: Frank Lee <somerlee.9@gmail.com>	3 years ago
アマデウス	e5b9f9a08d	added gpt model & benchmark (#95 )	3 years ago
アマデウス	01a80cd86d	Hotfix/Colossalai layers (#92 ) * optimized 1d layer apis; reorganized nn.layer modules; fixed tests * fixed 2.5d runtime issue * reworked split batch, now called in trainer.schedule.load_batch Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	3 years ago
アマデウス	0fedef4f3c	Layer integration (#83 ) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	3 years ago
shenggan	5c3843dc98	add colossalai kernel module (#55 )	3 years ago
Xin Zhang	648f806315	add example of self-supervised SimCLR training - V2 (#50 ) * add example of self-supervised SimCLR training * simclr v2, replace nvidia dali dataloader * updated * sync to latest code writing style * sync to latest code writing style and modify README * detail README & standardize dataset path	3 years ago
ver217	8f02a88db2	add interleaved pipeline, fix naive amp and update pipeline model initializer (#80 )	3 years ago
Frank Lee	91c327cb44	fixed zero level 3 dtype bug (#76 )	3 years ago
HELSON	632e622de8	overlap computation and communication in 2d operations (#75 )	3 years ago
Frank Lee	cd9c28e055	added CI for unit testing (#69 )	3 years ago
Frank Lee	45355a62f7	Update issue templates (#66 )	3 years ago
Frank Lee	35813ed3c4	update examples and sphnix docs for the new api (#63 )	3 years ago
ver217	7d3711058f	fix zero3 fp16 and add zero3 model context (#62 )	3 years ago
Frank Lee	9a0466534c	update markdown docs (english) (#60 )	3 years ago
Frank Lee	da01c234e1	Develop/experiments (#59 ) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit `2e0b0b7699`. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> * Split conv2d, class token, positional embedding in 2d, Fix random number in ddp Fix convergence in cifar10, Imagenet1000 * Integrate 1d tensor parallel in Colossal-AI (#39) * fixed 1D and 2D convergence (#38) * optimized 2D operations * fixed 1D ViT convergence problem * Feature/ddp (#49) * remove redundancy func in setup (#19) (#20) * use env to control the language of doc (#24) (#25) * Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit `2e0b0b7699`. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> * add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29) * add explanation for ViT example (#35) (#36) * support torch ddp * fix loss accumulation * add log for ddp * change seed * modify timing hook Co-authored-by: Frank Lee <somerlee.9@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> * Feature/pipeline (#40) * remove redundancy func in setup (#19) (#20) * use env to control the language of doc (#24) (#25) * Support TP-compatible Torch AMP and Update trainer API (#27) * Add gradient accumulation, fix lr scheduler * fix FP16 optimizer and adapted torch amp with tensor parallel (#18) * fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes * fixed trainer * Revert "fixed trainer" This reverts commit `2e0b0b7699`. * improved consistency between trainer, engine and schedule (#23) Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> * add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29) * add explanation for ViT example (#35) (#36) * optimize communication of pipeline parallel * fix grad clip for pipeline Co-authored-by: Frank Lee <somerlee.9@gmail.com> Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> * optimized 3d layer to fix slow computation ; tested imagenet performance with 3d; reworked lr_scheduler config definition; fixed launch args; fixed some printing issues; simplified apis of 3d layers (#51) * Update 2.5d layer code to get a similar accuracy on imagenet-1k dataset * update api for better usability (#58) update api for better usability Co-authored-by: 1SAA <c2h214748@gmail.com> Co-authored-by: ver217 <lhx0217@gmail.com> Co-authored-by: puck_WCR <46049915+WANG-CR@users.noreply.github.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> Co-authored-by: アマデウス <kurisusnowdeng@users.noreply.github.com> Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com>	3 years ago
ver217	eb2f8b1f6b	add how to build tfrecord dataset (#48 )	3 years ago
ver217	4da256a584	add some details in vit-b16 example (#46 )	3 years ago
ver217	e67dab92a9	add some details in vit-b16 example (#43 ) (#44 )	3 years ago

... 66 67 68 69 70

3466 Commits (457a0de79fd2d3602eba0ac78e606acb6401fc60) All Branches Search

3466 Commits (457a0de79fd2d3602eba0ac78e606acb6401fc60)

All Branches