ver217
578ea0583b
update setup and workflow ( #222 )
3 years ago
Frank Lee
3a1a9820b0
fixed mkdir conflict and align yapf config with flake ( #220 )
3 years ago
Frank Lee
65e72983dc
added flake8 config ( #219 )
3 years ago
アマデウス
9ee197d0e9
moved env variables to global variables; ( #215 )
...
added branch context;
added vocab parallel layers;
moved split_batch from load_batch to tensor parallel embedding layers;
updated gpt model;
updated unit test cases;
fixed few collective communicator bugs
3 years ago
Frank Lee
b82d60be02
updated github action for develop branch ( #214 )
3 years ago
BoxiangW
7d15ec7fe2
Update github actions ( #205 )
3 years ago
github-actions[bot]
5420809f43
Automated submodule synchronization ( #203 )
...
Co-authored-by: github-actions <github-actions@github.com>
3 years ago
Frank Lee
fd570ab285
add changelog and contributing doc ( #202 )
3 years ago
Frank Lee
02f13fa9d1
add code quality badge ( #201 )
3 years ago
Frank Lee
812357d63c
fixed utils docstring and add example to readme ( #200 )
3 years ago
Frank Lee
b9a761b9b8
added github action to synchronize submodule commits automatically ( #193 )
3 years ago
BoxiangW
a2f1565672
Update GitHub action and pre-commit settings ( #196 )
...
* Update GitHub action and pre-commit settings
* Update GitHub action and pre-commit settings (#198 )
3 years ago
Frank Lee
765db512b5
fixed ddp bug on torch 1.8 ( #194 )
3 years ago
Jiarui Fang
569357fea0
add pytorch hooks ( #179 )
...
* add pytorch hooks
fix #175
* remove licenses in src code
* add gpu memory tracer
* replacing print with logger in ophooks.
3 years ago
ver217
708404d5f8
fix pipeline forward return tensors ( #176 )
3 years ago
WANG-CR
6fb550acdb
update logo
3 years ago
HELSON
0f8c7f9804
Fixed docstring in colossalai ( #171 )
3 years ago
Frank Lee
e2089c5c15
adapted for sequence parallel ( #163 )
3 years ago
Frank Lee
a2e649da39
update readme ( #168 )
3 years ago
Frank Lee
9684bdce5c
fixed submodule url ( #167 )
...
* added examples as submodule
* update submodule url
3 years ago
BoxiangW
bd4840f1f1
Update workflow files and README.md ( #166 )
3 years ago
ver217
1949d3a889
update doc requirements and rtd conf ( #165 )
3 years ago
Frank Lee
be85a0f366
removed tutorial markdown and refreshed rst files for consistency
3 years ago
Frank Lee
ca4ae52d6b
Set examples as submodule ( #162 )
...
* remove examples folder
* added examples as submodule
* update .gitmodules
3 years ago
binmakeswell
17ce8569a8
add logo at homepage, add forum in issue template ( #161 )
3 years ago
puck_WCR
9473a1b9c8
AMP docstring/markdown update ( #160 )
3 years ago
Frank Lee
2499faa2db
update benchmark commit id ( #159 )
3 years ago
LuGY_mac
d143396cac
Added rand augment and update the dataloader
3 years ago
Frank Lee
c7b8ece736
set benchmarks as a git submodule ( #156 )
...
* remove benchmark folder
* added benchmark submodule
* update .gitmodules
3 years ago
Frank Lee
f3802d6b06
fixed jit default setting ( #154 )
3 years ago
Frank Lee
a1da3900c8
added docker documentation ( #152 )
3 years ago
ver217
7bf1e98b97
pipeline last stage supports multi output ( #151 )
3 years ago
HELSON
1ff5be36c2
Added moe parallel example ( #140 )
3 years ago
ver217
f68eddfb3d
refactor kernel ( #142 )
3 years ago
BoxiangW
4a3d3446b0
Update layer integration documentations ( #108 )
...
Update the documentations of layer integration
Update _log_hook.py
Update _operation.py
3 years ago
binmakeswell
3a61d785b5
add doc issue template ( #133 )
3 years ago
ver217
9ef05ed1fc
try import deepspeed when using zero ( #130 )
3 years ago
ver217
b7975d2bcd
add workflow for deploying doc ( #129 )
3 years ago
HELSON
dceae85195
Added MoE parallel ( #127 )
3 years ago
Frank Lee
42741dd4a3
added docker image ( #126 )
3 years ago
ver217
293fb40c42
add scatter/gather optim for pipeline ( #123 )
3 years ago
Frank Lee
404e6f88ed
Hotfix/gitact ( #125 )
...
* enable CI after PR sync
* Fixed github action
3 years ago
binmakeswell
43e7d54643
fix issue template ( #118 )
3 years ago
Jiarui Fang
2c0c85d3d3
fix a bug in timer ( #114 )
3 years ago
ver217
7904baf6e1
fix layers/schedule for hybrid parallelization ( #111 ) ( #112 )
3 years ago
ver217
f03bcb359b
update vit example for new API ( #98 ) ( #99 )
3 years ago
Frank Lee
d09a79bad5
enable CI after PR sync ( #97 )
3 years ago
ver217
a951bc6089
update default logger ( #100 ) ( #101 )
3 years ago
ver217
96780e6ee4
Optimize pipeline schedule ( #94 )
...
* add pipeline shared module wrapper and update load batch
* added model parallel process group for amp and clip grad (#86 )
* added model parallel process group for amp and clip grad
* update amp and clip with model parallel process group
* remove pipeline_prev/next group (#88 )
* micro batch offload
* optimize pipeline gpu memory usage
* pipeline can receive tensor shape (#93 )
* optimize pipeline gpu memory usage
* fix grad accumulation step counter
* rename classes and functions
Co-authored-by: Frank Lee <somerlee.9@gmail.com>
3 years ago
アマデウス
e5b9f9a08d
added gpt model & benchmark ( #95 )
3 years ago