binmakeswell
6582aedc94
fix format ( #583 )
2022-04-06 11:40:59 +08:00
DouJS
f08fc17f2b
block_reduce.h fix format ( #581 )
2022-04-06 11:40:59 +08:00
wky
174b9c1d85
fix format ( #574 )
2022-04-06 11:40:59 +08:00
BoxiangW
dfe423ae42
fix format ( #572 )
2022-04-06 11:40:59 +08:00
Xu Kai
2a915a8b62
fix format ( #568 )
2022-04-06 11:40:59 +08:00
YuliangLiu0306
9420d3ae31
fix format ( #567 )
2022-04-06 11:40:59 +08:00
Jie Zhu
0f1da44e5e
[format]colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp ( #566 )
2022-04-06 11:40:59 +08:00
coder-chin
5835631218
fix format ( #564 )
2022-04-06 11:40:59 +08:00
Luxios22
e014144c44
fix format ( #565 )
2022-04-06 11:40:59 +08:00
Ziyue Jiang
1762ba14ab
fix format ( #563 )
2022-04-06 11:40:59 +08:00
HELSON
b31daed4cf
fix bugs in CPU adam ( #633 )
...
* add cpu adam counter for all cpu adam
* fixed updating error in adam kernel
2022-04-02 17:04:05 +08:00
KAIYUAN GAN
229382c844
[NFC] polish colossalai/kernel/cuda_native/csrc/kernels/cuda_util.cu code stype ( #625 )
2022-04-01 17:45:53 +08:00
LuGY
6a3f9fda83
[cuda] modify the fused adam, support hybrid of fp16 and fp32 ( #497 )
2022-03-25 14:15:53 +08:00
ExtremeViscent
eaac03ae1d
[formart] format fixed for kernel\cuda_native codes ( #335 )
2022-03-11 15:50:28 +08:00
LuGY
a3269de5c9
[zero] cpu adam kernel ( #288 )
...
* Added CPU Adam
* finished the cpu adam
* updated the license
* delete useless parameters, removed resnet
* modified the method off cpu adam unittest
* deleted some useless codes
* removed useless codes
Co-authored-by: ver217 <lhx0217@gmail.com>
Co-authored-by: Frank Lee <somerlee.9@gmail.com>
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>
2022-03-11 15:50:28 +08:00
1SAA
219df6e685
Optimized MoE layer and fixed some bugs;
...
Decreased moe tests;
Added FFNExperts and ViTMoE model
2022-03-11 15:50:28 +08:00
アマデウス
9ee197d0e9
moved env variables to global variables; ( #215 )
...
added branch context;
added vocab parallel layers;
moved split_batch from load_batch to tensor parallel embedding layers;
updated gpt model;
updated unit test cases;
fixed few collective communicator bugs
2022-02-15 11:31:13 +08:00
HELSON
0f8c7f9804
Fixed docstring in colossalai ( #171 )
2022-01-21 10:44:30 +08:00
Frank Lee
e2089c5c15
adapted for sequence parallel ( #163 )
2022-01-20 13:44:51 +08:00
Frank Lee
f3802d6b06
fixed jit default setting ( #154 )
2022-01-18 13:37:20 +08:00
ver217
f68eddfb3d
refactor kernel ( #142 )
2022-01-13 16:47:17 +08:00
shenggan
5c3843dc98
add colossalai kernel module ( #55 )
2021-12-21 12:19:52 +08:00