github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,401 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1401 Commits (eb7742a4bbff25a9e9cae923e3b490e54038b9f3)

Author SHA1 Message Date
YuliangLiu0306 0ed7042f42
[pipeline] refactor pipeline (#679) 
* refactor pipeline---put runtime schedule into engine.

* add type hint for schedule Optional[BaseSchedule]

* preprocess schedule during engine initializing

* infer pipeline schedule params from config
 2022-04-07 15:54:14 +08:00
Frank Lee eace69387d
[ci] fixed compatibility workflow (#678) 2022-04-06 16:19:34 +08:00
Jiarui Fang 59bf2dc590
[zero] initialize a stateful tensor manager (#614) 2022-04-06 16:18:49 +08:00
Frank Lee cc236916c6
[ci] replace the dngc ocker image with self-built pytorch image (#672) 2022-04-06 14:10:17 +08:00
ver217 03e1d35931
[release] update version (#673) 2022-04-06 12:03:23 +08:00
encmps 79ccfa4310 [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu code style (#667) 2022-04-06 11:40:59 +08:00
lucasliunju e4bcff9b0f [NFC] polish colossalai/builder/builder.py code style (#662) 2022-04-06 11:40:59 +08:00
shenggan 331683bf82 [NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu code style (#661) 2022-04-06 11:40:59 +08:00
FredHuang99 c336cd3066 [NFC] polish colossalai/communication/utils.py code style (#656) 2022-04-06 11:40:59 +08:00
MaxT 5ab9a71299 [NFC] polish colossalai/kernel/cuda_native/csrc/moe_cuda.cpp code style (#642) 2022-04-06 11:40:59 +08:00
Xue Fuzhao 10afec728f [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h code style (#641) 2022-04-06 11:40:59 +08:00
Cautiousss 055d0270c8 [NFC] polish colossalai/context/process_group_initializer/initializer_sequence.py colossalai/context/process_group_initializer initializer_tensor.py code style (#639) 
Co-authored-by: 何晓昕 <cautious@r-236-100-25-172.comp.nus.edu.sg>
 2022-04-06 11:40:59 +08:00
Ziheng Qin c7c224ee17 [NFC] polish colossalai/builder/pipeline.py code style (#638) 2022-04-06 11:40:59 +08:00
Sze-qq 10591ecdf9 [NFC] polish colossalai/kernel/cuda_native/csrc/cpu_adam.cpp code style (#636) 2022-04-06 11:40:59 +08:00
Wangbo Zhao 6fcb381801 [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu code style (#635) 2022-04-06 11:40:59 +08:00
ExtremeViscent 8a5d526e95 [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/dropout_kernels.cu and cross_entropy.cu code style (#634) 2022-04-06 11:40:59 +08:00
RichardoLuo ad1e7ab2b2 '[NFC] polish <colossalai/engine/_base_engine.py> code style' (#631) 
Co-authored-by: RichardoLuo <14049555596@qq.com>
 2022-04-06 11:40:59 +08:00
Zangwei 2e11853d04 [NFC] polish colossalai/communication/ring.py code style (#630) 2022-04-06 11:40:59 +08:00
puck_WCR 01cc941e1d [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/transform_kernels.cu code stype (#629) 2022-04-06 11:40:59 +08:00
superhao1995 c1bed0d998 [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu code stype (#628) 2022-04-06 11:40:59 +08:00
Jiang Zhuo 0a96338b13 [NFC] polish <colossalai/context/process_group_initializer/initializer_data.py> code stype (#626) 
Co-authored-by: 姜卓 <jiangzhuo@jiangzhuodeMacBook-Pro.local>
 2022-04-06 11:40:59 +08:00
ziyu huang 701bad439b [NFC] polish colossalai/context/process_group_initializer/process_group_initializer.py code stype (#617) 
Co-authored-by: “Arsmart123 <202476410arsmart@gmail.com>
 2022-04-06 11:40:59 +08:00
Shawn-Kong db54419409 fix format (#613) 
Co-authored-by: evin K <evink@evins-MacBook-Air.local>
 2022-04-06 11:40:59 +08:00
Yuer867 5ecef13c16 fix format (#611) 2022-04-06 11:40:59 +08:00
xyupeng d3d5bedc65 fix format (#607) 2022-04-06 11:40:59 +08:00
xuqifan897 f2d2a1597a fix format (#608) 2022-04-06 11:40:59 +08:00
doubleHU f2da21a827 fix format (#586) 2022-04-06 11:40:59 +08:00
fanjinfucool ffad81e1d1 fix format (#585) 
Co-authored-by: fanjifu <FAN>
 2022-04-06 11:40:59 +08:00
binmakeswell 6582aedc94 fix format (#583) 2022-04-06 11:40:59 +08:00
DouJS f08fc17f2b block_reduce.h fix format (#581) 2022-04-06 11:40:59 +08:00
Maruyama_Aya d2dc6049b5 fix format (#580) 2022-04-06 11:40:59 +08:00
wky 174b9c1d85 fix format (#574) 2022-04-06 11:40:59 +08:00
BoxiangW dfe423ae42 fix format (#572) 2022-04-06 11:40:59 +08:00
yuxuan-lou cfb41297ff 'fix/format' (#573) 2022-04-06 11:40:59 +08:00
Kai Wang (Victor Kai) b0f708dfc1 fix format (#570) 2022-04-06 11:40:59 +08:00
Xu Kai 2a915a8b62 fix format (#568) 2022-04-06 11:40:59 +08:00
YuliangLiu0306 9420d3ae31 fix format (#567) 2022-04-06 11:40:59 +08:00
Jie Zhu 0f1da44e5e [format]colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp (#566) 2022-04-06 11:40:59 +08:00
coder-chin 5835631218 fix format (#564) 2022-04-06 11:40:59 +08:00
Luxios22 e014144c44 fix format (#565) 2022-04-06 11:40:59 +08:00
Ziyue Jiang 1762ba14ab fix format (#563) 2022-04-06 11:40:59 +08:00
Sze-qq ce8a3eae5b
update GPT-2 experiment result (#666) 2022-04-04 13:47:43 +08:00
HELSON 17e73e62cc
[hotfix] fix bugs for unsharded parameters when restore data (#664) 2022-04-03 22:02:11 +08:00
Jiarui Fang 0aab52301e
[hotfix] fix a bug in model data stats tracing (#655) 2022-04-03 21:48:06 +08:00
YuliangLiu0306 ade05a5d83
[refactor] pipeline, put runtime schedule into engine. (#627) 2022-04-03 20:46:45 +08:00
HELSON e5d615aeee
[hotfix] fix bugs in testing (#659) 
* remove hybrid adam in test_moe_zero_optim

* fix activation checkpointing and its unitest
 2022-04-02 21:58:47 +08:00
Jiarui Fang 036404ca8a
Revert "[zero] polish init context (#645)" (#657) 2022-04-02 18:30:06 +08:00
HELSON b31daed4cf
fix bugs in CPU adam (#633) 
* add cpu adam counter for all cpu adam

* fixed updating error in adam kernel
 2022-04-02 17:04:05 +08:00
LuGY 1e2557e801
[zero] fixed the activation offload (#647) 
* fixed the activation offload

* polish
 2022-04-02 16:21:32 +08:00
Liang Bowen 828e465622
[hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622) 2022-04-02 16:12:04 +08:00