275 Commits (20ab1f55204ee18aca204f6faf60ca27b509b509)

Author	SHA1	Message	Date
Jiarui Fang	193dc8dacb	[refactor] refactor the memory utils (#715)	3 years ago
HELSON	dbd96fe90a	[zero] check whether gradients have inf and nan in gpu (#712)	3 years ago
ver217	715b86eadd	[hotfix] fix stm cuda model data size (#710)	3 years ago
LuGY	140263a394	[hotfix]fixed bugs of assigning grad states to non leaf nodes (#711) ... * fixed bugs of assigning grad states to non leaf nodes * use detach()	3 years ago
Frank Lee	eda30a058e	[compatibility] fixed tensor parallel compatibility with torch 1.9 (#700)	3 years ago
HELSON	a9b8300d54	[zero] improve adaptability for not-shard parameters (#708) ... * adapt post grad hooks for not-shard parameters * adapt optimizer for not-shard parameters * offload gradients for not-replicated parameters	3 years ago
ver217	ab8c6b4a0e	[zero] refactor memstats collector (#706) ... * refactor memstats collector * fix disposable * polish code	3 years ago
アマデウス	3fc8a204dc	[]Corrected 3d vocab parallel embedding (#707)	3 years ago
HELSON	ee112fe1da	[zero] adapt zero hooks for unsharded module (#699)	3 years ago
ver217	3c9cd5bb5e	[zero] stateful tensor manager (#687) ... * [WIP] stateful tensor manager * add eviction strategy * polish code * polish code * polish comment * add unit test * fix sampler bug * polish code * fix max sampling cnt resetting bug * fix sampler bug * polish code * fix bug * fix unit test Co-authored-by: jiaruifang <fangjiarui123@gmail.com>	3 years ago
HELSON	d7ecaf362b	[zero] fix init bugs in zero context (#686) ... * adapt model weight initialization for methods in Pytorch nn.init	3 years ago
YuliangLiu0306	0ed7042f42	[pipeline] refactor pipeline (#679) ... * refactor pipeline---put runtime schedule into engine. * add type hint for schedule Optional[BaseSchedule] * preprocess schedule during engine initializing * infer pipeline schedule params from config	3 years ago
Jiarui Fang	59bf2dc590	[zero] initialize a stateful tensor manager (#614)	3 years ago
encmps	79ccfa4310	[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_adam.cu code style (#667)	3 years ago
lucasliunju	e4bcff9b0f	[NFC] polish colossalai/builder/builder.py code style (#662)	3 years ago
shenggan	331683bf82	[NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu code style (#661)	3 years ago
FredHuang99	c336cd3066	[NFC] polish colossalai/communication/utils.py code style (#656)	3 years ago
MaxT	5ab9a71299	[NFC] polish colossalai/kernel/cuda_native/csrc/moe_cuda.cpp code style (#642)	3 years ago
Xue Fuzhao	10afec728f	[NFC] polish colossalai/kernel/cuda_native/csrc/kernels/include/cuda_util.h code style (#641)	3 years ago
Cautiousss	055d0270c8	[NFC] polish colossalai/context/process_group_initializer/initializer_sequence.py colossalai/context/process_group_initializer initializer_tensor.py code style (#639) ... Co-authored-by: 何晓昕 <cautious@r-236-100-25-172.comp.nus.edu.sg>	3 years ago
Ziheng Qin	c7c224ee17	[NFC] polish colossalai/builder/pipeline.py code style (#638)	3 years ago
Sze-qq	10591ecdf9	[NFC] polish colossalai/kernel/cuda_native/csrc/cpu_adam.cpp code style (#636)	3 years ago
Wangbo Zhao	6fcb381801	[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu code style (#635)	3 years ago
ExtremeViscent	8a5d526e95	[NFC] polish colossalai/kernel/cuda_native/csrc/kernels/dropout_kernels.cu and cross_entropy.cu code style (#634)	3 years ago
RichardoLuo	ad1e7ab2b2	'[NFC] polish <colossalai/engine/_base_engine.py> code style' (#631) ... Co-authored-by: RichardoLuo <14049555596@qq.com>	3 years ago
Zangwei	2e11853d04	[NFC] polish colossalai/communication/ring.py code style (#630)	3 years ago
puck_WCR	01cc941e1d	[NFC] polish colossalai/kernel/cuda_native/csrc/kernels/transform_kernels.cu code stype (#629)	3 years ago
superhao1995	c1bed0d998	[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu code stype (#628)	3 years ago
Jiang Zhuo	0a96338b13	[NFC] polish <colossalai/context/process_group_initializer/initializer_data.py> code stype (#626) ... Co-authored-by: 姜卓 <jiangzhuo@jiangzhuodeMacBook-Pro.local>	3 years ago
ziyu huang	701bad439b	[NFC] polish colossalai/context/process_group_initializer/process_group_initializer.py code stype (#617) ... Co-authored-by: “Arsmart123 <202476410arsmart@gmail.com>	3 years ago
Shawn-Kong	db54419409	fix format (#613) ... Co-authored-by: evin K <evink@evins-MacBook-Air.local>	3 years ago
Yuer867	5ecef13c16	fix format (#611)	3 years ago
xyupeng	d3d5bedc65	fix format (#607)	3 years ago
xuqifan897	f2d2a1597a	fix format (#608)	3 years ago
doubleHU	f2da21a827	fix format (#586)	3 years ago
fanjinfucool	ffad81e1d1	fix format (#585) ... Co-authored-by: fanjifu <FAN>	3 years ago
binmakeswell	6582aedc94	fix format (#583)	3 years ago
DouJS	f08fc17f2b	block_reduce.h fix format (#581)	3 years ago
Maruyama_Aya	d2dc6049b5	fix format (#580)	3 years ago
wky	174b9c1d85	fix format (#574)	3 years ago
BoxiangW	dfe423ae42	fix format (#572)	3 years ago
yuxuan-lou	cfb41297ff	'fix/format' (#573)	3 years ago
Kai Wang (Victor Kai)	b0f708dfc1	fix format (#570)	3 years ago
Xu Kai	2a915a8b62	fix format (#568)	3 years ago
YuliangLiu0306	9420d3ae31	fix format (#567)	3 years ago
Jie Zhu	0f1da44e5e	[format]colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp (#566)	3 years ago
coder-chin	5835631218	fix format (#564)	3 years ago
Luxios22	e014144c44	fix format (#565)	3 years ago
Ziyue Jiang	1762ba14ab	fix format (#563)	3 years ago
HELSON	17e73e62cc	[hotfix] fix bugs for unsharded parameters when restore data (#664)	3 years ago