You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Hongxin Liu ae02d4e4f7 [bf16] add bf16 support (#3882) ... * [bf16] add bf16 support for fused adam (#3844) * [bf16] fused adam kernel support bf16 * [test] update fused adam kernel test * [test] update fused adam test * [bf16] cpu adam and hybrid adam optimizers support bf16 (#3860) * [bf16] implement mixed precision mixin and add bf16 support for low level zero (#3869) * [bf16] add mixed precision mixin * [bf16] low level zero optim support bf16 * [text] update low level zero test * [text] fix low level zero grad acc test * [bf16] add bf16 support for gemini (#3872) * [bf16] gemini support bf16 * [test] update gemini bf16 test * [doc] update gemini docstring * [bf16] add bf16 support for plugins (#3877) * [bf16] add bf16 support for legacy zero (#3879) * [zero] init context support bf16 * [zero] legacy zero support bf16 * [test] add zero bf16 test * [doc] add bf16 related docstring for legacy zero		2 years ago
..
kernels	[doc] add deepspeed citation and copyright (#2996)	2 years ago
colossal_C_frontend.cpp	[optimizer] add div_scale for optimizers (#2117)	2 years ago
compat.h	refactor kernel (#142)	3 years ago
cpu_adam.cpp	[hotfix] fix CPUAdam kernel nullptr (#1410)	2 years ago
cpu_adam.h	[hotfix] fix CPUAdam kernel nullptr (#1410)	2 years ago
layer_norm_cuda.cpp	[NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp code style (#973)	3 years ago
layer_norm_cuda_kernel.cu	[NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu code style (#661)	3 years ago
moe_cuda.cpp	[NFC] polish colossalai/kernel/cuda_native/csrc/moe_cuda.cpp code style (#942)	3 years ago
moe_cuda_kernel.cu	[NFC] polish moe_cuda_kernel.cu code style (#940)	3 years ago
multi_tensor_adam.cu	[doc] add deepspeed citation and copyright (#2996)	2 years ago
multi_tensor_apply.cuh	[doc] add deepspeed citation and copyright (#2996)	2 years ago
multi_tensor_l2norm_kernel.cu	[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu code style (#958)	3 years ago
multi_tensor_lamb.cu	[NFC] Polish colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu code style. (#937)	3 years ago
multi_tensor_scale_kernel.cu	[NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu code style (#977)	3 years ago
multi_tensor_sgd_kernel.cu	[optim] refactor fused sgd (#1134)	2 years ago
multihead_attention_1d.cpp	[hotfix] fix error for torch 2.0 (#2243)	2 years ago
multihead_attention_1d.h	[hotfix] fix error for torch 2.0 (#2243)	2 years ago
scaled_masked_softmax.cpp	Recover kernal files	2 years ago
scaled_masked_softmax.h	Recover kernal files	2 years ago
scaled_masked_softmax_cuda.cu	[NFC] polish colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu code style (#949)	3 years ago
scaled_upper_triang_masked_softmax.cpp	[NFC] polish colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp code style (#959)	3 years ago
scaled_upper_triang_masked_softmax.h	Recover kernal files	2 years ago
scaled_upper_triang_masked_softmax_cuda.cu	[NFC] polish pre-commit run --files colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu code style (#943)	3 years ago
type_shim.h	[bf16] add bf16 support (#3882)	2 years ago