ColossalAI/colossalai/kernel/cuda_native/csrc
HELSON e7d3afc9cc
[optimizer] add div_scale for optimizers (#2117)
* [optimizer] add div_scale for optimizers

* [zero] use div_scale in zero optimizer

* fix testing error
2022-12-12 17:58:57 +08:00
..
kernels Recover kernal files 2022-07-13 12:08:21 +08:00
colossal_C_frontend.cpp [optimizer] add div_scale for optimizers (#2117) 2022-12-12 17:58:57 +08:00
compat.h refactor kernel (#142) 2022-01-13 16:47:17 +08:00
cpu_adam.cpp [hotfix] fix CPUAdam kernel nullptr (#1410) 2022-08-05 19:45:45 +08:00
cpu_adam.h [hotfix] fix CPUAdam kernel nullptr (#1410) 2022-08-05 19:45:45 +08:00
layer_norm_cuda.cpp [NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda.cpp code style (#973) 2022-05-17 10:25:06 +08:00
layer_norm_cuda_kernel.cu [NFC] polish colossalai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu code style (#661) 2022-04-06 11:40:59 +08:00
moe_cuda.cpp [NFC] polish colossalai/kernel/cuda_native/csrc/moe_cuda.cpp code style (#942) 2022-05-17 10:25:06 +08:00
moe_cuda_kernel.cu [NFC] polish moe_cuda_kernel.cu code style (#940) 2022-05-17 10:25:06 +08:00
multi_tensor_adam.cu [optimizer] add div_scale for optimizers (#2117) 2022-12-12 17:58:57 +08:00
multi_tensor_apply.cuh Recover kernal files 2022-07-13 12:08:21 +08:00
multi_tensor_l2norm_kernel.cu [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_l2norm_kernel.cu code style (#958) 2022-05-17 10:25:06 +08:00
multi_tensor_lamb.cu [NFC] Polish colossalai/kernel/cuda_native/csrc/multi_tensor_lamb.cu code style. (#937) 2022-05-17 10:25:06 +08:00
multi_tensor_scale_kernel.cu [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_scale_kernel.cu code style (#977) 2022-05-17 10:25:06 +08:00
multi_tensor_sgd_kernel.cu [optim] refactor fused sgd (#1134) 2022-06-20 11:19:38 +08:00
multihead_attention_1d.cpp [hotfix] fix build error when torch version >= 1.13 (#1803) 2022-11-08 09:40:24 +08:00
multihead_attention_1d.h [hotfix] fix build error when torch version >= 1.13 (#1803) 2022-11-08 09:40:24 +08:00
scaled_masked_softmax.cpp Recover kernal files 2022-07-13 12:08:21 +08:00
scaled_masked_softmax.h Recover kernal files 2022-07-13 12:08:21 +08:00
scaled_masked_softmax_cuda.cu [NFC] polish colossalai/kernel/cuda_native/csrc/scaled_masked_softmax_cuda.cu code style (#949) 2022-05-17 10:25:06 +08:00
scaled_upper_triang_masked_softmax.cpp [NFC] polish colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax.cpp code style (#959) 2022-05-17 10:25:06 +08:00
scaled_upper_triang_masked_softmax.h Recover kernal files 2022-07-13 12:08:21 +08:00
scaled_upper_triang_masked_softmax_cuda.cu [NFC] polish pre-commit run --files colossalai/kernel/cuda_native/csrc/scaled_upper_triang_masked_softmax_cuda.cu code style (#943) 2022-05-17 10:25:06 +08:00
type_shim.h Recover kernal files 2022-07-13 12:08:21 +08:00