You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ColossalAI/colossalai
HELSON d7ea63992b
[MOE] add FP32LinearGate for MOE in NaiveAMP context (#480)
3 years ago
..
amp [hotfix] fix initialize bug with zero (#442) 3 years ago
builder add pytorch hooks (#179) 3 years ago
communication fix format (#332) 3 years ago
context [format] polish name format for MOE (#481) 3 years ago
engine [format] polish name format for MOE (#481) 3 years ago
kernel [formart] format fixed for kernel\cuda_native codes (#335) 3 years ago
logging [log] better logging display with rich (#426) 3 years ago
nn [MOE] add FP32LinearGate for MOE in NaiveAMP context (#480) 3 years ago
registry add pytorch hooks (#179) 3 years ago
testing [test] added rerun on exception for testing (#475) 3 years ago
trainer Added profiler communication operations 3 years ago
utils [format] polish name format for MOE (#481) 3 years ago
zero [zero] ZeRO supports pipeline parallel (#477) 3 years ago
__init__.py Develop/experiments (#59) 3 years ago
constants.py fix format constants.py (#358) 3 years ago
core.py [MOE] polish moe_env (#467) 3 years ago
global_variables.py [MOE] add unitest for MOE experts layout, gradient handler and kernel (#469) 3 years ago
initialize.py [MOE] add unitest for MOE experts layout, gradient handler and kernel (#469) 3 years ago