mirror of https://github.com/hpcaitech/ColossalAI
e6d50ec107
* support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler |
||
|---|---|---|
| .. | ||
| components_to_test | ||
| test_amp | ||
| test_comm | ||
| test_config | ||
| test_context | ||
| test_data | ||
| test_data_pipeline_tensor_parallel | ||
| test_engine | ||
| test_layers | ||
| test_moe | ||
| test_optimizer | ||
| test_trainer | ||
| test_utils | ||
| test_zero_data_parallel | ||
| __init__.py | ||