mirror of https://github.com/hpcaitech/ColossalAI
e6d50ec107
* support existing sharded and unsharded parameters in zero * add unitest for moe-zero model init * polish moe gradient handler |
||
|---|---|---|
| .. | ||
| data_sampler | ||
| gradient_accumulation | ||
| memory_tracer | ||
| memory_utils | ||
| multi_tensor_apply | ||
| profiler | ||
| tensor_detector | ||
| __init__.py | ||
| activation_checkpoint.py | ||
| checkpointing.py | ||
| common.py | ||
| cuda.py | ||
| moe.py | ||
| timer.py | ||