mirror of https://github.com/hpcaitech/ColossalAI
1468e4bcfc
* fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters. |
||
|---|---|---|
| .. | ||
| checkpoint | ||
| data_sampler | ||
| model | ||
| multi_tensor_apply | ||
| profiler | ||
| rank_recorder | ||
| tensor_detector | ||
| __init__.py | ||
| activation_checkpoint.py | ||
| checkpointing.py | ||
| common.py | ||
| cuda.py | ||
| memory.py | ||
| moe.py | ||
| timer.py | ||