HELSON 1468e4bcfc [zero] add constant placement policy (#1705) ... * fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.		2022-10-14 17:53:16 +08:00
..
checkpoint	…
data_sampler	…
model	[zero] add constant placement policy (#1705)	2022-10-14 17:53:16 +08:00
multi_tensor_apply	[NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style (#1559)	2022-09-08 22:04:34 +08:00
profiler	…
rank_recorder	[pipeline/rank_recorder] fix bug when process data before backward | add a tool for multiple ranks debug (#1681)	2022-10-09 17:32:57 +08:00
tensor_detector	[NFC] polish utils/tensor_detector/__init__.py code style (#1573)	2022-09-08 22:11:04 +08:00
__init__.py	…
activation_checkpoint.py	…
checkpointing.py	…
common.py	…
cuda.py	…
memory.py	…
moe.py	…
timer.py	…