HELSON 1468e4bcfc [zero] add constant placement policy (#1705) ... * fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.		2022-10-14 17:53:16 +08:00
..
test_chunk_mgrv2.py	[feature] A new ZeRO implementation (#1644)	2022-10-09 09:18:51 +08:00
test_chunkv2.py	[feature] A new ZeRO implementation (#1644)	2022-10-09 09:18:51 +08:00
test_fwd_bwd.py	[zero] add constant placement policy (#1705)	2022-10-14 17:53:16 +08:00
test_optim.py	[zero] add constant placement policy (#1705)	2022-10-14 17:53:16 +08:00
test_search.py	[feature] A new ZeRO implementation (#1644)	2022-10-09 09:18:51 +08:00
test_zeroddp_state_dict.py	[zero] add constant placement policy (#1705)	2022-10-14 17:53:16 +08:00
test_zerooptim_state_dict.py	[feature] A new ZeRO implementation (#1644)	2022-10-09 09:18:51 +08:00