6 Commits (638a07a7f9b504e6c9781e9aa2a9b6c5e9dc49ed)

Author	SHA1	Message	Date
LuGY	56ff1921e9	[NFC] polish colossalai/context/moe_context.py code style (#2693)	2 years ago
HELSON	1468e4bcfc	[zero] add constant placement policy (#1705) ... * fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.	2 years ago
HELSON	95c35f73bd	[moe] initialize MoE groups by ProcessGroup (#1640)	2 years ago
Jiarui Fang	a445e118cf	[polish] polish singleton and global context (#500)	3 years ago
Jiarui Fang	65c0f380c2	[format] polish name format for MOE (#481)	3 years ago
HELSON	84fd7c1d4d	add moe context, moe utilities and refactor gradient handler (#455)	3 years ago