7 Commits (74257cb4461702deb357eb1b599788993a7757ad)

Author	SHA1	Message	Date
digger-yu	b9a8dff7e5	[doc] Fix typo under colossalai and doc(#3618) ... * Fixed several spelling errors under colossalai * Fix the spelling error in colossalai and docs directory * Cautious Changed the spelling error under the example folder * Update runtime_preparation_pass.py revert autograft to autograd * Update search_chunk.py utile to until * Update check_installation.py change misteach to mismatch in line 91 * Update 1D_tensor_parallel.md revert to perceptron * Update 2D_tensor_parallel.md revert to perceptron in line 73 * Update 2p5D_tensor_parallel.md revert to perceptron in line 71 * Update 3D_tensor_parallel.md revert to perceptron in line 80 * Update README.md revert to resnet in line 42 * Update reorder_graph.py revert to indice in line 7 * Update p2p.py revert to megatron in line 94 * Update initialize.py revert to torchrun in line 198 * Update routers.py change to detailed in line 63 * Update routers.py change to detailed in line 146 * Update README.md revert random number in line 402	2 years ago
LuGY	56ff1921e9	[NFC] polish colossalai/context/moe_context.py code style (#2693)	2 years ago
HELSON	1468e4bcfc	[zero] add constant placement policy (#1705) ... * fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.	2 years ago
HELSON	95c35f73bd	[moe] initialize MoE groups by ProcessGroup (#1640)	2 years ago
Jiarui Fang	a445e118cf	[polish] polish singleton and global context (#500)	3 years ago
Jiarui Fang	65c0f380c2	[format] polish name format for MOE (#481)	3 years ago
HELSON	84fd7c1d4d	add moe context, moe utilities and refactor gradient handler (#455)	3 years ago