282 Commits (feature/elixir)

Author	SHA1	Message	Date
digger yu	9265f2d4d7	[NFC]fix typo colossalai/auto_parallel nn utils etc. (#3779) ... * fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc.	2 years ago
digger-yu	b9a8dff7e5	[doc] Fix typo under colossalai and doc(#3618) ... * Fixed several spelling errors under colossalai * Fix the spelling error in colossalai and docs directory * Cautious Changed the spelling error under the example folder * Update runtime_preparation_pass.py revert autograft to autograd * Update search_chunk.py utile to until * Update check_installation.py change misteach to mismatch in line 91 * Update 1D_tensor_parallel.md revert to perceptron * Update 2D_tensor_parallel.md revert to perceptron in line 73 * Update 2p5D_tensor_parallel.md revert to perceptron in line 71 * Update 3D_tensor_parallel.md revert to perceptron in line 80 * Update README.md revert to resnet in line 42 * Update reorder_graph.py revert to indice in line 7 * Update p2p.py revert to megatron in line 94 * Update initialize.py revert to torchrun in line 198 * Update routers.py change to detailed in line 63 * Update routers.py change to detailed in line 146 * Update README.md revert random number in line 402	2 years ago
Hongxin Liu	152239bbfa	[gemini] gemini supports lazy init (#3379) ... * [gemini] fix nvme optimizer init * [gemini] gemini supports lazy init * [gemini] add init example * [gemini] add fool model * [zero] update gemini ddp * [zero] update init example * add chunk method * add chunk method * [lazyinit] fix lazy tensor tolist * [gemini] fix buffer materialization * [misc] remove useless file * [booster] update gemini plugin * [test] update gemini plugin test * [test] fix gemini plugin test * [gemini] fix import * [gemini] fix import * [lazyinit] use new metatensor * [lazyinit] use new metatensor * [lazyinit] fix __set__ method	2 years ago
ver217	26b7aac0be	[zero] reorganize zero/gemini folder structure (#3424) ... * [zero] refactor low-level zero folder structure * [zero] fix legacy zero import path * [zero] fix legacy zero import path * [zero] remove useless import * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] fix test import path * [zero] fix test * [zero] fix circular import * [zero] update import	2 years ago
HELSON	1a1d68b053	[moe] add checkpoint for moe models (#3354) ... * [moe] add checkpoint for moe models * [hotfix] fix bugs in unit test	2 years ago
Tong Li	196d4696d0	[NFC] polish colossalai/nn/_ops/addmm.py code style (#3274)	2 years ago
Yuanchen	d58fa705b2	[NFC] polish code style (#3268) ... Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>	2 years ago
github-actions[bot]	82503a96f2	[format] applied code formatting on changed files in pull request 2997 (#3008) ... Co-authored-by: github-actions <github-actions@github.com>	2 years ago
binmakeswell	52a5078988	[doc] add ISC tutorial (#2997) ... * [doc] add ISC tutorial * [doc] add ISC tutorial * [doc] add ISC tutorial * [doc] add ISC tutorial	2 years ago
ver217	823f3b9cf4	[doc] add deepspeed citation and copyright (#2996) ... * [doc] add deepspeed citation and copyright * [doc] add deepspeed citation and copyright * [doc] add deepspeed citation and copyright	2 years ago
zbian	61e687831d	fixed using zero with tp cannot access weight correctly	2 years ago
Jiatong (Julius) Han	8c8a39be95	[hotfix]: Remove math.prod dependency (#2837) ... * Remove math.prod dependency * Fix style * Fix style --------- Co-authored-by: Jiatong Han <jiatong.han@u.nus.edu>	2 years ago
junxu	c52edcf0eb	Rename class method of ZeroDDP (#2692)	2 years ago
HELSON	56ddc9ca7a	[hotfix] add correct device for fake_param (#2796)	2 years ago
HELSON	8213f89fd2	[gemini] add fake_release_chunk for keep-gathered chunk in the inference mode (#2671)	2 years ago
binmakeswell	9ab14b20b5	[doc] add CVPR tutorial (#2666)	2 years ago
ver217	5b1854309a	[hotfix] fix zero ddp warmup check (#2545)	2 years ago
HELSON	a4ed9125ac	[hotfix] fix lightning error (#2529)	2 years ago
HELSON	66dfcf5281	[gemini] update the gpt example (#2527)	2 years ago
HELSON	b528eea0f0	[zero] add zero wrappers (#2523) ... * [zero] add zero wrappers * change names * add wrapper functions to init	2 years ago
HELSON	707b11d4a0	[gemini] update ddp strict mode (#2518) ... * [zero] add strict ddp mode for chunk init * [gemini] update gpt example	2 years ago
HELSON	2d1a7dfe5f	[zero] add strict ddp mode (#2508) ... * [zero] add strict ddp mode * [polish] add comments for strict ddp mode * [zero] fix test error	2 years ago
HELSON	2bfeb24308	[zero] add warning for ignored parameters (#2446)	2 years ago
HELSON	5521af7877	[zero] fix state_dict and load_state_dict for ddp ignored parameters (#2443) ... * [ddp] add is_ddp_ignored [ddp] rename to is_ddp_ignored * [zero] fix state_dict and load_state_dict * fix bugs * [zero] update unit test for ZeroDDP	2 years ago
HELSON	7829aa094e	[ddp] add is_ddp_ignored (#2434) ... [ddp] rename to is_ddp_ignored	2 years ago
HELSON	bb4e9a311a	[zero] add inference mode and its unit test (#2418)	2 years ago
HELSON	dddacd2d2c	[hotfix] add norm clearing for the overflow step (#2416)	2 years ago
HELSON	ea13a201bb	[polish] polish code for get_static_torch_model (#2405) ... * [gemini] polish code * [testing] remove code * [gemini] make more robust	2 years ago
Frank Lee	551cafec14	[doc] updated kernel-related optimisers' docstring (#2385) ... * [doc] updated kernel-related optimisers' docstring * polish doc	2 years ago
eric8607242	9880fd2cd8	Fix state_dict key missing issue of the ZeroDDP (#2363) ... * Fix state_dict output for ZeroDDP duplicated parameters * Rewrite state_dict based on get_static_torch_model * Modify get_static_torch_model to be compatible with the lower version (ZeroDDP)	2 years ago
Frank Lee	40d376c566	[setup] support pre-build and jit-build of cuda kernels (#2374) ... * [setup] support pre-build and jit-build of cuda kernels * polish code * polish code * polish code * polish code * polish code * polish code	2 years ago
HELSON	48d33b1b17	[gemini] add get static torch model (#2356)	2 years ago
Jiarui Fang	16cc8e6aa7	[builder] MOE builder (#2277)	2 years ago
zbian	e94c79f15b	improved allgather & reducescatter for 3d	2 years ago
Jiarui Fang	af32022f74	[Gemini] fix the convert_to_torch_module bug (#2269)	2 years ago
HELSON	2458659919	[zero] fix error for BEiT models (#2169) ... * [zero] fix error for BEiT models * [ColoParameter] add unpack operation for tuple arguments * fix bugs * fix chunkv2 unit testing * add assertion for gradient state	2 years ago
Jiarui Fang	355ffb386e	[builder] unified cpu_optim fused_optim inferface (#2190)	2 years ago
Jiarui Fang	9587b080ba	[builder] use runtime builder for fused_optim (#2189)	2 years ago
Jiarui Fang	d42afd30f8	[builder] runtime adam and fused_optim builder (#2184)	2 years ago
Tongping Liu	ab54fed292	[hotfix] add kwargs for colo_addmm (#2171)	2 years ago
アマデウス	622f863291	[hotfix] Jit type hint #2161 (#2164)	2 years ago
Jiarui Fang	2827f41898	[Gemini] GeminiDPP convert to PyTorch Module. (#2151)	2 years ago
Jiarui Fang	bdef9dfdbe	[NFC] remove useless graph node code (#2150)	2 years ago
Jiarui Fang	9214d1fe28	[Gemini] chunk init using runtime visited param order (#2115)	2 years ago
HELSON	e7d3afc9cc	[optimizer] add div_scale for optimizers (#2117) ... * [optimizer] add div_scale for optimizers * [zero] use div_scale in zero optimizer * fix testing error	2 years ago
Jiarui Fang	e5aa8333e4	[NFC] update chunk manager API (#2119)	2 years ago
Jiarui Fang	e99edfcb51	[NFC] polish comments for Chunk class (#2116)	2 years ago
HELSON	63fbba3c19	[zero] add L2 gradient clipping for ZeRO (#2112) ... * [zero] add L2 gradient clipping * [testing] add MlpModel * [zero] add unit test for grad clipping * fix atol	2 years ago
Jiarui Fang	1f99205827	[Gemini] remove static tracer (#2083)	2 years ago
Jiarui Fang	b3b89865e2	[Gemini] ParamOpHook -> ColoParamOpHook (#2080)	2 years ago