ColossalAI

Commit Graph

Author	SHA1	Message	Date
Frank Lee	80eba05b0a	[test] refactor tests with spawn (#3452 ) * [test] added spawn decorator * polish code * polish code * polish code * polish code * polish code * polish code	2023-04-06 14:51:35 +08:00
ver217	933048ad3e	[test] reorganize zero/gemini tests (#3445 )	2023-04-06 09:38:25 +08:00
ver217	26b7aac0be	[zero] reorganize zero/gemini folder structure (#3424 ) * [zero] refactor low-level zero folder structure * [zero] fix legacy zero import path * [zero] fix legacy zero import path * [zero] remove useless import * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor gemini folder structure * [zero] refactor legacy zero import path * [zero] fix test import path * [zero] fix test * [zero] fix circular import * [zero] update import	2023-04-04 13:48:16 +08:00
Jiarui Fang	1e885329f4	[test] align model name with the file name. (#2045 )	2022-11-30 15:45:26 +08:00
HELSON	a088022efc	[moe] fix moe bugs (#1633 )	2022-09-23 15:33:57 +08:00
HELSON	f7f2248771	[moe] fix MoE bugs (#1628 ) * remove forced FP32 modules * correct no_shard-contexts' positions	2022-09-22 13:56:30 +08:00
Frank Lee	5a1a095b92	[test] refactored with the new rerun decorator (#763 ) * [test] refactored with the new rerun decorator * polish test case	2022-04-15 00:33:04 +08:00
ver217	e396bb71f2	[zero] add tensor placement policies (#743 ) * add tensor placement policies * polish comments * polish comments * update moe unit tests	2022-04-13 15:00:48 +08:00
HELSON	22c4b88d56	[zero] refactor ShardedParamV2 for convenience (#742 )	2022-04-13 14:54:26 +08:00
Jiarui Fang	53cb584808	[utils] correct cpu memory used and capacity in the context of multi-process (#726 )	2022-04-12 14:57:54 +08:00
HELSON	b9b469ea50	[moe] add checkpoint for moe zero test (#729 )	2022-04-12 12:11:54 +08:00
Jiarui Fang	193dc8dacb	[refactor] refactor the memory utils (#715 )	2022-04-11 16:47:57 +08:00
HELSON	a9b8300d54	[zero] improve adaptability for not-shard parameters (#708 ) * adapt post grad hooks for not-shard parameters * adapt optimizer for not-shard parameters * offload gradients for not-replicated parameters	2022-04-11 13:38:51 +08:00
HELSON	ee112fe1da	[zero] adapt zero hooks for unsharded module (#699 )	2022-04-08 20:23:26 +08:00
HELSON	d7ecaf362b	[zero] fix init bugs in zero context (#686 ) * adapt model weight initialization for methods in Pytorch nn.init	2022-04-07 17:38:45 +08:00
HELSON	e5d615aeee	[hotfix] fix bugs in testing (#659 ) * remove hybrid adam in test_moe_zero_optim * fix activation checkpointing and its unitest	2022-04-02 21:58:47 +08:00
HELSON	b31daed4cf	fix bugs in CPU adam (#633 ) * add cpu adam counter for all cpu adam * fixed updating error in adam kernel	2022-04-02 17:04:05 +08:00
HELSON	055fbf5be6	[zero] adapt zero for unsharded paramters (Optimizer part) (#601 )	2022-04-01 20:10:47 +08:00

18 Commits (890774b2fba6b5cb737b00466c89334b73a2be69)