4 Commits (a3e549615627c7893f1b7189719644a02d0f0319)

Author	SHA1	Message	Date
HELSON	dddacd2d2c	[hotfix] add norm clearing for the overflow step (#2416)	2023-01-10 15:43:06 +08:00
HELSON	e7d3afc9cc	[optimizer] add div_scale for optimizers (#2117) ... * [optimizer] add div_scale for optimizers * [zero] use div_scale in zero optimizer * fix testing error	2022-12-12 17:58:57 +08:00
HELSON	63fbba3c19	[zero] add L2 gradient clipping for ZeRO (#2112) ... * [zero] add L2 gradient clipping * [testing] add MlpModel * [zero] add unit test for grad clipping * fix atol	2022-12-09 18:09:17 +08:00
Jiarui Fang	f7e276fa71	[Gemini] add GeminiAdamOptimizer (#1960)	2022-11-16 14:44:28 +08:00