InternLM/internlm/core
Wenwen Qu 655e9dae40
Feat(norm)/support fused precision (#319)
* add fused precision support for norm

* refactor code

* refactor code

* change the granularity of hook

* fix bugs if self.model is ModuleList

* add dtype condition for post hook

* refactor code for split group

* refactor code for pre/post hook

* refactor code for split group

* remove fp32 hook for norm

* unit tests for fused precision

* add doc for fused precision

* add doc for En. version

* reformat docs

* Update mixed_precision.rst

* Update mixed_precision.po

* update mixed_precision.po
2023-09-26 20:39:55 +08:00
..
communication Merge develop to main (#233) 2023-08-24 22:03:04 +08:00
context feat(utils): add timeout warpper for key functions (#286) 2023-09-07 17:26:17 +08:00
scheduler Merge remote-tracking branch 'origin/main' into develop 2023-09-08 10:19:54 +08:00
__init__.py initial commit 2023-07-06 12:55:23 +08:00
engine.py initial commit 2023-07-06 12:55:23 +08:00
gradient_handler.py initial commit 2023-07-06 12:55:23 +08:00
naive_amp.py Feat(norm)/support fused precision (#319) 2023-09-26 20:39:55 +08:00
trainer.py feat: more tgs (#310) 2023-09-15 18:56:11 +08:00