ColossalAI/colossalai/engine/ophooks
HELSON a30e2b4c24
[zero] adapt for no-leaf module in zero (#535)
only process module's own parameters in Zero context

add zero hooks for all modules that contrain parameters

gather parameters only belonging to module itself
2022-03-28 17:42:18 +08:00
..
__init__.py [zero] adapt for no-leaf module in zero (#535) 2022-03-28 17:42:18 +08:00
_base_ophook.py add pytorch hooks (#179) 2022-01-25 22:20:54 +08:00
_memtracer_ophook.py Flake8 code restyle 2022-03-11 15:50:28 +08:00
_shard_grad_ophook.py [zero] add sharded grad and refactor grad hooks for ShardedModel (#287) 2022-03-11 15:50:28 +08:00
_shard_param_ophook.py fix sharded param hook and unit test 2022-03-11 15:50:28 +08:00
zero_hook.py [zero] adapt for no-leaf module in zero (#535) 2022-03-28 17:42:18 +08:00