ColossalAI/colossalai/zero
HELSON a30e2b4c24
[zero] adapt for no-leaf module in zero (#535)
only process module's own parameters in Zero context

add zero hooks for all modules that contrain parameters

gather parameters only belonging to module itself
2022-03-28 17:42:18 +08:00
..
init_ctx [zero] adapt for no-leaf module in zero (#535) 2022-03-28 17:42:18 +08:00
shard_utils [zero] refactor model data tracing (#537) 2022-03-28 16:38:18 +08:00
sharded_model [zero] refactor model data tracing (#537) 2022-03-28 16:38:18 +08:00
sharded_optim [zero] refactor model data tracing (#537) 2022-03-28 16:38:18 +08:00
sharded_param [zero] improve the accuracy of get_memory_usage of sharded param (#538) 2022-03-28 16:19:19 +08:00
__init__.py [refactor] remove old zero code (#517) 2022-03-25 14:54:39 +08:00