2 Commits (f4c5aafe2987c0c78c4989c178874dc8c1cc7369)

Author	SHA1	Message	Date
Hongxin Liu	e094933da1	[shardformer] fix pipeline grad ckpt (#5620) ... * [shardformer] fix pipeline grad ckpt	2024-04-22 11:25:39 +08:00
Wenhao Chen	e614aa34f3	[shardformer, pipeline] add `gradient_checkpointing_ratio` and heterogenous shard policy for llama (#5508) ... * feat: add `GradientCheckpointConfig` and `PipelineGradientCheckpointConfig` * feat: apply `GradientCheckpointConfig` to policy and llama_forward * feat: move `distribute_layer` and `get_stage_index` to PipelineStageManager * fix: add optional args for `distribute_layer` and `get_stage_index` * fix: fix changed API calls * test: update llama tests * style: polish `GradientCheckpointConfig` * fix: fix pipeline utils tests	2024-04-01 11:34:58 +08:00