3 Commits (5ed5e8cfbaa7cc56a2f8716a2fbbf7139b9071f1)

Author	SHA1	Message	Date
Hongxin Liu	1b387ca9fe	[shardformer] refactor pipeline grad ckpt config (#5646) ... * [shardformer] refactor pipeline grad ckpt config * [shardformer] refactor pipeline grad ckpt config * [pipeline] fix stage manager	2024-04-25 15:19:30 +08:00
Hongxin Liu	e094933da1	[shardformer] fix pipeline grad ckpt (#5620) ... * [shardformer] fix pipeline grad ckpt	2024-04-22 11:25:39 +08:00
Wenhao Chen	e614aa34f3	[shardformer, pipeline] add `gradient_checkpointing_ratio` and heterogenous shard policy for llama (#5508) ... * feat: add `GradientCheckpointConfig` and `PipelineGradientCheckpointConfig` * feat: apply `GradientCheckpointConfig` to policy and llama_forward * feat: move `distribute_layer` and `get_stage_index` to PipelineStageManager * fix: add optional args for `distribute_layer` and `get_stage_index` * fix: fix changed API calls * test: update llama tests * style: polish `GradientCheckpointConfig` * fix: fix pipeline utils tests	2024-04-01 11:34:58 +08:00