flybird1111 7a3dfd0c64 [shardformer] update shardformer to use flash attention 2 (#4392) ... * cherry-pick flash attention 2 cherry-pick flash attention 2 * [shardformer] update shardformer to use flash attention 2 [shardformer] update shardformer to use flash attention 2, fix [shardformer] update shardformer to use flash attention 2, fix [shardformer] update shardformer to use flash attention 2, fix		2023-08-15 23:25:14 +08:00
..
test_checkpoint	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00
test_activation_checkpointing.py	[CI] fix some spelling errors (#3707)	2023-05-10 17:12:03 +08:00
test_colo_checkpoint.py	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00
test_commons.py	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00
test_flash_attention.py	[shardformer] update shardformer to use flash attention 2 (#4392)	2023-08-15 23:25:14 +08:00
test_memory.py	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00
test_norm_gradient_clipping.py	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00
test_zero_gradient_clippling.py	[test] refactor tests with spawn (#3452)	2023-04-06 14:51:35 +08:00