ColossalAI/tests/test_shardformer/test_layer
Insu Jang 00525f7772
[shardformer] fix pipeline forward error if custom layer distribution is used (#5189)
* Use self.[distribute_layers|get_stage_index] to exploit custom layer distribution

* Change static methods for t5 layer distribution to member functions

* Change static methods for whisper layer distribution to member functions

* Replace whisper policy usage with self one

* Fix test case to use non-static layer distribution methods

* fix: fix typo

---------

Co-authored-by: Wenhao Chen <cwher@outlook.com>
2024-03-27 13:57:00 +08:00
..
test_dist_crossentropy.py [shardformer] fix pipeline forward error if custom layer distribution is used (#5189) 2024-03-27 13:57:00 +08:00
test_dropout.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
test_embedding.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
test_gpt2_qkv_fused_linear_1d.py [example]add gpt2 benchmark example script. (#5295) 2024-03-04 16:18:13 +08:00
test_layernorm.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
test_linear_1d.py [example]add gpt2 benchmark example script. (#5295) 2024-03-04 16:18:13 +08:00
test_qkv_fused_linear_1d.py [example]add gpt2 benchmark example script. (#5295) 2024-03-04 16:18:13 +08:00
test_vocab_parallel_embedding_1d.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00