github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/tests/test_shardformer/test_model
History
Hongxin Liu 19e1a5cf16
[shardformer] update colo attention to support custom mask (#5510) 
* [feature] refactor colo attention (#5462)

* [extension] update api

* [feature] add colo attention

* [feature] update sdpa

* [feature] update npu attention

* [feature] update flash-attn

* [test] add flash attn test

* [test] update flash attn test

* [shardformer] update modeling to fit colo attention (#5465)

* [misc] refactor folder structure

* [shardformer] update llama flash-attn

* [shardformer] fix llama policy

* [devops] update tensornvme install

* [test] update llama test

* [shardformer] update colo attn kernel dispatch

* [shardformer] update blip2

* [shardformer] update chatglm

* [shardformer] update gpt2

* [shardformer] update gptj

* [shardformer] update opt

* [shardformer] update vit

* [shardformer] update colo attention mask prep

* [shardformer] update whisper

* [test] fix shardformer tests (#5514)

* [test] fix shardformer tests

* [test] fix shardformer tests
 		2024-03-27 11:19:32 +08:00
..
__init__.py [shardformer] adapted T5 and LLaMa test to use kit (#4049) 2023-07-04 16:05:01 +08:00
_utils.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_bert.py [pipeline]: fix p2p comm, add metadata cache and support llama interleaved pp (#5134) 2023-12-22 10:44:00 +08:00
test_shard_blip2.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_bloom.py [hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926) 2023-11-03 13:32:43 +08:00
test_shard_chatglm2.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_falcon.py [devops] fix compatibility (#5444) 2024-03-13 15:24:13 +08:00
test_shard_gpt2.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_gptj.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_llama.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_mistral.py [shardformer]: support gpt-j, falcon, Mistral and add interleaved pipeline for bert (#5088) 2023-11-28 16:54:42 +08:00
test_shard_opt.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_sam.py [misc] update pre-commit and run all files (#4752) 2023-09-19 14:20:26 +08:00
test_shard_t5.py [shardformer] update colo attention to support custom mask (#5510) 2024-03-27 11:19:32 +08:00
test_shard_vit.py [hotfix] fix torch 2.0 compatibility (#4936) 2023-10-18 11:05:25 +08:00
test_shard_whisper.py [ci] fix shardformer tests. (#5255) 2024-01-11 19:07:45 +08:00