8795bb2e80
Support 4d parallel + flash attention ( #5789 )
...
* support tp + sp + pp
* remove comments
---------
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
2024-06-17 17:40:47 +08:00
2ddf624a86
[shardformer] upgrade transformers to 4.39.3 ( #5815 )
...
* [shardformer]upgrade transformers for gpt2/gptj/whisper (#5807 )
* [shardformer] fix modeling of gpt2 and gptj
* [shardformer] fix whisper modeling
* [misc] update requirements
---------
Co-authored-by: ver217 <lhx0217@gmail.com>
* [shardformer]upgrade transformers for mistral (#5808 )
* upgrade transformers for mistral
* fix
* fix
* [shardformer]upgrade transformers for llama (#5809 )
* update transformers
fix
* fix
* fix
* [inference] upgrade transformers (#5810 )
* update transformers
fix
* fix
* fix
* fix
* fix
* [gemini] update transformers for gemini (#5814 )
---------
Co-authored-by: ver217 <lhx0217@gmail.com>
2024-06-14 10:59:33 +08:00
3bcbba9262
[gemini] quick fix on possible async operation ( #5803 )
...
* [gemini] quick fix on possible async operation
* [gemini] quick fix on possible async operation
2024-06-13 10:35:17 +08:00
d9dddf574f
[Gemini] Use async stream to prefetch and h2d data moving ( #5781 )
...
* use async stream to prefetch and h2d data moving
* Remove redundant code
2024-06-12 15:48:52 +08:00
8554585a5f
[Inference] Fix flash-attn import and add model test ( #5794 )
...
* Fix torch int32 dtype
Signed-off-by: char-1ee <xingjianli59@gmail.com>
* Fix flash-attn import
Signed-off-by: char-1ee <xingjianli59@gmail.com>
* Add generalized model test
Signed-off-by: char-1ee <xingjianli59@gmail.com>
* Remove exposed path to model
Signed-off-by: char-1ee <xingjianli59@gmail.com>
* Add default value for use_flash_attn
Signed-off-by: char-1ee <xingjianli59@gmail.com>
* Rename model test
Signed-off-by: char-1ee <xingjianli59@gmail.com>
---------
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-12 14:13:50 +08:00
aac941ef78
[test] fix qwen2 pytest distLarge ( #5797 )
2024-06-12 12:13:51 +08:00
aa125bcc91
[shardformer] fix modeling of bloom and falcon ( #5796 )
2024-06-11 17:43:50 +08:00
587bbf4c6d
[test] fix chatglm test kit ( #5793 )
2024-06-11 16:54:31 +08:00
74f4a29734
Merge pull request #5759 from hpcaitech/colossalchat_upgrade
...
[ColossalChat] Colossalchat upgrade
2024-06-11 12:49:53 +08:00
c0948aff97
[Inference]refactor baichuan ( #5791 )
...
* refactor baichuan
* remove unused code and add TODO for lazyinit
2024-06-11 10:52:01 +08:00
84eab13078
update sft trainning script
2024-06-11 02:44:20 +00:00
77a219a082
Merge pull request #5771 from char-1ee/refactor/modeling
...
[Inference] Refactor modeling attention layer by abstracting attention backends
2024-06-10 11:52:22 +08:00
b303976a27
Fix test import
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-10 02:03:30 +00:00
2abdede1d7
fix readme
2024-06-10 01:08:42 +00:00
f5981e808e
Remove flash attention backend
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-07 10:02:19 +00:00
77db21610a
replace the customized dataloader setup with the build-in one
2024-06-07 09:44:25 +00:00
0d7ff10ea5
replace the customized dataloader setup with the build-in one
2024-06-07 09:43:42 +00:00
ceba662d22
Clean up
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-07 09:09:29 +00:00
5f398fc000
Pass inference model shard configs for module init
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-07 08:33:52 +00:00
eec77e5702
Fix tests and naming
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-07 08:33:47 +00:00
04386d9eff
Refactor modeling by adding attention backend
...
Signed-off-by: char-1ee <xingjianli59@gmail.com>
2024-06-07 08:33:47 +00:00
790e1362a6
merge
2024-06-07 07:01:32 +00:00
ac1520cb8f
remove baichuan from template test due to transformer version conflict
2024-06-07 07:01:32 +00:00
e16ccc272a
update ci
2024-06-07 07:01:32 +00:00
45195ac53d
remove local data path
2024-06-07 07:01:31 +00:00
bf57b13dda
remove models that require huggingface auth from ci
2024-06-07 07:01:31 +00:00
0bbac158ed
fix datasets version
2024-06-07 07:01:31 +00:00
62eb28b929
remove duplicated test
2024-06-07 07:01:31 +00:00
b8b5cacf38
fix transformers version
2024-06-07 07:01:31 +00:00
1b880ce095
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2024-06-07 07:01:31 +00:00
b1031f7244
fix ci
2024-06-07 07:01:31 +00:00
7ae87b3159
fix training script
2024-06-07 07:01:31 +00:00
0b4a33548c
moupdate ci tests, st ci test cases passed, tp failed in generation for ppo, sp is buggy
2024-06-07 07:01:31 +00:00
7e65b71815
run pre-commit
2024-06-07 07:01:30 +00:00
929e1e3da4
upgrade ppo dpo rm script
2024-06-07 07:01:30 +00:00
7a7e86987d
upgrade colossal-chat support tp_group>1, add sp for sft
2024-06-07 07:01:30 +00:00
73e88a5553
[shardformer] fix import ( #5788 )
2024-06-06 19:09:50 +08:00
5ead00ffc5
[misc] update requirements ( #5787 )
2024-06-06 15:55:34 +08:00
a1e39f4c0d
[install]fix setup ( #5786 )
...
* fix
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-06-06 11:47:48 +08:00
b9d646fe9e
[misc] fix dist logger ( #5782 )
2024-06-05 15:04:22 +08:00
c46e09715c
Allow building cuda extension without a device. ( #5535 )
...
Added FORCE_CUDA environment variable support, to enable building extensions where a GPU device is not present but cuda libraries are.
2024-06-05 14:26:30 +08:00
3f7e3131d9
[gemini] optimize reduce scatter d2h copy ( #5760 )
...
* [gemini] optimize reduce scatter d2h copy
* [fix] fix missing reduce variable
* [refactor] remove legacy async reduce scatter code
* [gemini] missing sync
* Revert "[refactor] remove legacy async reduce scatter code"
This reverts commit 58ad76d466
2024-06-05 14:23:13 +08:00
10a19e22c6
[hotfix] fix testcase in test_fx/test_tracer ( #5779 )
...
* [fix] branch for fix testcase;
* [fix] fix test_analyzer & test_auto_parallel;
* [fix] remove local change about moe;
* [fix] rm local change moe;
* [fix] fix test_deepfm_model & test_dlrf_model；
* [fix] fix test_hf_albert & test_hf_gpt;
2024-06-05 11:29:32 +08:00
80c3c8789b
[Test/CI] remove test cases to reduce CI duration ( #5753 )
...
* [test] smaller gpt2 test case
* [test] reduce test cases: tests/test_zero/test_gemini/test_zeroddp_state_dict.py
* [test] reduce test cases: tests/test_zero/test_gemini/test_grad_accum.py
* [test] reduce test cases tests/test_zero/test_gemini/test_optim.py
* Revert "[test] smaller gpt2 test case"
Some tests might depend on the size of model (num of chunks)
This reverts commit df705a5210
2024-06-05 11:29:04 +08:00
79f7a7b211
[misc] Accelerate CI for zero and dist optim ( #5758 )
...
* remove fp16 from lamb
* remove d2h copy in checking states
---------
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
2024-06-05 11:25:19 +08:00
50b4c8e8cf
[hotfix] fix llama flash attention forward ( #5777 )
2024-06-05 10:56:47 +08:00
b45000f839
[Inference]Add Streaming LLM ( #5745 )
...
* Add Streaming LLM
* add some parameters to llama_generation.py
* verify streamingllm config
* add test_streamingllm.py
* modified according to the opinions of review
* add Citation
* change _block_tables tolist
2024-06-05 10:51:19 +08:00
ee6fd38373
[devops] fix docker ci ( #5780 )
2024-06-04 17:47:39 +08:00
32f4187806
[misc] update dockerfile ( #5776 )
...
* [misc] update dockerfile
* [misc] update dockerfile
2024-06-04 16:15:41 +08:00
e22b82755d
[CI/tests] simplify some test case to reduce testing time ( #5755 )
...
* [ci/tests] simplify some test case to reduce testing time
* [ci/tests] continue to remove test case to reduce ci time cost
* restore some test config
* [ci/tests] continue to reduce ci time cost
2024-06-04 13:57:54 +08:00
Edenzzzz