ColossalAI

Commit Graph

Author	SHA1	Message	Date
FoolPlayer	dd2bf02679	[shardformer] support SAM (#4231 ) * 1.support sam 2.add fused qkv for nn.Linear * update utils support set element in list * overtwrite SamVisionAttention foward to use DropoutForParallelInput * remove unused code	2023-08-15 23:25:14 +08:00
Baizhou Zhang	0ceec8f9a9	[pipeline] support fp32 for HybridPlugin/merge shardformer test and pipeline test into one file (#4354 ) * add naive optimizer for 3DPlugin/refactor gpt2 shardformer test * merge tests of PP/DP/TP combinations into one test file * fix bug when sync grad for dp in HybridPlugin * update supported precisions for 3DPlugin/fix bug when shifting tp_degree * improve the passing of lazy_init * modify lazy_init/use sync_shared_params	2023-08-15 23:25:14 +08:00
FoolPlayer	b3f5d7a3ba	[shardformer] support pipeline base vit model (#4284 ) * Feature/vit support (#4182) * [shardformer] added tests * [shardformer] vit test finish and support * fix attention dropout * support base vit pipeline * support vit downstream model * fix vit shard test * modify hidden states return type --------- Co-authored-by: Kun Lin <81014421+klhhhhh@users.noreply.github.com>	2023-08-15 23:25:14 +08:00
Baizhou Zhang	36e546b2cc	[pipeline] add pipeline support for T5Stack/T5EncoderModel (#4300 ) * modify t5 policy & add test * pipeline stage distribution for t5 * complete t5 base policy * t5 stack: halfway * modify gpt2 pipeline test * complete pipeline forward for T5Stack/T5EncoderModel * fix docstring * move t5 util tests to test_pipeline	2023-08-15 23:25:14 +08:00
Baizhou Zhang	2a2eacfaf1	[pipeline] support shardformer for GPT2ForQuestionAnswering & complete pipeline support for GPT2 (#4245 ) * change for transformers loggers * add forward for GPT2ForQuestionAnswering * fix assert * fix torchrec test	2023-08-15 23:25:14 +08:00
Jianghai	e7cc62d735	[pipeline] All bert models (#4233 ) * bloom policy * llama pipeline forward and tests * fix the output and attention_mask * fix name * bind argument to policy * Revert "bloom policy" This reverts commit `8dee68a0a2`. This policy should be revert and copied to feature/bloom * revert the bloom changes * cancel unneeded inputs * gpt * finish llama * causal lm and sequence classification * revision * add pure pipeline test * finish some bert models * finish all bert models * finish bert tests * fix bugs * fix bugs * fix test pipeline * fix data gen for qa * update the set pipeline forward * shared params * fix bugs	2023-08-15 23:25:14 +08:00
Jianghai	37d22f6878	[pipeline] add bloom model pipeline (#4210 ) * bloom policy * llama pipeline forward and tests * fix the output and attention_mask * fix name * bind argument to policy * finish bloom model * test shard gpt2 * clear cache	2023-08-15 23:25:14 +08:00
Jianghai	31bcf867ae	[pipeline] Llama causal lm and llama for sequence classification pipeline (#4208 ) * bloom policy * llama pipeline forward and tests * fix the output and attention_mask * fix name * bind argument to policy * Revert "bloom policy" This reverts commit `8dee68a0a2`. This policy should be revert and copied to feature/bloom * revert the bloom changes * cancel unneeded inputs * gpt * finish llama * causal lm and sequence classification * revision	2023-08-15 23:25:14 +08:00
Jianghai	1622031058	[pipeline] Llama pipeline (#4205 ) * bloom policy * llama pipeline forward and tests * fix the output and attention_mask * fix name * bind argument to policy * Revert "bloom policy" This reverts commit `8dee68a0a2`. This policy should be revert and copied to feature/bloom * revert the bloom changes * cancel unneeded inputs * gpt	2023-08-15 23:25:14 +08:00
Frank Lee	ae035d305d	[shardformer] added embedding gradient check (#4124 )	2023-07-04 16:05:01 +08:00
Frank Lee	b1c2901530	[shardformer] supported bloom model (#4098 )	2023-07-04 16:05:01 +08:00
jiangmingyan	ac80937138	[shardformer] shardformer support opt models (#4091 ) * [shardformer] shardformer support opt models * [shardformer] shardformer support opt models, fix * [shardformer] shardformer support opt models, fix * [shardformer] shardformer support opt models, fix	2023-07-04 16:05:01 +08:00
FoolPlayer	7740c55c55	support kit use for bert/gpt test (#4055 ) * support kit use for bert test * support kit test for gpt2	2023-07-04 16:05:01 +08:00
Frank Lee	58df720570	[shardformer] adapted T5 and LLaMa test to use kit (#4049 ) * [shardformer] adapted T5 and LLaMa test to use kit * polish code	2023-07-04 16:05:01 +08:00
digger yu	e61ffc77c6	fix typo tests/ (#3936 )	2023-06-09 09:49:41 +08:00
Hongxin Liu	4b3240cb59	[booster] add low level zero plugin (#3594 ) * [booster] add low level zero plugin * [booster] fix gemini plugin test * [booster] fix precision * [booster] add low level zero plugin test * [test] fix booster plugin test oom * [test] fix booster plugin test oom * [test] fix googlenet and inception output trans * [test] fix diffuser clip vision model * [test] fix torchaudio_wav2vec2_base * [test] fix low level zero plugin test	2023-04-26 14:37:25 +08:00
YuliangLiu0306	f57d34958b	[FX] refactor experimental tracer and adapt it with hf models (#3157 ) * pass gpt trace and meta_prop * pass t5 trace and meta_prop * [FX] refactor experimental tracer and adapt it with hf models * pass all mainstream model zoo * fix CI * fix CI * fix CI * fix CI * fix CI * fix CI * fix CI * fix CI * skip tests * fix CI * using packaging version * polish	2023-03-22 10:40:33 +08:00
Frank Lee	085e7f4eff	[test] fixed torchrec registration in model zoo (#3177 ) * [test] fixed torchrec registration in model zoo * polish code * polish code * polish code	2023-03-20 16:19:06 +08:00
Frank Lee	1ad3a636b1	[test] fixed torchrec model test (#3167 ) * [test] fixed torchrec model test * polish code * polish code * polish code * polish code * polish code * polish code	2023-03-20 11:40:25 +08:00
ver217	6ae8ed0407	[lazyinit] add correctness verification (#3147 ) * [lazyinit] fix shared module * [tests] add lazy init test utils * [tests] add torchvision for lazy init * [lazyinit] fix pre op fn * [lazyinit] handle legacy constructor * [tests] refactor lazy init test models * [tests] refactor lazy init test utils * [lazyinit] fix ops don't support meta * [tests] lazy init test timm models * [lazyinit] fix set data * [lazyinit] handle apex layers * [tests] lazy init test transformers models * [tests] lazy init test torchaudio models * [lazyinit] fix import path * [tests] lazy init test torchrec models * [tests] update torch version in CI * [tests] revert torch version in CI * [tests] skip lazy init test	2023-03-17 13:49:04 +08:00
Frank Lee	ed19290560	[booster] implemented mixed precision class (#3151 ) * [booster] implemented mixed precision class * polish code	2023-03-17 11:00:15 +08:00
YuliangLiu0306	ecd643f1e4	[test] add torchrec models to test model zoo (#3139 )	2023-03-15 05:46:04 +00:00
ver217	14a115000b	[tests] model zoo add torchaudio models (#3138 ) * [tests] model zoo add torchaudio models * [tests] refactor torchaudio wavernn * [tests] refactor fx torchaudio tests	2023-03-15 11:51:16 +08:00
Frank Lee	6d48eb0560	[test] added transformers models to test model zoo (#3135 )	2023-03-15 11:26:10 +08:00
Frank Lee	a674c63348	[test] added torchvision models to test model zoo (#3132 ) * [test] added torchvision models to test model zoo * polish code * polish code * polish code * polish code * polish code * polish code	2023-03-15 10:42:07 +08:00
HELSON	1216d1e7bd	[tests] diffuser models in model zoo (#3136 ) * [tests] diffuser models in model zoo * remove useless code * [tests] add diffusers to requirement-test	2023-03-14 17:20:28 +08:00
Frank Lee	86ac782d7c	[test] added timm models to test model zoo (#3129 ) * [test] added timm models to test model zoo * polish code * polish code * polish code * polish code * polish code	2023-03-14 14:29:18 +08:00

27 Commits (dd2bf026797fb94d5120b481145f37a9661c4a6c)