Commit Graph

3764 Commits (cf519dac6a5799b8f314aac6f510e2a98d3af9c6)
 

Author SHA1 Message Date
Hongxin Liu adae123df3
[release] update version (#5380)
10 months ago
Frank Lee efef43b53c
Merge pull request #5372 from hpcaitech/exp/mixtral
10 months ago
yuehuayingxueluo 8c69debdc7
[Inference]Support vllm testing in benchmark scripts (#5379)
10 months ago
Frank Lee 4c03347fc7
Merge pull request #5377 from hpcaitech/example/llama-npu
10 months ago
Frank Lee 9afa52061f
[inference] refactored config (#5376)
10 months ago
ver217 06db94fbc9 [moe] fix tests
10 months ago
Hongxin Liu 65e5d6baa5 [moe] fix mixtral optim checkpoint (#5344)
10 months ago
Hongxin Liu 956b561b54 [moe] fix mixtral forward default value (#5329)
10 months ago
Hongxin Liu b60be18dcc [moe] fix mixtral checkpoint io (#5314)
10 months ago
Hongxin Liu da39d21b71 [moe] support mixtral (#5309)
10 months ago
Hongxin Liu c904d2ae99 [moe] update capacity computing (#5253)
10 months ago
Xuanlei Zhao 7d8e0338a4 [moe] init mixtral impl
10 months ago
Jianghai 1f8c7e7046
[Inference] User Experience: update the logic of default tokenizer and generation config. (#5337)
10 months ago
yuehuayingxueluo 6fb4bcbb24
[Inference/opt] Fused KVCahce Memcopy (#5374)
10 months ago
Frank Lee 58740b5f68
[inference] added inference template (#5375)
10 months ago
Frank Lee 8106ede07f
Revert "[Inference] Adapt to Fused rotary (#5348)" (#5373)
10 months ago
Jianghai 9f4ab2eb92
[Inference] Adapt to Fused rotary (#5348)
10 months ago
yuehuayingxueluo 35382a7fbf
[Inference]Fused the gate and up proj in mlp,and optimized the autograd process. (#5365)
10 months ago
Hongxin Liu 084c91246c
[llama] fix memory issue (#5371)
10 months ago
Yuanheng Zhao 1dedb57747
[Fix/Infer] Remove unused deps and revise requirements (#5341)
10 months ago
Hongxin Liu c53ddda88f
[lr-scheduler] fix load state dict and add test (#5369)
10 months ago
Hongxin Liu eb4f2d90f9
[llama] polish training script and fix optim ckpt (#5368)
10 months ago
Camille Zhong a5756a8720
[eval] update llama npu eval (#5366)
10 months ago
Camille Zhong 44ca61a22b
[llama] fix neftune & pbar with start_step (#5364)
10 months ago
Hongxin Liu a4cec1715b
[llama] add flash attn patch for npu (#5362)
10 months ago
Hongxin Liu 73f9f23fc6
[llama] update training script (#5360)
10 months ago
Hongxin Liu 6c0fa7b9a8
[llama] fix dataloader for hybrid parallel (#5358)
10 months ago
Hongxin Liu 2dd01e3a14
[gemini] fix param op hook when output is tuple (#5355)
10 months ago
yuehuayingxueluo 631862f339
[Inference]Optimize generation process of inference engine (#5356)
10 months ago
yuehuayingxueluo 21ad4a27f9
[Inference/opt]Optimize the mid tensor of RMS Norm (#5350)
10 months ago
Wenhao Chen 1c790c0877
[fix] remove unnecessary dp_size assert (#5351)
10 months ago
Frank Lee 027aa1043f
[doc] updated inference readme (#5343)
10 months ago
Frank Lee e76acbb076
[inference] moved ops tests to test_infer (#5354)
10 months ago
Frank Lee db1a763307
[inference] removed redundancy init_batch (#5353)
10 months ago
Hongxin Liu ffffc32dc7
[checkpointio] fix gemini and hybrid parallel optim checkpoint (#5347)
10 months ago
yuehuayingxueluo 249644c23b
[Inference]Repalce Attention layer and MLP layer by shardformer to optimize the weight transpose operation,add fused_qkv and fused linear_add (#5340)
10 months ago
Frank Lee f8e456d202
[inference] simplified config verification (#5346)
10 months ago
YeAnbang c5239840e6
[Chat] fix sft loss nan (#5345)
10 months ago
Frank Lee abd8e77ad8
[extension] fixed exception catch (#5342)
10 months ago
Jianghai df0aa49585
[Inference] Kernel Fusion, fused copy kv cache into rotary embedding (#5336)
10 months ago
Frank Lee 1336838a91
Merge pull request #5339 from FrankLeeeee/sync/merge-main
10 months ago
FrankLeeeee c565519913 merge commit
10 months ago
Yuanheng Zhao 5f98a9d68a
[Infer] Optimize Blocked KVCache And Kernels Using It (#5325)
10 months ago
yuehuayingxueluo e8f0642f28
[Inference]Add Nopadding Llama Modeling (#5327)
10 months ago
digger yu 71321a07cf
fix typo change dosen't to doesn't (#5308)
10 months ago
digger yu 6a3086a505
fix typo under extensions/ (#5330)
10 months ago
Frank Lee febed23288
[doc] added docs for extensions (#5324)
10 months ago
flybird11111 388179f966
[tests] fix t5 test. (#5322)
10 months ago
Jianghai c7c104cb7c
[DOC] Update inference readme (#5280)
10 months ago
Frank Lee a6709afe66
Merge pull request #5321 from FrankLeeeee/hotfix/accelerator-api
10 months ago