Commit Graph

3342 Commits (ca674549e0ea815f949d19654b110d7f065e3496)
 

Author SHA1 Message Date
傅剑寒 d4cb023b62
[Inference/Refactor] Delete Duplicated code and refactor vec_copy utils and reduce utils (#5593)
8 months ago
傅剑寒 a21912339a
refactor csrc (#5582)
8 months ago
Yuanheng Zhao 25928d8496
[Inference/Spec-Dec] Merge pull request #5565 from hpcaitech/feat/speculative-decoding
8 months ago
Yuanheng f8598e3ec5 [Fix] Llama Modeling Control with Spec-Dec (#5580)
8 months ago
Yuanheng Zhao e60d430cf5 [Fix] resolve conflicts of rebasing feat/speculative-decoding (#5557)
8 months ago
Yuanheng Zhao e1acb58423 [doc] Add inference/speculative-decoding README (#5552)
8 months ago
Yuanheng Zhao d85d91435a [Inference/SpecDec] Support GLIDE Drafter Model (#5455)
8 months ago
Yuanheng Zhao 912e24b2aa [SpecDec] Fix inputs for speculation and revise past KV trimming (#5449)
8 months ago
Yuanheng Zhao a37f82629d [Inference/SpecDec] Add Speculative Decoding Implementation (#5423)
8 months ago
Yuanheng Zhao 5a9b05f7b2 [Inference/SpecDec] Add Basic Drafter Model Container (#5405)
8 months ago
Yuanheng Zhao d63c469f45 [Infer] Revise and Adapt Triton Kernels for Spec-Dec (#5401)
8 months ago
Yuanheng Zhao d56c96334e
Sync main to feature/colossal-infer
8 months ago
Yuanheng 7ca1d1c545 remove outdated triton test
8 months ago
pre-commit-ci[bot] d78817539e [pre-commit.ci] auto fixes from pre-commit.com hooks
8 months ago
Yuanheng ce9401ad52 remove unused triton kernels
8 months ago
Yuanheng ed5ebd1735 [Fix] resolve conflicts of merging main
8 months ago
Hongxin Liu 641b1ee71a
[devops] remove post commit ci (#5566)
8 months ago
傅剑寒 7ebdf48ac5
add cast and op_functor for cuda build-in types (#5546)
8 months ago
digger yu 341263df48
[hotfix] fix typo s/get_defualt_parser /get_default_parser (#5548)
8 months ago
digger yu a799ca343b
[fix] fix typo s/muiti-node /multi-node etc. (#5448)
8 months ago
Edenzzzz 15055f9a36
[hotfix] quick fixes to make legacy tutorials runnable (#5559)
8 months ago
Zhongkai Zhao 8e412a548e
[shardformer] Sequence Parallelism Optimization (#5533)
8 months ago
Edenzzzz 7e0ec5a85c
fix incorrect sharding without zero (#5545)
8 months ago
Yuanheng Zhao 4bb5d8923a
[Fix/Inference] Remove unused and non-functional functions (#5543)
8 months ago
傅剑寒 a2878e39f4
[Inference] Add Reduce Utils (#5537)
8 months ago
yuehuayingxueluo 04aca9e55b
[Inference/Kernel]Add get_cos_and_sin Kernel (#5528)
8 months ago
Wenhao Chen e614aa34f3
[shardformer, pipeline] add `gradient_checkpointing_ratio` and heterogenous shard policy for llama (#5508)
8 months ago
YeAnbang df5e9c53cf
[ColossalChat] Update RLHF V2 (#5286)
8 months ago
Yuanheng Zhao 36c4bb2893
[Fix] Grok-1 use tokenizer from the same pretrained path (#5532)
8 months ago
yuehuayingxueluo 934e31afb2
The writing style of tail processing and the logic related to macro definitions have been optimized. (#5519)
8 months ago
Insu Jang 00525f7772
[shardformer] fix pipeline forward error if custom layer distribution is used (#5189)
8 months ago
github-actions[bot] e6707a6e8d
[format] applied code formatting on changed files in pull request 5510 (#5517)
8 months ago
Hongxin Liu 19e1a5cf16
[shardformer] update colo attention to support custom mask (#5510)
8 months ago
Edenzzzz 9a3321e9f4
Merge pull request #5515 from Edenzzzz/fix_layout_convert
8 months ago
Edenzzzz 18edcd5368 Empty-Commit
8 months ago
Edenzzzz 61da3fbc52 fixed layout converter caching and updated tester
8 months ago
傅剑寒 e6496dd371
[Inference] Optimize request handler of llama (#5512)
8 months ago
Rocky Duan cbe34c557c
Fix ColoTensorSpec for py11 (#5440)
8 months ago
Hongxin Liu a7790a92e8
[devops] fix example test ci (#5504)
8 months ago
Yuanheng Zhao 131f32a076
[fix] fix grok-1 example typo (#5506)
8 months ago
flybird11111 0688d92e2d
[shardformer]Fix lm parallel. (#5480)
8 months ago
Runyu Lu 6251d68dc9
[fix] PR #5354 (#5501)
8 months ago
Runyu Lu 1d626233ce
Merge pull request #5434 from LRY89757/colossal-infer-cuda-graph
8 months ago
Runyu Lu 68e9396bc0 [fix] merge conflicts
8 months ago
binmakeswell 34e909256c
[release] grok-1 inference benchmark (#5500)
8 months ago
yuehuayingxueluo 87079cffe8
[Inference]Support FP16/BF16 Flash Attention 2 And Add high_precision Flag To Rotary Embedding (#5461)
8 months ago
Wenhao Chen bb0a668fee
[hotfix] set return_outputs=False in examples and polish code (#5404)
8 months ago
Runyu Lu ff4998c6f3 [fix] remove unused comment
8 months ago
Runyu Lu 9fe61b4475 [fix]
8 months ago
Yuanheng Zhao 5fcd7795cd
[example] update Grok-1 inference (#5495)
8 months ago