github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
3,531 Commits
33 Branches
47 Tags
230 MiB
Commit Graph

3531 Commits (fe24789eb178236ad77112824a7d6081ed50dabc)

Author SHA1 Message Date
botbw 13c06d36a3
[bug] fix early return (#5740) 
* [bug] fix silly bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [chore] add test for prefetch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-05-21 14:21:58 +08:00
Yuanheng Zhao c06208e72c
Merge pull request #5737 from yuanheng-zhao/inference/sync/main 
[sync] Sync feature/colossal-infer with main
 2024-05-21 11:26:37 +08:00
Haze188 22ce873c3f
[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) 
* [pre-commit.ci] auto fixes from pre-commit.com hooks

* add parallel cross entropy output for falcon model & fix some typos in bloom.py

* fix module name error, self.model -> self.transformers in bloom, falcon model

* Fix the overflow bug of distributed cross entropy loss function when training with fp16

* add dtype to parallel cross entropy loss function

* fix dtype related typos adn prettify the loss.py

* fix grad dtype and update dtype mismatch error

* fix typo bugs
 2024-05-21 11:07:13 +08:00
Haze188 83716e9feb
Merge pull request #5738 from botbw/prefetch 
[chore] fix init error
 2024-05-21 10:40:56 +08:00
pre-commit-ci[bot] b3c0e6d871 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-21 02:09:15 +00:00
hxwang 137a7c341b [chore] fix init error 2024-05-21 02:07:21 +00:00
Yuanheng Zhao 8633c15da9 [sync] Sync feature/colossal-infer with main 2024-05-20 15:50:53 +00:00
Yuanheng Zhao d8b1ea4ac9
[doc] Update Inference Readme (#5736) 
* [doc] update inference readme

* add contents

* trivial
 2024-05-20 22:50:04 +08:00
Yuanheng Zhao bdf9a001d6
[Fix/Inference] Add unsupported auto-policy error message (#5730) 
* [fix] auto policy error message

* trivial
 2024-05-20 22:49:18 +08:00
botbw f5b7de38a4
Merge pull request #5733 from Hz188/feature/prefetch 
[Gemini] implement auto policy prefetch and a little origin code modification.
 2024-05-20 15:31:34 +08:00
genghaozhe 90d8d0183c remove personal comments 2024-05-20 07:28:20 +00:00
genghaozhe bfcb2d1ff8 refactor the code structure to solve the circular import 2024-05-20 07:25:24 +00:00
genghaozhe a280517dd9 remove unrelated file 2024-05-20 05:25:35 +00:00
genghaozhe 3b363d44cc Merge branch 'feature/prefetch' of https://github.com/Hz188/ColossalAI into feature/prefetch 2024-05-20 05:23:40 +00:00
genghaozhe 1ec92d29af remove perf log, unrelated file and so on 2024-05-20 05:23:26 +00:00
genghaozhe 5c6c5d6be3 remove comments 2024-05-20 05:23:12 +00:00
genghaozhe df63db7e63 remote comments 2024-05-20 05:15:51 +00:00
genghaozhe 7416e4943b fix conflicts to beautify the code 2024-05-20 04:09:51 +00:00
botbw f5a5287f87
Merge pull request #5731 from botbw/prefetch 
[gemini] prefetch for auto policy
 2024-05-20 12:04:33 +08:00
genghaozhe d22bf30ca6 implement auto policy prefetch and modify a little origin code. 2024-05-20 04:01:53 +00:00
pre-commit-ci[bot] f1918e18a5 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-20 03:00:07 +00:00
hxwang a55a9e298b [gemini] init auto policy prefetch 2024-05-20 02:21:17 +00:00
Yuanheng Zhao 283c407a19
[Inference] Fix Inference Generation Config and Sampling (#5710) 
* refactor and add

* config default values

* fix gen config passing

* fix rpc generation config
 2024-05-19 15:08:42 +08:00
Haze188 c5ddf17c76
Merge branch 'hpcaitech:feature/prefetch' into feature/prefetch 2024-05-17 18:58:53 +08:00
genghaozhe 06a3a100b3 remove unrelated code 2024-05-17 10:57:49 +00:00
genghaozhe 3d625ca836 add some todo Message 2024-05-17 10:55:28 +00:00
flybird11111 9d83c6d715
[lazy] fix lazy cls init (#5720) 
* fix

* fix

* fix

* fix

* fix

* remove kernel intall

* rebase

revert

fix

* fix

* fix
 2024-05-17 18:18:59 +08:00
botbw 9690981601
Merge pull request #5722 from botbw/prefetch 
[gemini] prefetch chunks
 2024-05-17 13:46:18 +08:00
botbw e57812c672
[chore] Update placement_policy.py 2024-05-17 13:42:18 +08:00
Yuanheng Zhao 8bcfe360fd
[example] Update Inference Example (#5725) 
* [example] update inference example
 2024-05-17 11:28:53 +08:00
genghaozhe 013690a86b remove set(all_chunks) 2024-05-16 09:57:51 +00:00
hxwang 6efbadba25 [chore] remove debugging info 2024-05-16 16:46:39 +08:00
hxwang 20701d4533 [chore] remove print 2024-05-16 16:45:50 +08:00
hxwang f45f8a2aa7 [gemini] maxprefetch means maximum work to keep 2024-05-16 16:12:53 +08:00
genghaozhe fc2248cf99 Merge branch 'prefetch' of github.com:botbw/ColossalAI into feature/prefetch 2024-05-16 08:05:32 +00:00
genghaozhe 5470e5f94e a commit for fake push test 2024-05-16 08:03:40 +00:00
pre-commit-ci[bot] 6bbe956316 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-16 07:26:20 +00:00
hxwang 82b25524ff Merge branch 'prefetch' of github.com:botbw/ColossalAI into prefetch 2024-05-16 07:25:22 +00:00
genghaozhe 1f6b57099c Merge branch 'prefetch' of github.com:botbw/ColossalAI into botbw-prefetch 2024-05-16 07:23:40 +00:00
hxwang 2e68eebdfe [chore] refactor & sync 2024-05-16 07:22:10 +00:00
binmakeswell 2011b1356a
[misc] Update PyTorch version in docs (#5724) 
* [misc] Update PyTorch version in docs

* [misc] Update PyTorch version in docs
 2024-05-16 13:54:32 +08:00
pre-commit-ci[bot] 5bedea6e10 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-16 05:20:01 +00:00
hxwang 4148ceed9f [gemini] use compute_chunk to find next chunk 2024-05-16 13:17:26 +08:00
hxwang b2e9745888 [chore] sync 2024-05-16 04:45:06 +00:00
傅剑寒 a8d459f99a
【Inference] Delete duplicated package (#5723) 2024-05-16 10:49:03 +08:00
hxwang 6e38eafebe [gemini] prefetch chunks 2024-05-15 16:51:44 +08:00
Jianghai f47f2fbb24
[Inference] Fix API server, test and example (#5712) 
* fix api server

* fix generation config

* fix api server

* fix comments

* fix infer hanging bug

* resolve comments, change backend to free port
 2024-05-15 15:47:31 +08:00
Tong Li 913c920ecc
[Colossal-LLaMA] Fix sft issue for llama2 (#5719) 
* fix minor issue

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-05-15 10:52:11 +08:00
Runyu Lu 74c47921fa
[Fix] Llama3 Load/Omit CheckpointIO Temporarily (#5717) 
* Fix Llama3 Load error
* Omit Checkpoint IO Temporarily
 2024-05-14 20:17:43 +08:00
Yuanheng Zhao 5bbab1533a
[ci] Fix example tests (#5714) 
* [fix] revise timeout value on example CI

* trivial
 2024-05-14 16:08:51 +08:00