github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
3,495 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

3495 Commits (544b7a38a167cb05cdc7590cfc100e23c0ed5ab7)

Author SHA1 Message Date
genghaozhe fba04e857b [bugs] fix args.profile=False DummyProfiler errro 2024-05-25 14:55:09 +00:00
Yuanheng Zhao b96c6390f4
[inference] Fix running time of test_continuous_batching (#5750) 2024-05-24 19:34:15 +08:00
Edenzzzz 5f8c0a0ac3
[Feature] auto-cast optimizers to distributed version (#5746) 
* auto-cast optimizers to distributed

* fix galore casting

* logger

---------

Co-authored-by: Edenzzzz <wtan45@wisc.edu>
 2024-05-24 17:24:16 +08:00
hxwang ca674549e0 [chore] remove unnecessary test & changes 2024-05-24 06:09:36 +00:00
hxwang ff507b755e Merge branch 'main' of github.com:hpcaitech/ColossalAI into prefetch 2024-05-24 04:05:07 +00:00
hxwang 63c057cd8e [example] add profile util for llama 2024-05-24 03:59:36 +00:00
botbw 2fc85abf43
[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) 
* [gemini] async grad chunk reduce (all-reduce&reduce-scatter)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [gemini] add test

* [gemini] rename func

* [gemini] update llama benchmark

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [gemini] use tensor counter

* [gemini] change default config in GeminiPlugin and GeminiDDP

* [chore] typo

* [gemini] fix sync issue & add test cases

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-05-24 10:31:16 +08:00
Jianghai 85946d4236
[Inference]Fix readme and example for API server (#5742) 
* fix chatapi readme and example

* updating doc

* add an api and change the doc

* remove

* add credits and del 'API' heading

* readme

* readme
 2024-05-24 10:03:05 +08:00
hxwang 15d21a077a Merge remote-tracking branch 'origin/main' into prefetch 2024-05-23 15:49:33 +00:00
binmakeswell 4647ec28c8
[inference] release (#5747) 
* [inference] release

* [inference] release

* [inference] release

* [inference] release

* [inference] release

* [inference] release

* [inference] release
 2024-05-23 17:44:06 +08:00
Yuanheng Zhao df6747603f
[Colossal-Inference] (v0.1.0) Merge pull request #5739 from hpcaitech/feature/colossal-infer 
[Inference] Merge feature/colossal-infer
 2024-05-22 14:31:09 +08:00
Yuanheng Zhao 498f42c45b
[NFC] fix requirements (#5744) 2024-05-22 12:08:49 +08:00
Yuanheng Zhao bd38fe6b91
[NFC] Fix code factors on inference triton kernels (#5743) 2024-05-21 22:12:15 +08:00
Yuanheng Zhao c2c8c9cf17
[ci] Temporary fix for build on pr (#5741) 
* temporary fix for CI

* timeout to 90
 2024-05-21 18:20:57 +08:00
botbw 13c06d36a3
[bug] fix early return (#5740) 
* [bug] fix silly bug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [chore] add test for prefetch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-05-21 14:21:58 +08:00
Yuanheng Zhao c06208e72c
Merge pull request #5737 from yuanheng-zhao/inference/sync/main 
[sync] Sync feature/colossal-infer with main
 2024-05-21 11:26:37 +08:00
Haze188 22ce873c3f
[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) 
* [pre-commit.ci] auto fixes from pre-commit.com hooks

* add parallel cross entropy output for falcon model & fix some typos in bloom.py

* fix module name error, self.model -> self.transformers in bloom, falcon model

* Fix the overflow bug of distributed cross entropy loss function when training with fp16

* add dtype to parallel cross entropy loss function

* fix dtype related typos adn prettify the loss.py

* fix grad dtype and update dtype mismatch error

* fix typo bugs
 2024-05-21 11:07:13 +08:00
Haze188 83716e9feb
Merge pull request #5738 from botbw/prefetch 
[chore] fix init error
 2024-05-21 10:40:56 +08:00
pre-commit-ci[bot] b3c0e6d871 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-21 02:09:15 +00:00
hxwang 137a7c341b [chore] fix init error 2024-05-21 02:07:21 +00:00
Yuanheng Zhao 8633c15da9 [sync] Sync feature/colossal-infer with main 2024-05-20 15:50:53 +00:00
Yuanheng Zhao d8b1ea4ac9
[doc] Update Inference Readme (#5736) 
* [doc] update inference readme

* add contents

* trivial
 2024-05-20 22:50:04 +08:00
Yuanheng Zhao bdf9a001d6
[Fix/Inference] Add unsupported auto-policy error message (#5730) 
* [fix] auto policy error message

* trivial
 2024-05-20 22:49:18 +08:00
botbw f5b7de38a4
Merge pull request #5733 from Hz188/feature/prefetch 
[Gemini] implement auto policy prefetch and a little origin code modification.
 2024-05-20 15:31:34 +08:00
genghaozhe 90d8d0183c remove personal comments 2024-05-20 07:28:20 +00:00
genghaozhe bfcb2d1ff8 refactor the code structure to solve the circular import 2024-05-20 07:25:24 +00:00
genghaozhe a280517dd9 remove unrelated file 2024-05-20 05:25:35 +00:00
genghaozhe 3b363d44cc Merge branch 'feature/prefetch' of https://github.com/Hz188/ColossalAI into feature/prefetch 2024-05-20 05:23:40 +00:00
genghaozhe 1ec92d29af remove perf log, unrelated file and so on 2024-05-20 05:23:26 +00:00
genghaozhe 5c6c5d6be3 remove comments 2024-05-20 05:23:12 +00:00
genghaozhe df63db7e63 remote comments 2024-05-20 05:15:51 +00:00
genghaozhe 7416e4943b fix conflicts to beautify the code 2024-05-20 04:09:51 +00:00
botbw f5a5287f87
Merge pull request #5731 from botbw/prefetch 
[gemini] prefetch for auto policy
 2024-05-20 12:04:33 +08:00
genghaozhe d22bf30ca6 implement auto policy prefetch and modify a little origin code. 2024-05-20 04:01:53 +00:00
pre-commit-ci[bot] f1918e18a5 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2024-05-20 03:00:07 +00:00
hxwang a55a9e298b [gemini] init auto policy prefetch 2024-05-20 02:21:17 +00:00
Yuanheng Zhao 283c407a19
[Inference] Fix Inference Generation Config and Sampling (#5710) 
* refactor and add

* config default values

* fix gen config passing

* fix rpc generation config
 2024-05-19 15:08:42 +08:00
Haze188 c5ddf17c76
Merge branch 'hpcaitech:feature/prefetch' into feature/prefetch 2024-05-17 18:58:53 +08:00
genghaozhe 06a3a100b3 remove unrelated code 2024-05-17 10:57:49 +00:00
genghaozhe 3d625ca836 add some todo Message 2024-05-17 10:55:28 +00:00
flybird11111 9d83c6d715
[lazy] fix lazy cls init (#5720) 
* fix

* fix

* fix

* fix

* fix

* remove kernel intall

* rebase

revert

fix

* fix

* fix
 2024-05-17 18:18:59 +08:00
botbw 9690981601
Merge pull request #5722 from botbw/prefetch 
[gemini] prefetch chunks
 2024-05-17 13:46:18 +08:00
botbw e57812c672
[chore] Update placement_policy.py 2024-05-17 13:42:18 +08:00
Yuanheng Zhao 8bcfe360fd
[example] Update Inference Example (#5725) 
* [example] update inference example
 2024-05-17 11:28:53 +08:00
genghaozhe 013690a86b remove set(all_chunks) 2024-05-16 09:57:51 +00:00
hxwang 6efbadba25 [chore] remove debugging info 2024-05-16 16:46:39 +08:00
hxwang 20701d4533 [chore] remove print 2024-05-16 16:45:50 +08:00
hxwang f45f8a2aa7 [gemini] maxprefetch means maximum work to keep 2024-05-16 16:12:53 +08:00
genghaozhe fc2248cf99 Merge branch 'prefetch' of github.com:botbw/ColossalAI into feature/prefetch 2024-05-16 08:05:32 +00:00
genghaozhe 5470e5f94e a commit for fake push test 2024-05-16 08:03:40 +00:00