1982 Commits (10a19e22c63aa9963a889874b63c47ccd0e6db42)

Author	SHA1	Message	Date
Edenzzzz	79f7a7b211	[misc] Accelerate CI for zero and dist optim (#5758) ... * remove fp16 from lamb * remove d2h copy in checking states --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
flybird11111	50b4c8e8cf	[hotfix] fix llama flash attention forward (#5777)	6 months ago
yuehuayingxueluo	b45000f839	[Inference]Add Streaming LLM (#5745) ... * Add Streaming LLM * add some parameters to llama_generation.py * verify streamingllm config * add test_streamingllm.py * modified according to the opinions of review * add Citation * change _block_tables tolist	6 months ago
Yuanheng Zhao	406443200f	[Hotfix] Add missing init file in inference.executor (#5774)	6 months ago
duanjunwen	1b76564e16	[test] Fix/fix testcase (#5770) ... * [fix] branch for fix testcase; * [fix] fix test_analyzer & test_auto_parallel; * [fix] remove local change about moe; * [fix] rm local change moe;	6 months ago
flybird11111	3f2be80530	fix (#5765)	6 months ago
botbw	023ea13cb5	Merge pull request #5749 from hpcaitech/prefetch ... [Gemini] Prefetch next chunk before each op	6 months ago
hxwang	8547562884	[chore] remove unnecessary assert since compute list might not be recorded	6 months ago
hxwang	e5e3320948	[bug] continue fix	6 months ago
hxwang	936dd96dbb	[bug] workaround for idx fix	6 months ago
Edenzzzz	5f8c0a0ac3	[Feature] auto-cast optimizers to distributed version (#5746) ... * auto-cast optimizers to distributed * fix galore casting * logger --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
hxwang	ff507b755e	Merge branch 'main' of github.com:hpcaitech/ColossalAI into prefetch	6 months ago
botbw	2fc85abf43	[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) ... * [gemini] async grad chunk reduce (all-reduce&reduce-scatter) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] add test * [gemini] rename func * [gemini] update llama benchmark * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] use tensor counter * [gemini] change default config in GeminiPlugin and GeminiDDP * [chore] typo * [gemini] fix sync issue & add test cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742) ... * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
hxwang	15d21a077a	Merge remote-tracking branch 'origin/main' into prefetch	6 months ago
binmakeswell	4647ec28c8	[inference] release (#5747) ... * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release	6 months ago
Yuanheng Zhao	df6747603f	[Colossal-Inference] (v0.1.0) Merge pull request #5739 from hpcaitech/feature/colossal-infer ... [Inference] Merge feature/colossal-infer	6 months ago
Yuanheng Zhao	bd38fe6b91	[NFC] Fix code factors on inference triton kernels (#5743)	6 months ago
botbw	13c06d36a3	[bug] fix early return (#5740) ... * [bug] fix silly bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [chore] add test for prefetch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Haze188	22ce873c3f	[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) ... * [pre-commit.ci] auto fixes from pre-commit.com hooks * add parallel cross entropy output for falcon model & fix some typos in bloom.py * fix module name error, self.model -> self.transformers in bloom, falcon model * Fix the overflow bug of distributed cross entropy loss function when training with fp16 * add dtype to parallel cross entropy loss function * fix dtype related typos adn prettify the loss.py * fix grad dtype and update dtype mismatch error * fix typo bugs	6 months ago
pre-commit-ci[bot]	b3c0e6d871	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	137a7c341b	[chore] fix init error	6 months ago
Yuanheng Zhao	8633c15da9	[sync] Sync feature/colossal-infer with main	6 months ago
Yuanheng Zhao	d8b1ea4ac9	[doc] Update Inference Readme (#5736) ... * [doc] update inference readme * add contents * trivial	6 months ago
Yuanheng Zhao	bdf9a001d6	[Fix/Inference] Add unsupported auto-policy error message (#5730) ... * [fix] auto policy error message * trivial	6 months ago
genghaozhe	90d8d0183c	remove personal comments	6 months ago
genghaozhe	bfcb2d1ff8	refactor the code structure to solve the circular import	6 months ago
genghaozhe	1ec92d29af	remove perf log, unrelated file and so on	6 months ago
genghaozhe	5c6c5d6be3	remove comments	6 months ago
genghaozhe	7416e4943b	fix conflicts to beautify the code	6 months ago
genghaozhe	d22bf30ca6	implement auto policy prefetch and modify a little origin code.	6 months ago
pre-commit-ci[bot]	f1918e18a5	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	a55a9e298b	[gemini] init auto policy prefetch	6 months ago
Yuanheng Zhao	283c407a19	[Inference] Fix Inference Generation Config and Sampling (#5710) ... * refactor and add * config default values * fix gen config passing * fix rpc generation config	6 months ago
genghaozhe	06a3a100b3	remove unrelated code	6 months ago
genghaozhe	3d625ca836	add some todo Message	6 months ago
flybird11111	9d83c6d715	[lazy] fix lazy cls init (#5720) ... * fix * fix * fix * fix * fix * remove kernel intall * rebase revert fix * fix * fix	6 months ago
botbw	e57812c672	[chore] Update placement_policy.py	6 months ago
Yuanheng Zhao	8bcfe360fd	[example] Update Inference Example (#5725) ... * [example] update inference example	6 months ago
genghaozhe	013690a86b	remove set(all_chunks)	6 months ago
hxwang	6efbadba25	[chore] remove debugging info	6 months ago
hxwang	20701d4533	[chore] remove print	6 months ago
hxwang	f45f8a2aa7	[gemini] maxprefetch means maximum work to keep	6 months ago
genghaozhe	fc2248cf99	Merge branch 'prefetch' of github.com:botbw/ColossalAI into feature/prefetch	6 months ago
pre-commit-ci[bot]	6bbe956316	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	82b25524ff	Merge branch 'prefetch' of github.com:botbw/ColossalAI into prefetch	6 months ago
genghaozhe	1f6b57099c	Merge branch 'prefetch' of github.com:botbw/ColossalAI into botbw-prefetch	6 months ago
hxwang	2e68eebdfe	[chore] refactor & sync	6 months ago
pre-commit-ci[bot]	5bedea6e10	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	4148ceed9f	[gemini] use compute_chunk to find next chunk	6 months ago