ColossalAI

Commit Graph

Author	SHA1	Message	Date
Edenzzzz	79f7a7b211	[misc] Accelerate CI for zero and dist optim (#5758 ) * remove fp16 from lamb * remove d2h copy in checking states --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
flybird11111	50b4c8e8cf	[hotfix] fix llama flash attention forward (#5777 )	6 months ago
yuehuayingxueluo	b45000f839	[Inference]Add Streaming LLM (#5745 ) * Add Streaming LLM * add some parameters to llama_generation.py * verify streamingllm config * add test_streamingllm.py * modified according to the opinions of review * add Citation * change _block_tables tolist	6 months ago
Hongxin Liu	ee6fd38373	[devops] fix docker ci (#5780 )	6 months ago
Hongxin Liu	32f4187806	[misc] update dockerfile (#5776 ) * [misc] update dockerfile * [misc] update dockerfile	6 months ago
Haze188	e22b82755d	[CI/tests] simplify some test case to reduce testing time (#5755 ) * [ci/tests] simplify some test case to reduce testing time * [ci/tests] continue to remove test case to reduce ci time cost * restore some test config * [ci/tests] continue to reduce ci time cost	6 months ago
Yuanheng Zhao	406443200f	[Hotfix] Add missing init file in inference.executor (#5774 )	6 months ago
duanjunwen	1b76564e16	[test] Fix/fix testcase (#5770 ) * [fix] branch for fix testcase; * [fix] fix test_analyzer & test_auto_parallel; * [fix] remove local change about moe; * [fix] rm local change moe;	6 months ago
flybird11111	3f2be80530	fix (#5765 )	6 months ago
Hongxin Liu	68359ed1e1	[release] update version (#5752 ) * [release] update version * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [test] fix ddp plugin test * [test] fix gptj and rpc test * [devops] fix cuda ext compatibility * [inference] fix flash decoding test * [inference] fix flash decoding test	6 months ago
Yuanheng Zhao	677cbfacf8	[Fix/Example] Fix Llama Inference Loading Data Type (#5763 ) * [fix/example] fix llama inference loading dtype * revise loading dtype of benchmark llama3	6 months ago
botbw	023ea13cb5	Merge pull request #5749 from hpcaitech/prefetch [Gemini] Prefetch next chunk before each op	6 months ago
hxwang	154720ba6e	[chore] refactor profiler utils	6 months ago
hxwang	8547562884	[chore] remove unnecessary assert since compute list might not be recorded	6 months ago
hxwang	e5e3320948	[bug] continue fix	6 months ago
hxwang	936dd96dbb	[bug] workaround for idx fix	6 months ago
botbw	e0dde8fda5	Merge pull request #5754 from Hz188/prefetch [Gemini]Prefetch benchmark	6 months ago
botbw	157b4cc357	Merge branch 'prefetch' into prefetch	6 months ago
genghaozhe	87665d7922	correct argument help message	6 months ago
Haze188	4d097def96	[Gemini] add some code for reduce-scatter overlap, chunk prefetch in llama benchmark. (#5751 ) * [bugs] fix args.profile=False DummyProfiler errro * add args.prefetch_num for benchmark	6 months ago
genghaozhe	b9269d962d	add args.prefetch_num for benchmark	6 months ago
genghaozhe	fba04e857b	[bugs] fix args.profile=False DummyProfiler errro	6 months ago
Yuanheng Zhao	b96c6390f4	[inference] Fix running time of test_continuous_batching (#5750 )	6 months ago
Edenzzzz	5f8c0a0ac3	[Feature] auto-cast optimizers to distributed version (#5746 ) * auto-cast optimizers to distributed * fix galore casting * logger --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
hxwang	ca674549e0	[chore] remove unnecessary test & changes	6 months ago
hxwang	ff507b755e	Merge branch 'main' of github.com:hpcaitech/ColossalAI into prefetch	6 months ago
hxwang	63c057cd8e	[example] add profile util for llama	6 months ago
botbw	2fc85abf43	[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713 ) * [gemini] async grad chunk reduce (all-reduce&reduce-scatter) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] add test * [gemini] rename func * [gemini] update llama benchmark * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] use tensor counter * [gemini] change default config in GeminiPlugin and GeminiDDP * [chore] typo * [gemini] fix sync issue & add test cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742 ) * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
hxwang	15d21a077a	Merge remote-tracking branch 'origin/main' into prefetch	6 months ago
binmakeswell	4647ec28c8	[inference] release (#5747 ) * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release	6 months ago
Yuanheng Zhao	df6747603f	[Colossal-Inference] (v0.1.0) Merge pull request #5739 from hpcaitech/feature/colossal-infer [Inference] Merge feature/colossal-infer	6 months ago
Yuanheng Zhao	498f42c45b	[NFC] fix requirements (#5744 )	6 months ago
Yuanheng Zhao	bd38fe6b91	[NFC] Fix code factors on inference triton kernels (#5743 )	6 months ago
Yuanheng Zhao	c2c8c9cf17	[ci] Temporary fix for build on pr (#5741 ) * temporary fix for CI * timeout to 90	6 months ago
botbw	13c06d36a3	[bug] fix early return (#5740 ) * [bug] fix silly bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [chore] add test for prefetch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Yuanheng Zhao	c06208e72c	Merge pull request #5737 from yuanheng-zhao/inference/sync/main [sync] Sync feature/colossal-infer with main	6 months ago
Haze188	22ce873c3f	[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702 ) * [pre-commit.ci] auto fixes from pre-commit.com hooks * add parallel cross entropy output for falcon model & fix some typos in bloom.py * fix module name error, self.model -> self.transformers in bloom, falcon model * Fix the overflow bug of distributed cross entropy loss function when training with fp16 * add dtype to parallel cross entropy loss function * fix dtype related typos adn prettify the loss.py * fix grad dtype and update dtype mismatch error * fix typo bugs	6 months ago
Haze188	83716e9feb	Merge pull request #5738 from botbw/prefetch [chore] fix init error	6 months ago
pre-commit-ci[bot]	b3c0e6d871	[pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci	6 months ago
hxwang	137a7c341b	[chore] fix init error	6 months ago
Yuanheng Zhao	8633c15da9	[sync] Sync feature/colossal-infer with main	6 months ago
Yuanheng Zhao	d8b1ea4ac9	[doc] Update Inference Readme (#5736 ) * [doc] update inference readme * add contents * trivial	6 months ago
Yuanheng Zhao	bdf9a001d6	[Fix/Inference] Add unsupported auto-policy error message (#5730 ) * [fix] auto policy error message * trivial	6 months ago
botbw	f5b7de38a4	Merge pull request #5733 from Hz188/feature/prefetch [Gemini] implement auto policy prefetch and a little origin code modification.	6 months ago
genghaozhe	90d8d0183c	remove personal comments	6 months ago
genghaozhe	bfcb2d1ff8	refactor the code structure to solve the circular import	6 months ago
genghaozhe	a280517dd9	remove unrelated file	6 months ago
genghaozhe	3b363d44cc	Merge branch 'feature/prefetch' of https://github.com/Hz188/ColossalAI into feature/prefetch	6 months ago
genghaozhe	1ec92d29af	remove perf log, unrelated file and so on	6 months ago

1 2 3 4 5 ...

3466 Commits (457a0de79fd2d3602eba0ac78e606acb6401fc60) All Branches Search

3466 Commits (457a0de79fd2d3602eba0ac78e606acb6401fc60)

All Branches