3764 Commits (cf519dac6a5799b8f314aac6f510e2a98d3af9c6)

 

All Branches   

Search

Author	SHA1	Message	Date
yuehuayingxueluo	b45000f839	[Inference]Add Streaming LLM (#5745) ... * Add Streaming LLM * add some parameters to llama_generation.py * verify streamingllm config * add test_streamingllm.py * modified according to the opinions of review * add Citation * change _block_tables tolist	6 months ago
Hongxin Liu	ee6fd38373	[devops] fix docker ci (#5780)	6 months ago
Hongxin Liu	32f4187806	[misc] update dockerfile (#5776) ... * [misc] update dockerfile * [misc] update dockerfile	6 months ago
Haze188	e22b82755d	[CI/tests] simplify some test case to reduce testing time (#5755) ... * [ci/tests] simplify some test case to reduce testing time * [ci/tests] continue to remove test case to reduce ci time cost * restore some test config * [ci/tests] continue to reduce ci time cost	6 months ago
Yuanheng Zhao	406443200f	[Hotfix] Add missing init file in inference.executor (#5774)	6 months ago
duanjunwen	1b76564e16	[test] Fix/fix testcase (#5770) ... * [fix] branch for fix testcase; * [fix] fix test_analyzer & test_auto_parallel; * [fix] remove local change about moe; * [fix] rm local change moe;	6 months ago
flybird11111	3f2be80530	fix (#5765)	6 months ago
Hongxin Liu	68359ed1e1	[release] update version (#5752) ... * [release] update version * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [devops] update compatibility test * [test] fix ddp plugin test * [test] fix gptj and rpc test * [devops] fix cuda ext compatibility * [inference] fix flash decoding test * [inference] fix flash decoding test	6 months ago
Yuanheng Zhao	677cbfacf8	[Fix/Example] Fix Llama Inference Loading Data Type (#5763) ... * [fix/example] fix llama inference loading dtype * revise loading dtype of benchmark llama3	6 months ago
botbw	023ea13cb5	Merge pull request #5749 from hpcaitech/prefetch ... [Gemini] Prefetch next chunk before each op	6 months ago
hxwang	154720ba6e	[chore] refactor profiler utils	6 months ago
hxwang	8547562884	[chore] remove unnecessary assert since compute list might not be recorded	6 months ago
hxwang	e5e3320948	[bug] continue fix	6 months ago
hxwang	936dd96dbb	[bug] workaround for idx fix	6 months ago
botbw	e0dde8fda5	Merge pull request #5754 from Hz188/prefetch ... [Gemini]Prefetch benchmark	6 months ago
botbw	157b4cc357	Merge branch 'prefetch' into prefetch	6 months ago
genghaozhe	87665d7922	correct argument help message	6 months ago
Haze188	4d097def96	[Gemini] add some code for reduce-scatter overlap, chunk prefetch in llama benchmark. (#5751) ... * [bugs] fix args.profile=False DummyProfiler errro * add args.prefetch_num for benchmark	6 months ago
genghaozhe	b9269d962d	add args.prefetch_num for benchmark	6 months ago
genghaozhe	fba04e857b	[bugs] fix args.profile=False DummyProfiler errro	6 months ago
Yuanheng Zhao	b96c6390f4	[inference] Fix running time of test_continuous_batching (#5750)	6 months ago
Edenzzzz	5f8c0a0ac3	[Feature] auto-cast optimizers to distributed version (#5746) ... * auto-cast optimizers to distributed * fix galore casting * logger --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
hxwang	ca674549e0	[chore] remove unnecessary test & changes	6 months ago
hxwang	ff507b755e	Merge branch 'main' of github.com:hpcaitech/ColossalAI into prefetch	6 months ago
hxwang	63c057cd8e	[example] add profile util for llama	6 months ago
botbw	2fc85abf43	[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) ... * [gemini] async grad chunk reduce (all-reduce&reduce-scatter) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] add test * [gemini] rename func * [gemini] update llama benchmark * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] use tensor counter * [gemini] change default config in GeminiPlugin and GeminiDDP * [chore] typo * [gemini] fix sync issue & add test cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742) ... * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
hxwang	15d21a077a	Merge remote-tracking branch 'origin/main' into prefetch	6 months ago
binmakeswell	4647ec28c8	[inference] release (#5747) ... * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release	6 months ago
Yuanheng Zhao	df6747603f	[Colossal-Inference] (v0.1.0) Merge pull request #5739 from hpcaitech/feature/colossal-infer ... [Inference] Merge feature/colossal-infer	6 months ago
Yuanheng Zhao	498f42c45b	[NFC] fix requirements (#5744)	6 months ago
Yuanheng Zhao	bd38fe6b91	[NFC] Fix code factors on inference triton kernels (#5743)	6 months ago
Yuanheng Zhao	c2c8c9cf17	[ci] Temporary fix for build on pr (#5741) ... * temporary fix for CI * timeout to 90	6 months ago
botbw	13c06d36a3	[bug] fix early return (#5740) ... * [bug] fix silly bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [chore] add test for prefetch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Yuanheng Zhao	c06208e72c	Merge pull request #5737 from yuanheng-zhao/inference/sync/main ... [sync] Sync feature/colossal-infer with main	6 months ago
Haze188	22ce873c3f	[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) ... * [pre-commit.ci] auto fixes from pre-commit.com hooks * add parallel cross entropy output for falcon model & fix some typos in bloom.py * fix module name error, self.model -> self.transformers in bloom, falcon model * Fix the overflow bug of distributed cross entropy loss function when training with fp16 * add dtype to parallel cross entropy loss function * fix dtype related typos adn prettify the loss.py * fix grad dtype and update dtype mismatch error * fix typo bugs	6 months ago
Haze188	83716e9feb	Merge pull request #5738 from botbw/prefetch ... [chore] fix init error	6 months ago
pre-commit-ci[bot]	b3c0e6d871	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	137a7c341b	[chore] fix init error	6 months ago
Yuanheng Zhao	8633c15da9	[sync] Sync feature/colossal-infer with main	6 months ago
Yuanheng Zhao	d8b1ea4ac9	[doc] Update Inference Readme (#5736) ... * [doc] update inference readme * add contents * trivial	6 months ago
Yuanheng Zhao	bdf9a001d6	[Fix/Inference] Add unsupported auto-policy error message (#5730) ... * [fix] auto policy error message * trivial	6 months ago
botbw	f5b7de38a4	Merge pull request #5733 from Hz188/feature/prefetch ... [Gemini] implement auto policy prefetch and a little origin code modification.	6 months ago
genghaozhe	90d8d0183c	remove personal comments	6 months ago
genghaozhe	bfcb2d1ff8	refactor the code structure to solve the circular import	6 months ago
genghaozhe	a280517dd9	remove unrelated file	6 months ago
genghaozhe	3b363d44cc	Merge branch 'feature/prefetch' of https://github.com/Hz188/ColossalAI into feature/prefetch	6 months ago
genghaozhe	1ec92d29af	remove perf log, unrelated file and so on	6 months ago
genghaozhe	5c6c5d6be3	remove comments	6 months ago
genghaozhe	df63db7e63	remote comments	6 months ago