3456 Commits (7997683aac44cf99529589af4262fba52b29a74b)

 

All Branches   

Search

Author	SHA1	Message	Date
Yuanheng Zhao	677cbfacf8	[Fix/Example] Fix Llama Inference Loading Data Type (#5763) ... * [fix/example] fix llama inference loading dtype * revise loading dtype of benchmark llama3	6 months ago
botbw	023ea13cb5	Merge pull request #5749 from hpcaitech/prefetch ... [Gemini] Prefetch next chunk before each op	6 months ago
hxwang	154720ba6e	[chore] refactor profiler utils	6 months ago
hxwang	8547562884	[chore] remove unnecessary assert since compute list might not be recorded	6 months ago
hxwang	e5e3320948	[bug] continue fix	6 months ago
hxwang	936dd96dbb	[bug] workaround for idx fix	6 months ago
botbw	e0dde8fda5	Merge pull request #5754 from Hz188/prefetch ... [Gemini]Prefetch benchmark	6 months ago
botbw	157b4cc357	Merge branch 'prefetch' into prefetch	6 months ago
genghaozhe	87665d7922	correct argument help message	6 months ago
Haze188	4d097def96	[Gemini] add some code for reduce-scatter overlap, chunk prefetch in llama benchmark. (#5751) ... * [bugs] fix args.profile=False DummyProfiler errro * add args.prefetch_num for benchmark	6 months ago
genghaozhe	b9269d962d	add args.prefetch_num for benchmark	6 months ago
genghaozhe	fba04e857b	[bugs] fix args.profile=False DummyProfiler errro	6 months ago
Yuanheng Zhao	b96c6390f4	[inference] Fix running time of test_continuous_batching (#5750)	6 months ago
Edenzzzz	5f8c0a0ac3	[Feature] auto-cast optimizers to distributed version (#5746) ... * auto-cast optimizers to distributed * fix galore casting * logger --------- Co-authored-by: Edenzzzz <wtan45@wisc.edu>	6 months ago
hxwang	ca674549e0	[chore] remove unnecessary test & changes	6 months ago
hxwang	ff507b755e	Merge branch 'main' of github.com:hpcaitech/ColossalAI into prefetch	6 months ago
hxwang	63c057cd8e	[example] add profile util for llama	6 months ago
botbw	2fc85abf43	[gemini] async grad chunk reduce (all-reduce&reduce-scatter) (#5713) ... * [gemini] async grad chunk reduce (all-reduce&reduce-scatter) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] add test * [gemini] rename func * [gemini] update llama benchmark * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [gemini] use tensor counter * [gemini] change default config in GeminiPlugin and GeminiDDP * [chore] typo * [gemini] fix sync issue & add test cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Jianghai	85946d4236	[Inference]Fix readme and example for API server (#5742) ... * fix chatapi readme and example * updating doc * add an api and change the doc * remove * add credits and del 'API' heading * readme * readme	6 months ago
hxwang	15d21a077a	Merge remote-tracking branch 'origin/main' into prefetch	6 months ago
binmakeswell	4647ec28c8	[inference] release (#5747) ... * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release * [inference] release	6 months ago
Yuanheng Zhao	df6747603f	[Colossal-Inference] (v0.1.0) Merge pull request #5739 from hpcaitech/feature/colossal-infer ... [Inference] Merge feature/colossal-infer	6 months ago
Yuanheng Zhao	498f42c45b	[NFC] fix requirements (#5744)	6 months ago
Yuanheng Zhao	bd38fe6b91	[NFC] Fix code factors on inference triton kernels (#5743)	6 months ago
Yuanheng Zhao	c2c8c9cf17	[ci] Temporary fix for build on pr (#5741) ... * temporary fix for CI * timeout to 90	6 months ago
botbw	13c06d36a3	[bug] fix early return (#5740) ... * [bug] fix silly bug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [chore] add test for prefetch * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	6 months ago
Yuanheng Zhao	c06208e72c	Merge pull request #5737 from yuanheng-zhao/inference/sync/main ... [sync] Sync feature/colossal-infer with main	6 months ago
Haze188	22ce873c3f	[Shardformer] Add parallel output for shardformer models(bloom, falcon) (#5702) ... * [pre-commit.ci] auto fixes from pre-commit.com hooks * add parallel cross entropy output for falcon model & fix some typos in bloom.py * fix module name error, self.model -> self.transformers in bloom, falcon model * Fix the overflow bug of distributed cross entropy loss function when training with fp16 * add dtype to parallel cross entropy loss function * fix dtype related typos adn prettify the loss.py * fix grad dtype and update dtype mismatch error * fix typo bugs	6 months ago
Haze188	83716e9feb	Merge pull request #5738 from botbw/prefetch ... [chore] fix init error	6 months ago
pre-commit-ci[bot]	b3c0e6d871	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	137a7c341b	[chore] fix init error	6 months ago
Yuanheng Zhao	8633c15da9	[sync] Sync feature/colossal-infer with main	6 months ago
Yuanheng Zhao	d8b1ea4ac9	[doc] Update Inference Readme (#5736) ... * [doc] update inference readme * add contents * trivial	6 months ago
Yuanheng Zhao	bdf9a001d6	[Fix/Inference] Add unsupported auto-policy error message (#5730) ... * [fix] auto policy error message * trivial	6 months ago
botbw	f5b7de38a4	Merge pull request #5733 from Hz188/feature/prefetch ... [Gemini] implement auto policy prefetch and a little origin code modification.	6 months ago
genghaozhe	90d8d0183c	remove personal comments	6 months ago
genghaozhe	bfcb2d1ff8	refactor the code structure to solve the circular import	6 months ago
genghaozhe	a280517dd9	remove unrelated file	6 months ago
genghaozhe	3b363d44cc	Merge branch 'feature/prefetch' of https://github.com/Hz188/ColossalAI into feature/prefetch	6 months ago
genghaozhe	1ec92d29af	remove perf log, unrelated file and so on	6 months ago
genghaozhe	5c6c5d6be3	remove comments	6 months ago
genghaozhe	df63db7e63	remote comments	6 months ago
genghaozhe	7416e4943b	fix conflicts to beautify the code	6 months ago
botbw	f5a5287f87	Merge pull request #5731 from botbw/prefetch ... [gemini] prefetch for auto policy	6 months ago
genghaozhe	d22bf30ca6	implement auto policy prefetch and modify a little origin code.	6 months ago
pre-commit-ci[bot]	f1918e18a5	[pre-commit.ci] auto fixes from pre-commit.com hooks ... for more information, see https://pre-commit.ci	6 months ago
hxwang	a55a9e298b	[gemini] init auto policy prefetch	6 months ago
Yuanheng Zhao	283c407a19	[Inference] Fix Inference Generation Config and Sampling (#5710) ... * refactor and add * config default values * fix gen config passing * fix rpc generation config	6 months ago
Haze188	c5ddf17c76	Merge branch 'hpcaitech:feature/prefetch' into feature/prefetch	6 months ago
genghaozhe	06a3a100b3	remove unrelated code	6 months ago