9 Commits (7f548a4297cfbe162aac42d7df9a309ea2a84beb)

Author	SHA1	Message	Date
Matt Proud	7f0d816574	Schedule the background compactors to run. ... This commit introduces three background compactors, which compact sparse samples together. 1. Older than five minutes is grouped together into chunks of 50 every 30 minutes. 2. Older than 60 minutes is grouped together into chunks of 250 every 50 minutes. 3. Older than one day is grouped together into chunks of 5000 every 70 minutes.	12 years ago
Matt T. Proud	a3f1d81e24	Publicize a few storage components for curation. ... This commit introduces the publicization of Stop and other components, which the compaction curator shall take advantage of.	12 years ago
Matt T. Proud	4298bab2b0	Publicize Curator and Processors. ... This commit publicizes the curation and processor frameworks for purposes of making them available in the main processor loop.	12 years ago
Matt T. Proud	3362bf36e2	Include curator status in web heads-up-display.	12 years ago
Matt T. Proud	e527941b6a	Use tagged struct fields.	12 years ago
Matt T. Proud	b3e34c6658	Implement batch database sample curator. ... This commit introduces to Prometheus a batch database sample curator, which corroborates the high watermarks for sample series against the curation watermark table to see whether a curator of a given type needs to be run. The curator is an abstract executor, which runs various curation strategies across the database. It remarks the progress for each type of curation processor that runs for a given sample series. A curation procesor is responsible for effectuating the underlying batch changes that are request. In this commit, we introduce the CompactionProcessor, which takes several bits of runtime metadata and combine sparse sample entries in the database together to form larger groups. For instance, for a given series it would be possible to have the curator effectuate the following grouping: - Samples Older than Two Weeks: Grouped into Bunches of 10000 - Samples Older than One Week: Grouped into Bunches of 1000 - Samples Older than One Day: Grouped into Bunches of 100 - Samples Older than One Hour: Grouped into Bunches of 10 The benefits hereof of such a compaction are 1. a smaller search space in the database keyspace, 2. better employment of compression for repetious values, and 3. reduced seek times.	12 years ago
Matt T. Proud	c3e3460ca6	Spin up curator run in the tests. ... After this commit, we'll need to add validations that it does the desired work, which we presently know that it doesn't. Given the changes I made with a plethora of renamings, I want to commit this now before it gets even larger.	12 years ago
Matt T. Proud	c53a72a894	Test data for the curator.	12 years ago
Matt T. Proud	3e97a3630d	Include nascent curator scaffolding. ... The curator doesn't do anything yet; rather, this is the type definition including the anciliary testing scaffold. Improve Makefile and Git developer experience. The top-level Makefile was a bit overloaded in terms of generation of assets and their management. This has been offloaded into separate Makefiles. The Git developer experience sucked due to lack of .gitignore policies. Also: Fix faulty skiplist naming from old merge.	12 years ago