prometheus

Commit Graph

Author	SHA1	Message	Date
Fabian Reinartz	e5ce2bef43	Add explicit error to Querier.Select This has been a frequent source of debugging pain since errors are potentially delayed to a much later point. They bubble up in an unrelated execution path.	7 years ago
Fabian Reinartz	82796db37b	Ensure near-empty chunks end at correct boundary We were determining a chunk's end time once it was one quarter full to compute it so all chunks have uniform number of samples. This accidentally skipped the case where series started near the end of a chunk range/block and never reached that threshold. As a result they got persisted but were continued across the range. This resulted in corrupted persisted data.	7 years ago
Fabian Reinartz	f8e88bfdb7	Close previous block queriers on error This ensures we close all previously opened queriers if on of the block querier fails to open. Also swap in new blocks before closing old ones to avoid the situation in general. Make read locking of blocks more conservative to avoid unnecessary retries by clients, e.g. when blocks are getting closed before we can successfully instantiate querier against them.	7 years ago
Fabian Reinartz	665955da48	Clarify postings index semantics, handle staleness The postings list index may point to series that no longer exist during garbage collection. This clarifies that this is valid behavior. It would be possible, though more complex, to always keep them in sync. However, series existance means nothing in itself as the queried time range defines whether there's actual data. Thus our definition is sane overall as long as drift is kept small.	7 years ago
Fabian Reinartz	fb9da52b11	Add more verbose error handling for closing, reduce locking This commit introduces error returns in various places and is explicit about closing persisted blocks. {Index,Chunk,Tombstone}Readers are more consistent about their Close() method. Whenever a reader is retrieved, the corresponding close method must eventually be called. We use this to track pending readers against persisted blocks. Querier's against the DB no longer hold a read lock for their entire lifecycle. This avoids long running queriers to starve new ones when we have to acquire a write lock when reloading blocks.	7 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	7 years ago
Goutham Veeramachaneni	c463b0c8c8	Expose NewMergedSeriesSet for merging SeriesSets Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	5b242f35ba	Expose a Querier with manually passed in readers. Allows people to not copy the querying code. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	7438ed7035	Expose Intervals type for use by TombstoneReader. TombstoneReader is exposed but Intervals is not. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	2644c8665c	Don't allocate ChunkMetas, reuse postings slices	7 years ago
Fabian Reinartz	96d7f540d4	Persist series without allocating the full set Change index persistence for series to not be accumulated in memory before being written as one large batch. `Labels` and `ChunkMeta` objects are reused. This cuts down memory spikes during compaction of multiple blocks significantly. As part of the the Index{Reader,Writer} now have an explicit notion of symbols and series must be inserted in order.	7 years ago
Dmitry Ilyevsky	37194b7a30	Add prefix label matcher. Implement labels.PrefixMatcher and use interface conversion in querier to optimize label tuples search. [unit-tests]: Fix bug and populate label index for mock index. Signed-off-by: Dmitry Ilyevsky <ilyevsky@gmail.com>	7 years ago
Fabian Reinartz	3065be97d8	Fix and document locking order for DB	7 years ago
Fabian Reinartz	bda3ed20ac	Fix omitting of chunk on Seek()	8 years ago
Fabian Reinartz	3410559c1b	Compact head block early Let older head blocks be compacted once the newest once has samples at 50% of its total range. This allows the memory of the compacted blocks to be released and garbage collected before a new head block gets created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3 and memory spikes are reduced.	8 years ago
Fabian Reinartz	a2948f3c5f	Merge pull request #97 from Gouthamve/nit Fix bug with Seek() and optimise bounding params.	8 years ago
Fabian Reinartz	eb8c9759fc	Properly balance k-way operations	8 years ago
Goutham Veeramachaneni	9e1f34dae3	Fix bug with Seek() and optimise bounding params. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	44e9ae38b5	Incorporate PR feedback. * Expose Stone as it is used in an exported method. * Move from tombstoneReader to []Stone for the same reason as above. * Make WAL reading a little cleaner. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	f29fb62fba	Make TombstoneReader a Getter. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	244b73fce1	Rename for clarity and consistency. Misc. changes for code cleanliness. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	d6bd64357b	Fix Delete on HeadBlock Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	22c1b5b492	Make SeriesSets use tombstones. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	cea3c88f17	Add Tombstones() method to Block. Also add Seek() to TombstoneReader Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	d57f269eb4	Make Select() reusable. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	c1939b7136	Simply loop away from using tracking variables. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	7bbbd55aad	Fix bug where having one chunk can cause panic When we have only one chunk that is out of range, then we are returning it unpopulated (w/o calling `Chunk(ref)`). This would cause a panic downstream. Fixes: prometheus/prometheus#2629 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	085991c9da	Merge pull request #71 from Gouthamve/tests Fix #59	8 years ago
Goutham Veeramachaneni	8b43b0d2c1	Fix broken tests Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	6169c33fb8	Fix #59 Mutating the chunks can change their length. Hence referencing using previous indices can cause panics. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	a5322c50fc	Merge pull request #36 from Gouthamve/tests Tests For Querier and Postings	8 years ago
Goutham Veeramachaneni	239f8b9eb5	Modify tests to check for mint, maxt Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	b60c2068bc	Make blockQuerier return data in valid time-range Fixes #43 Added mint, maxt to chunkSeriesIterator. Adding a field there is inevitable as something similar is required for ignoring deleted time-ranges. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	778103b450	Add liecence file and headers	8 years ago
Goutham Veeramachaneni	0c5c4f07c5	Test "Stitching" SeriesIterators Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	7de2217011	Add fast-path for equality matching	8 years ago
Fabian Reinartz	10c7c9acbe	Adjust import names to new repository organisation	8 years ago
Fabian Reinartz	767b88a3a5	Parallelize commits to prevent deadlocks This parallelizes commits to prevent deadlocks across inconsistently locked heads. As commits are currently not fully atomic across heads, this does decrease our guarantees.	8 years ago
Fabian Reinartz	3be4ef94ce	Move BufferedSeriesIterator in own package This functionality is useful for a lot of clients but not relevant to the TSDB's core features.	8 years ago
Fabian Reinartz	a4be181d3c	Merge branch 'master' into lint-vet	8 years ago
Fabian Reinartz	2c999836fb	Add Queryable interface to Block This adds the Queryable interface to the Block interface. Head and persisted blocks now implement their own Querier() method and thus isolate customization (e.g. remapPostings) more cleanly.	8 years ago
Goutham Veeramachaneni	761e4768f3	Lint and Vet Fixes	8 years ago
Fabian Reinartz	65b846ae5b	Remove unreturned locks, detect writes on closed heads	8 years ago
Fabian Reinartz	a8e8903350	Use ChunkMeta references for clarity This has been a common source of hard to debug issues. Its a premature and unbenchmarked optimization and semantically, we want ChunkMetas to be references in all changed cases.	8 years ago
Fabian Reinartz	d6fb6aaaa8	Rename paritionSeriesSet to mergedSeriesSet	8 years ago
Fabian Reinartz	2841499212	Simplify and split up iterators This simplifies some of the iterators by loading chunks from the ChunkReader earlier, filtering of chunks vs filtering or series is split into separate iterators for easier testing	8 years ago
Fabian Reinartz	87805fb83f	Remove Partitioned* code	8 years ago
Fabian Reinartz	55a9b5428a	Add separate head mutex Introduce a seperate mutex for the head blocks to avoid a race where a post-compaction reload may run between switching the DB's base mutex to create a new head block in an appender.	8 years ago
Fabian Reinartz	78780cd2ba	Segment chunk file This adds write path support for segmented chunk data files. Files of 512MB are pre-allocated and written to. If the file size is exceeded, the next file is started. On completion, files are truncated to their final size.	8 years ago
Fabian Reinartz	a3d042b54e	Support multiple chunk files in read path	8 years ago

1 2

96 Commits (e5ce2bef43af82d5943d8eeda3613c1c98397cc9)