prometheus

Commit Graph

Author	SHA1	Message	Date
naivewong	13c80a5979	Optimize queries using regex matchers for set lookups (#602 ) * Original version of the set optimization Signed-off-by: naivewong <867245430@qq.com> * simple set matcher Signed-off-by: naivewong <867245430@qq.com> * simple set matcher Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * add benchmark Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update benchmark Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update benchmark Signed-off-by: naivewong <867245430@qq.com> * update benchmark Signed-off-by: naivewong <867245430@qq.com> * update benchmark Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com> * use genSeries from #467 Signed-off-by: naivewong <867245430@qq.com> * update Signed-off-by: naivewong <867245430@qq.com>	6 years ago
zhulongcheng	19d402d154	cleanup tmp files (#570 ) Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>	6 years ago
zhulongcheng	4d03c70800	Verify chunk format version (#544 ) Verify chunk format version.	6 years ago
Krasi Georgiev	c3ffdf1a99	Test createBlock and check all os.RemoveAll in the tests for errors. (#549 ) Testing that createBlock creates blocks that can be opened. and checking the os.RemoveAll for errors will catch errors for un-closed files under windows. Many missing .Close() calls were added for fixing failing os.RemoveAll Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	c59ed492b2	Vertical query merging and compaction (#370 ) * Vertical series iterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Select overlapped blocks first in compactor Plan() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Added vertical compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Code cleanup and comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add benchmark for compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Perform vertical compaction only when blocks are overlapping. Actions for vertical compaction: * Sorting chunk metas * Calling chunks.MergeOverlappingChunks on the chunks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for vertical compaction * BenchmarkNormalCompaction => BenchmarkCompaction * Moved the benchmark from db_test.go to compact_test.go Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for query iterator and seek for non overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Vertical query merge only for overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Simplify logging in Compact(...) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG.md Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Calculate overlapping inside populateBlock Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * MinTime and MaxTime for BlockReader. Using this to find overlapping blocks in populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Sort blocks w.r.t. MinTime in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping in LeveledCompactor.write() instead of returning bool Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping inside LeveledCompactor.populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor createBlock to take optional []Series Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * review1 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com> * Updated CHANGELOG and minor nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor iterator and seek benchmarks for Querier. Also has as overlapping blocks. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Additional test case Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Split genSeries into genSeries and populateSeries Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Check error in benchmark Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Warn about overlapping blocks in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Krasi Georgiev	ce4a2083fb	nit Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	00d5e19baf	Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	dac2b97dfd	make createBlock more generic so it can be used in other tests. (#489 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	cc277e398b	fix the refs logic for the addFast path for createBlock (#504 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	9874377ead	nits Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	921f82cfc1	WIP Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	0b31b8af88	Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	1a9d08adc5	Don't write empty blocks (#374 ) * Dont write empty blocks when a compaction results in a block with no samples. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
mknapphrt	ebf5d74325	Added storage size based retention method and new metrics (#343 ) Added methods needed to retain data based on a byte limitation rather than time. Limitation is only applied if the flag is set (defaults to 0). Both blocks that are older than the retention period and the blocks that make the size of the storage too large are removed. 2 new metrics for keeping track of the size of the local storage folder and the amount of times data has been deleted because the size restriction was exceeded. Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>	6 years ago
Krasi Georgiev	eb6586f513	rename createPopulatedBlock to createBlock and use it instead ot createEmptyBlock (#488 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	a90a7195d4	createPopulatedBlock returns the block dir instead of the block (#487 ) It is easy to forget to close the block returned by createPopulatedBlock which causes failures for windows so instead it returns the block dir and which can be used by OpenBlock explicitly. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	bd5ccee5c1	use context to cancel compactions Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	bac9cbed2e	no overlapping on compaction when an existing block is not within default boundaries. (#461 ) closes https://github.com/prometheus/prometheus/issues/4643 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	5a9ddeecef	fix lint errors (#439 ) unexported NewMemTombstones as this returns unexported memTombstones type which will not be shows in godoc. Added missing comments for exported methods. Removed unused RecordLogger,RecordReader interfaces. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Chris Marchbanks	f4afc7dff2	Add benchmark for querying a persisted block (#425 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
codwu	84a45cb79a	add rwmutex to prevent concurrent map read when delete series Signed-off-by: codwu <wuhan9087@163.com>	7 years ago
Fabian Reinartz	76c1b2cdb6	Ensure correct block order on reload Due to a regression blocks were no longer ordered by time before being stored in memory. This made data intermittently become unavailable for queries. Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Krasi Georgiev	eb6cdbcb5b	some test funcitons didn't cleanup after themselves.	7 years ago
Fabian Reinartz	a345231a0e	Add test for block meta version and comments	7 years ago
Shubheksha Jalan	510dc17066	fix tests, set version when initializing the index writer	7 years ago
Fabian Reinartz	da9614fa54	Allow tombstone file to not exist	7 years ago
Fabian Reinartz	54205903f7	Move index and chunk encoders to own packages	7 years ago
Fabian Reinartz	adb6be73dd	move label read function int labels pkg	7 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	7 years ago
Callum Styan	ca4e817372	move test utils to it's own package so we can use it in tests for other packages in tsdb repo	7 years ago
Goutham Veeramachaneni	f35e9c7fec	Merge branch 'master' into fix-dels Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	1fc94a02d1	Don't retry failed compactions. Fixes prometheus/prometheus#3487 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	44e9ae38b5	Incorporate PR feedback. * Expose Stone as it is used in an exported method. * Move from tombstoneReader to []Stone for the same reason as above. * Make WAL reading a little cleaner. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	34a86af3c6	Move tombstones to their own thing. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	3de55171d3	Add tests for tombstones and deletedIterator Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago

35 Commits (abd15845e226211122a7ee9c79a72017d199cf4e)