prometheus

Commit Graph

Author	SHA1	Message	Date
Callum Styan	3344bb5c33	Move WAL watcher code to tsdb/wal package. (#5999 ) * Move WAL watcher code to tsdb/wal package. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix tests after moving WAL watcher code. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Lint fixes. Signed-off-by: Callum Styan <callumstyan@gmail.com>	5 years ago
Bartek Plotka	f0863a604e	Removed extra tsdb/testutil after merge. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	5 years ago
Ganesh Vernekar	5ecef3542d	Cleanup after merging tsdb into prometheus Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ganesh Vernekar	7cf09b0395	Moving tsdb into its own subdirectory Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ganesh Vernekar	b1cd829030	Reuse Chunk Iterator (#642 ) * Reset method for chunkenc.Iterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Reset method only for XORIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Use Reset(...) in querier.go Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Reuse deletedIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Another way of reusing chunk iterators Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Unexport xorIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix memSeries.iterator(...) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add some comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Chris Marchbanks	b40cc43958	Provide option to compress WAL records (#609 ) In running Prometheus instances, compressing the records was shown to reduce disk usage by half while incurring a negligible CPU cost. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
Krasi Georgiev	69740485c1	move the wal repair logic in db.Open (#633 ) * move the wal repair logic in db.Open This is to allow opening a wal in a read oly mode without triggering a repair. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>	6 years ago
naivewong	b05ca5c36e	fix two benchmarks (#631 ) * fix two benchmarks Signed-off-by: naivewong <867245430@qq.com>	6 years ago
Krasi Georgiev	3549c097b4	add some comments for the different WAL repair tests (#630 ) The WAL repair tests look similar so added some extra comments to clarify their purpose. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>	6 years ago
Brian Brazil	be4edbe174	Start a new WAL segement on head truncation. (#605 ) This reduces disk space usage to not be a minimum of 3 128MB files in small setups. This will possibly also help debug wal data issues, by making things a bit more deterministic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	149c5dc73a	Handle multiple refs for the same series when WAL reading. (#623 ) This can happen if a given series is created/truncated/recreated. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	30d0ea59d7	Don't crash on an unknown tombstone ref. (#604 ) Fixes https://github.com/prometheus/prometheus/issues/5562 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Krasi Georgiev	288f67efbf	fix some unchecked errors and remove unused vars. (#592 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Brian Brazil	dfed85e4a4	Keep series that are still in WAL in checkpoints (#577 ) If all the samples are deleted for a series, we should still keep the series in the WAL as anything else reading the WAL will still care about it in order to understand the samples. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Krasi Georgiev	c3ffdf1a99	Test createBlock and check all os.RemoveAll in the tests for errors. (#549 ) Testing that createBlock creates blocks that can be opened. and checking the os.RemoveAll for errors will catch errors for un-closed files under windows. Many missing .Close() calls were added for fixing failing os.RemoveAll Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	c59ed492b2	Vertical query merging and compaction (#370 ) * Vertical series iterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Select overlapped blocks first in compactor Plan() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Added vertical compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Code cleanup and comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add benchmark for compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Perform vertical compaction only when blocks are overlapping. Actions for vertical compaction: * Sorting chunk metas * Calling chunks.MergeOverlappingChunks on the chunks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for vertical compaction * BenchmarkNormalCompaction => BenchmarkCompaction * Moved the benchmark from db_test.go to compact_test.go Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for query iterator and seek for non overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Vertical query merge only for overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Simplify logging in Compact(...) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG.md Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Calculate overlapping inside populateBlock Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * MinTime and MaxTime for BlockReader. Using this to find overlapping blocks in populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Sort blocks w.r.t. MinTime in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping in LeveledCompactor.write() instead of returning bool Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping inside LeveledCompactor.populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor createBlock to take optional []Series Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * review1 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com> * Updated CHANGELOG and minor nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor iterator and seek benchmarks for Querier. Also has as overlapping blocks. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Additional test case Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Split genSeries into genSeries and populateSeries Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Check error in benchmark Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Warn about overlapping blocks in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Krasi Georgiev	dac2b97dfd	make createBlock more generic so it can be used in other tests. (#489 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	4592b77035	Patch by Krasi (https://github.com/codesome/tsdb/pull/3 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Ganesh Vernekar	d7e505db34	Dont store stones in head, delete samples directly Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Simon Pasquier	d5d7a097e1	Update Makefile.common This change also uses the latest staticcheck version which comes with new verifications, hence some clean up in the code. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Krasi Georgiev	48c439d26d	fix statick check errors (#475 ) fix the tests for `check_license` and `staticcheck` the static check also found some actual bugs. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	520ab7dc53	re-add the missing prometheus_tsdb_wal_corruptions_total (#473 ) closes https://github.com/prometheus/tsdb/issues/471 after implementing the new WAL this metric was missing so adding it again. Also added it in a test to make sure it works as expected. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	2962202ed3	fix windows tests (#469 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	bac9cbed2e	no overlapping on compaction when an existing block is not within default boundaries. (#461 ) closes https://github.com/prometheus/prometheus/issues/4643 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	0493efb7c5	repair wal when the record cannot be decoded (#453 ) * repair wal when the record cannot be decoded Currently repair is run only when the error happens in the reader. A corruption can occur after the record is read and when it is decoded. This change wraps the error at decoding as a CorruptionErr as this error is expected to trigger a repair. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	5a9ddeecef	fix lint errors (#439 ) unexported NewMemTombstones as this returns unexported memTombstones type which will not be shows in godoc. Added missing comments for exported methods. Removed unused RecordLogger,RecordReader interfaces. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	d804a27062	refactor util funcs to allow re-usage. (#419 ) * refactor util funcs to allow reusage. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ben	a8351dc9d0	Using filepath.Join() instead of strings with slashes (#428 ) fixes: https://github.com/prometheus/tsdb/issues/426 Using `filepath.Join()` instead of strings containing forward slash path delimiters (needed for non-*nix OSes), as suggested by @krasi-georgiev	6 years ago
Krasi Georgiev	d05611c027	removed some unused code and moved mockSeriesSet in querier_test (#394 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Chris Marchbanks	2db59a71a6	Fix assert order being backwards in a few places (#388 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
Ganesh Vernekar	e779603633	Reuse testutil (#393 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Chris Marchbanks	a8966cb53d	Fix race condition between gc and committing (#378 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
Krasi Georgiev	ad459ca1f4	re-added TestDeleteUntilCurMax TestDelete_e2e (#382 ) no logic changes just uncommented and fixed the issues. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Alexey Palazhchenko	9be883670e	fix the test data file path in benchmarks. (#345 ) Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>	6 years ago
Fabian Reinartz	def912ce0e	Integrate new WAL and checkpoints Remove the old WAL and drop in the new one Signed-off-by: Fabian Reinartz <freinartz@google.com>	6 years ago
Fabian Reinartz	ea607b9fc3	Log series on rollback Signed-off-by: Fabian Reinartz <freinartz@google.com>	6 years ago
codwu	84a45cb79a	add rwmutex to prevent concurrent map read when delete series Signed-off-by: codwu <wuhan9087@163.com>	7 years ago
Alexey Palazhchenko	64cce683cc	gofmt -s. Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>	7 years ago
Mario Trangoni	b7173eb0e5	fix some comments typos (#315 )	7 years ago
Simon Pasquier	79defa54df	Fix crash when a series has no block	7 years ago
Fabian Reinartz	adb6be73dd	move label read function int labels pkg	7 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	7 years ago
Goutham Veeramachaneni	e6e3fa566a	Remove verbose log line that is causing travis issues Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Callum Styan	bcde61b237	we shouldn't assume we'll receive the labels in sorted order	7 years ago
Callum Styan	8326e410d0	decouple tsdb main.go (and tests) from prometheus/pkg	7 years ago
Goutham Veeramachaneni	05d62ca842	Make sure gc'ed chunks are handled properly Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Callum Styan	df7cc4dff5	remove Nil/NotNil functions	7 years ago
Callum Styan	ca4e817372	move test utils to it's own package so we can use it in tests for other packages in tsdb repo	7 years ago
Callum Styan	5aa2af255b	use test utils in head_test	7 years ago
Fabian Reinartz	3ef4326114	Refactor tombstone reader types	7 years ago

1 2 3 4 5

220 Commits (88bb05c3e8d84de147f219f6e4c891d8ba87edc2)