prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	90e98e0235	tsdb: create isolation transaction slice on demand When Prometheus restarts it creates every series read in from the WAL, but many of those series will be finished, and never receive any more samples. By defering allocation of the txRing slice to when it is first needed, we save 32 bytes per stale series. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Bryan Boreham	6fe8217ce4	tsdb: shrink txRing with smaller integers 4 billion active transactions ought to be enough for anyone. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
Matthieu MOREL	c8e7f95a3c	ci(lint): enable predeclared linter Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Matthieu MOREL	fb3eb21230	enable gocritic, unconvert and unused linters Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Bryan Boreham	ff00dee262	tsdb: turn off transaction isolation for head compaction (#11317 ) * tsdb: add a basic test for read/write isolation * tsdb: store the min time with isolationAppender So that we can see when appending has moved past a certain point in time. * tsdb: allow RangeHead to have isolation disabled This will be used when for head compaction. * tsdb: do head compaction with isolation disabled This saves a lot of work tracking appends done while compaction is ongoing. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	136f8b0ebb	tsdb: comment reason for isolation tracking reads (#11301 ) I find it useful to know why a restriction exists, to check whether that reason still applies, or in which other places it might apply. This is based on the note here: https://github.com/prometheus/prometheus/pull/9270#pullrequestreview-743820956 on the PR where the original comment was added. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Darshan Chaudhary	9dcf8b2208	Add the ability to disable tsdb isolation (#9270 ) * Disable isolation in isolation struct Signed-off-by: darshanime <deathbullet@gmail.com> * Run tsdb tests with isolation disabled Signed-off-by: darshanime <deathbullet@gmail.com> * Check for isolation disabled in isoState.Close() Signed-off-by: darshanime <deathbullet@gmail.com> * use t.Skip to skip isolation tests when disabled Signed-off-by: darshanime <deathbullet@gmail.com> * address review comments Signed-off-by: darshanime <deathbullet@gmail.com> * fix test for defaultIsolationState Signed-off-by: darshanime <deathbullet@gmail.com> * Change flag name. Set flag in DB. Do not init txRing. Close isoState. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Test disabled isolation in CircleCI test_go Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Skip isolation related tests in db_test.go Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Darshan Chaudhary	c4f2e9eec5	Add present_over_time (#9097 ) * Add present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Add tests for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Address PR comments Signed-off-by: darshanime <deathbullet@gmail.com> * Add documentation for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation comment Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
Ganesh Vernekar	59d02b5ef0	tsdb: Block Head GC till pending readers are done reading (#9081 ) * tsdb: Block Head GC till pending readers are done reading Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix review comments Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix review comments 2 Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix the exclusiveness of maxt in WaitForPendingReadersInTimeRange Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	dc8f505595	tsdb: coalesce lock/unlock operations for append (#9061 ) Fetch the low watermark value under the same lock as we need for the appender, rather than releasing then re-aquiring a lock on the same Mutex. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Peter Štibraný	ff80690a6e	Optimise lowWatermark in Isolation (#7332 ) * Track open appenders in doubly-linked list to make lowWatermark O(1). * Use RW locks. * Added BenchmarkIsolationWithState. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
beorn7	d9af11e636	Do not attempt isolation for appendID == 0 Signed-off-by: beorn7 <beorn@grafana.com>	5 years ago
beorn7	7f30b0984d	Implement isolation This has been ported from https://github.com/prometheus/tsdb/pull/306. Original implementation by @brian-brazil, explained in detail in the 2nd half of this talk: https://promcon.io/2017-munich/talks/staleness-in-prometheus-2-0/ The implementation was then processed by @gouthamve into the PR linked above. Relevant slide deck: https://docs.google.com/presentation/d/1-ICg7PEmDHYcITykD2SR2xwg56Tzf4gr8zfz1OerY5Y/edit?usp=drivesdk Signed-off-by: beorn7 <beorn@grafana.com> Co-authored-by: Brian Brazil <brian.brazil@robustperception.io> Co-authored-by: Goutham Veeramachaneni <gouthamve@gmail.com>	5 years ago

13 Commits (d541b3bbebbc889e02ff51dd54e5a1674bcf5b77)