prometheus

Commit Graph

Author	SHA1	Message	Date
Oleg Zaytsev	3dd18a1ece	Test that approxBytesPerEntry const is correct The comment on the constant mentions where it comes from, so we can actually check its value in that test. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	3 years ago
Christian Simon	50c1060328	Implement PostingsCloner and implement promise/future pattern for PostingsForMatchers (#14 ) * Fix tests not closing head/block properly Close Head properly in TestWalRepair_DecodingError Close Block properly in TestReadIndexFormatV1 Co-authored-by: Christian Simon <simon@swine.de> Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> * Implement PostingsCloner PostingsCloner allows obtaining independend clones of a Postings, that can be used for subsequent calls. Co-authored-by: Christian Simon <simon@swine.de> Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> * IndexReader.PostingsForMatchers() and implement Cache IndexReader now offers the PostingsForMatchers functionality, which is provided by PostingsForMatchersCache. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> Co-authored-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com> Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com> Co-authored-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Peter Štibraný <peter.stibrany@grafana.com>	3 years ago
Cyril Tovena	9173cade01	Merge pull request #15 from grafana/improvesetmatches Improve `findSetMatches `to support concatenation.	3 years ago
Cyril Tovena	ce28912439	Fixes a test now that we support more setMatches. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Peter Štibraný	b3ae917f96	Merge pull request #17 from grafana/symbols-cleanup Splitting compactor: remove unused symbols from index of compacted block	3 years ago
Cyril Tovena	92736182ce	Adding benchmark. ``` benchstat before.txt after.txt name old time/op new time/op delta PostingsForMatchers/Head/n="1"-16 480ns ± 4% 469ns ± 4% ~ (p=0.095 n=5+5) PostingsForMatchers/Head/n="1",j="foo"-16 720ns ± 1% 705ns ± 2% ~ (p=0.056 n=5+5) PostingsForMatchers/Head/j="foo",n="1"-16 712ns ± 3% 726ns ± 7% ~ (p=0.841 n=5+5) PostingsForMatchers/Head/n="1",j!="foo"-16 833ns ± 3% 827ns ± 1% ~ (p=0.548 n=5+5) PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo\|bar"-16 6.66ms ± 4% 0.00ms ± 1% -99.97% (p=0.008 n=5+5) PostingsForMatchers/Head/j=~"foo\|bar"-16 897ns ± 2% 675ns ± 2% -24.80% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"(1\|2\|3\|4\|5\|6\|20\|55)"-16 20.1ms ± 2% 0.0ms ± 2% -99.99% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"."-16 74.4ms ± 3% 81.3ms ± 3% +9.29% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"1."-16 20.3ms ±27% 17.2ms ± 2% -15.15% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".1"-16 6.22ms ±10% 5.60ms ± 1% -9.95% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".+"-16 146ms ± 4% 126ms ± 1% -13.63% (p=0.016 n=4+5) PostingsForMatchers/Head/i=~""-16 77.0ms ±23% 68.2ms ± 5% -11.47% (p=0.032 n=5+5) PostingsForMatchers/Head/i!=""-16 57.0ms ± 4% 56.9ms ± 6% ~ (p=1.000 n=5+5) PostingsForMatchers/Head/n="1",i=~".",j="foo"-16 77.1ms ± 2% 79.2ms ± 4% ~ (p=0.095 n=5+5) PostingsForMatchers/Head/n="1",i=~".",i!="2",j="foo"-16 77.2ms ± 3% 81.2ms ± 3% +5.17% (p=0.016 n=5+5) PostingsForMatchers/Head/n="1",i!=""-16 64.7ms ±34% 58.4ms ± 7% ~ (p=0.421 n=5+5) name old alloc/op new alloc/op delta PostingsForMatchers/Head/n="1"-16 80.0B ± 0% 80.0B ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",j="foo"-16 208B ± 0% 208B ± 0% ~ (all equal) PostingsForMatchers/Head/j="foo",n="1"-16 208B ± 0% 208B ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",j!="foo"-16 240B ± 0% 240B ± 0% ~ (all equal) PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo\|bar"-16 1.61MB ± 0% 0.00MB ± 0% ~ (p=0.079 n=4+5) PostingsForMatchers/Head/j=~"foo\|bar"-16 360B ± 0% 232B ± 0% -35.56% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"(1\|2\|3\|4\|5\|6\|20\|55)"-16 1.61MB ± 0% 0.00MB ± 0% -99.97% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"."-16 1.61MB ± 0% 1.61MB ± 0% ~ (all equal) PostingsForMatchers/Head/i=~"1."-16 3.15MB ± 0% 3.15MB ± 0% -0.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".1"-16 1.61MB ± 0% 1.61MB ± 0% -0.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".+"-16 17.3MB ± 0% 17.3MB ± 0% -0.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~""-16 17.3MB ± 0% 17.3MB ± 0% ~ (p=0.238 n=5+5) PostingsForMatchers/Head/i!=""-16 17.3MB ± 0% 17.3MB ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",i=~".",j="foo"-16 1.61MB ± 0% 1.61MB ± 0% -0.00% (p=0.032 n=5+5) PostingsForMatchers/Head/n="1",i=~".",i!="2",j="foo"-16 1.61MB ± 0% 1.61MB ± 0% ~ (p=0.651 n=5+4) PostingsForMatchers/Head/n="1",i!=""-16 17.3MB ± 0% 17.3MB ± 0% ~ (all equal) name old allocs/op new allocs/op delta PostingsForMatchers/Head/n="1"-16 4.00 ± 0% 4.00 ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",j="foo"-16 9.00 ± 0% 9.00 ± 0% ~ (all equal) PostingsForMatchers/Head/j="foo",n="1"-16 9.00 ± 0% 9.00 ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",j!="foo"-16 10.0 ± 0% 10.0 ± 0% ~ (all equal) PostingsForMatchers/Head/i=~"1[0-9]",j=~"foo\|bar"-16 19.0 ± 0% 23.0 ± 0% +21.05% (p=0.008 n=5+5) PostingsForMatchers/Head/j=~"foo\|bar"-16 13.0 ± 0% 7.0 ± 0% -46.15% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"(1\|2\|3\|4\|5\|6\|20\|55)"-16 4.00 ± 0% 13.00 ± 0% +225.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~"."-16 7.00 ± 0% 7.00 ± 0% ~ (all equal) PostingsForMatchers/Head/i=~"1."-16 11.1k ± 0% 11.1k ± 0% -0.01% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".1"-16 4.00 ± 0% 3.00 ± 0% -25.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~".+"-16 100k ± 0% 100k ± 0% -0.00% (p=0.008 n=5+5) PostingsForMatchers/Head/i=~""-16 100k ± 0% 100k ± 0% ~ (all equal) PostingsForMatchers/Head/i!=""-16 100k ± 0% 100k ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",i=~".",j="foo"-16 12.0 ± 0% 12.0 ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",i=~".*",i!="2",j="foo"-16 18.0 ± 0% 18.0 ± 0% ~ (all equal) PostingsForMatchers/Head/n="1",i!=""-16 100k ± 0% 100k ± 0% ~ (all equal) ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Peter Štibraný	b7b70066ae	Address review feedback.	3 years ago
Ganesh Vernekar	e37ff8f6b6	Merge remote-tracking branch 'upstream/main' into sync	3 years ago
Peter Štibraný	d116268e59	Always include empty symbol in the symbol table.	3 years ago
Peter Štibraný	9a50267ea5	Delete symbols files.	3 years ago
Ganesh Vernekar	10d4cb6dc0	Merge remote-tracking branch 'upstream/main' into release-2.30-merge	3 years ago
Peter Štibraný	259e09fe5f	When doing compaction with splitting, only use symbols from series that belong to given sharded block.	3 years ago
Cyril Tovena	bd4d997bee	lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Cyril Tovena	c751cfbfce	Improve `findSetMatches` to support concatenation. This improves how we find `SetMatches` for regexp. Notably it allows to support concatenation such as `api_(v1\|prom)_push` for which the resulting set matches are `api_v1_push` and `api_prom_push`. I had to support characters classes too since the syntax may try to optimize alternates with them. In the end the code is also more robust than the previous implementation relying on the stringyfied version of the regexp. This could be upstreamed later. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Ganesh Vernekar	10bc6e80ee	Fix panic on failed snapshot replay and don't hard fail replay on disabled exemplars (#9438 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	b30db03f35	Cut v2.30.2 (#9426 ) * Don't error on overlapping m-mapped chunks during WAL replay (#9381) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Reduce log level during WAL replay on overlapping m-map chunks (#9425) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Cut v2.30.2 Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	a7d499e19a	Reduce log level during WAL replay on overlapping m-map chunks (#9425 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8c597e5166	Don't error on overlapping m-mapped chunks during WAL replay (#9381 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Peter Štibraný	57daf79192	More review feedback.	3 years ago
Peter Štibraný	db7fa7621c	Use <ix+1>_of_<shardCount> formatting for better readibility.	3 years ago
Peter Štibraný	861f9083d8	Fix directory cleanup in case of compaction failure.	3 years ago
Peter Štibraný	ffd281ab9d	Address feedback.	3 years ago
Peter Štibraný	58dab3de8a	Source blocks are deletable only if they are ALL empty.	3 years ago
Peter Štibraný	336f4260db	Removed commented code.	3 years ago
Peter Štibraný	006c2d7d55	All output blocks will have the same timestamp. Minor updates.	3 years ago
Bryan Boreham	1fb3c1b598	Replace calls to strings.Compare (#9397 ) < is clearer and faster. As the documentation says, "Basically no one should use strings.Compare." Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Peter Štibraný	63dbb1c69a	Make lint happy.	3 years ago
Peter Štibraný	78396b67dd	Compactor with support for splitting input blocks into multiple output blocks.	3 years ago
Ganesh Vernekar	2bcd9f2f69	Link 2 more TSDB blog posts in tsdb/README.md (#9371 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	84f8307aa1	Merge remote-tracking branch 'upstream/main' into codesome/syncprom Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Darshan Chaudhary	1f688657bf	Call delete on head if interval overlaps (#9151 ) * Call delete on head if interval overlaps Signed-off-by: darshanime <deathbullet@gmail.com> * Garbage collect tombstones during head gc Signed-off-by: darshanime <deathbullet@gmail.com> * Truncate tombstones before min time during head gc Signed-off-by: darshanime <deathbullet@gmail.com> * Lock less by deleting all keys in a single pass Signed-off-by: darshanime <deathbullet@gmail.com> * Pass map to DeleteTombstones Signed-off-by: darshanime <deathbullet@gmail.com> * Create new slice to replace old one Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
Marco Pracucci	8708b2d086	Update upstream Prometheus Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Ganesh Vernekar	30534e99d9	Take snapshot only after closing the WAL (#9328 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8944520ccc	Fix deletion of old snapshots (#9314 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	2327236bb5	Decrement active_appenders metric when no samples added (#9230 ) * Decrement active_appenders metric when no samples added Also add a test that the metric is incremented and decremented as expected with and without samples. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Fix comment Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	87d909df4a	Remove symbols map from TSDB head (#9301 ) This saves memory, effort and locking. Since every symbol is also added to postings, `Symbols()` can be implemented there instead. This now has to build a map for deduplication, but `Symbols()` is only called for compaction, and `gc()` used to rebuild the symbols map after every compaction so not an additional cost. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Callum Styan	93886d8417	Fix div by 0 panic is resize function. (#9286 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Ganesh Vernekar	35b1a82594	Exemplars in snapshot (#9255 ) * Exemplars in snapshot Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix lint Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Add docs Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix lint Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix comments Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Levi Harrison	06afe6162c	Also ignore `func1` Signed-off-by: Levi Harrison <git@leviharrison.dev>	3 years ago
Julien Pivotto	d5676fb9e0	Merge pull request #9254 from prometheus/superq/go1.17 Build with Go 1.17 / npm 7 / node 16	3 years ago
SuperQ	e167a45c65	Add new Go build tags. Add new go:build comments based on 1.17 formatting[0]. [0]: https://golang.org/doc/go1.17#gofmt Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
Callum Styan	cc55e57c1b	Fix a data race in the loadWAL function caused by reusing the same error var in multiple goroutines (#9259 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Ganesh Vernekar	8a5d8c15e3	Do not replay checkpoint if it is covered by snapshot (#9226 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Marco Pracucci	6c38feef8e	Fixed BenchmarkLoadWAL Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Marco Pracucci	9537ca8b84	Merged upstream Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Marco Pracucci	175efada22	Avoid deadlock when processing duplicate series record (#9170 ) (#8 ) * Avoid deadlock when processing duplicate series record `processWALSamples()` needs to be able to send on its output channel before it can read the input channel, so reads to allow this in case the output channel is full. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * processWALSamples: update comment Previous text seems to relate to an earlier implementation. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Marco Pracucci	fbe211d3a8	Do not break exported functions signatures (#6 ) Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Bryan Boreham	9dfdc3eb36	Speed up BenchmarkPostings_Stats (#9213 ) The previous code re-used series IDs 1-1000 many times over, so a lot of time was spent ensuring the lists of series were in ascending order. The intended use of `MemPostings.Add()` is that all series IDs are unique, and changing the benchmark to do this lets it finish ten times faster. (It doesn't affect the benchmark result, just the setup code) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	328a74ca36	Fix bugs and add enhancements to the chunk snapshot (#9185 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Marco Pracucci	481299f4a5	Added series hash cache support to TSDB (#5 ) * Added series hash cache support to TSDB Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imports grouping Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago

1 2 3 4 5 ...

411 Commits (3dd18a1ece60f9dc77a41b1111a45226796a6650)