5f50d974c9
scraping: reset symbol table periodically
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
4e748b9cd8
scraping: re-use labels Builder in scrape report metrics
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
abb3a62f04
scraping: re-use symbol table for scrape loops
...
One symbol table for all loops in the same scrape pool, i.e. from the
same job.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
0403d098e1
scraping: re-use symbolTable for target discovery
...
Call labels.NewBuilderWithSymbolTable.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
9ba13de220
scraping: pass a Builder to get Target labels
...
This saves memory allocations from making a new Builder every time.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
123109e967
config: make work with SymbolTable labels
...
Need a SymbolTable when we expand ExternalLabels.
And we can't use labels.NewBuilder because we need to add blank values.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
3716326f3f
rules: call NewScratchBuilder
...
Need to initialize ScratchBuilder with a SymbolTable.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
925134e6de
tsdb tests: make work with labels SymbolTable
...
Need to initialize decoders with SymbolTable.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
93b72ec5dd
tsdb: create SymbolTables for labels as required
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
8c9b028536
Merge pull request #13641 from bboreham/pick-13633
...
Cut 2.50.1 with metadata api bugfix (cherry-picked #13633 )
2024-02-26 11:31:35 +00:00
e537d6d1c1
Cut 2.50.1 with metadata api bugfix
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 10:00:21 +00:00
eecaa71ff1
Fix: metadata API using wrong field names ( #13633 )
...
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.
Extend tests to verify the JSON response.
Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:54:39 +00:00
2507469291
Fix: metadata API using wrong field names ( #13633 )
...
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.
Extend tests to verify the JSON response.
Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-26 09:53:39 +00:00
22890b1eb3
PromQL: improve warning for mixed values in aggregations
...
Aggregations discard the metric name, so don't try to
include it in the error message.
Add a test that generates this warning.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-25 19:57:06 +00:00
37613ef3b6
Merge branch 'chore/bump-to-go1.21' of https://github.com/arukiidou/prometheus into chore/bump-to-go1.21
2024-02-24 00:34:33 +09:00
e171f6c8aa
go: fix go.mod version from 1.21.7 to 1.21
...
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-24 00:33:35 +09:00
2ac1632eec
storage/remote: improve symbol-table handling
...
On the incoming path, `writeHandler.write()` creates a new table for
each request.
`labelProtosToLabels` takes a `ScratchBuilder` now.
Call `NewScratchBuilder` as required in tests.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
8f525b4ba4
storage/remote tests: refactor: extract function newTestQueueManager
...
To reduce repetition.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
b39286fd1f
Add dedupelabels tag to not build regular labels
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
d51a5344cd
labels: new version de-duplicating strings in SymbolTables
...
The individual strings for label names and values are held in a table,
and each Labels value is a run of varint-encoded indexes into that table.
When creating new labels, a sync.Mutex is locked around reads and writes.
When reading labels, there is no locking because the table of strings
used by those labels is immutable.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
28191109a8
Labels: add fake versions of SymbolTable apis
...
So we can use them where necessary for internlabels implementation.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
d1af84f6ee
Labels: move Builder and Reset out of common
...
New internstrings implementation is different.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
5aa4473894
labels tests: extend TestBuilder
...
Start with empty base labels, also check new and re-used symbol tables
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
bb82a57e64
Labels: Call NewScratchBuilder in test_utils
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
cc5dc6a61b
labels: use Equal instead of DeepEqual
...
This will work better with a different data structure.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-23 13:50:27 +00:00
4d6bb2e0e4
Merge pull request #13628 from bboreham/cleanup-13583
...
tsdb/wlog: small cleanup of WAL watcher after #13583
2024-02-23 12:39:17 +00:00
cc2c3f7c2b
Merge pull request #10782 from prymitive/gzip
...
Use github.com/klauspost/compress for gzip and zlib
2024-02-23 10:42:02 +00:00
08832e2db8
histogram: add UI tests for rendering histogram ( #13631 )
...
* histogram: add tests for rendering histograms
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* histogram: add tests for rendering histograms
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unused import
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
2024-02-23 11:31:39 +01:00
c6c8f63516
Merge pull request #13607 from fionaliao/ooo-samples-appended-type
...
Add sample type label to outOfOrderSamplesAppended metric
2024-02-23 09:41:45 +01:00
daeef6ffc3
Add lint checks for gzip/zlib
...
Since we want to use a faster gzip/zlib libraries we should add a lint check so that it's used everywhere
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:08:40 +00:00
5597020a60
Use github.com/klauspost/compress for gzip and zlib
...
klauspost/compress is a high quality drop-in replacement for common Go
compression libraries. Since Prometheus sends out a lot of HTTP requests
that often return compressed output having improved compression
libraries helps to save cpu & memory resources.
On a test Prometheus server I was able to see cpu reduction from 31 to
30 cores.
Benchmark results:
name old time/op new time/op delta
TargetScraperGzip/metrics=1-8 69.4µs ± 4% 69.2µs ± 3% ~ (p=0.122 n=50+50)
TargetScraperGzip/metrics=100-8 84.3µs ± 2% 80.9µs ± 2% -4.02% (p=0.000 n=48+46)
TargetScraperGzip/metrics=1000-8 296µs ± 1% 274µs ±14% -7.35% (p=0.000 n=47+45)
TargetScraperGzip/metrics=10000-8 2.06ms ± 1% 1.66ms ± 2% -19.34% (p=0.000 n=47+45)
TargetScraperGzip/metrics=100000-8 20.9ms ± 2% 17.5ms ± 3% -16.50% (p=0.000 n=49+50)
name old alloc/op new alloc/op delta
TargetScraperGzip/metrics=1-8 6.06kB ± 0% 6.07kB ± 0% +0.24% (p=0.000 n=48+48)
TargetScraperGzip/metrics=100-8 7.04kB ± 0% 6.89kB ± 0% -2.17% (p=0.000 n=49+50)
TargetScraperGzip/metrics=1000-8 9.02kB ± 0% 8.35kB ± 1% -7.49% (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8 18.1kB ± 1% 16.1kB ± 2% -10.87% (p=0.000 n=47+47)
TargetScraperGzip/metrics=100000-8 1.21MB ± 0% 1.01MB ± 2% -16.69% (p=0.000 n=36+50)
name old allocs/op new allocs/op delta
TargetScraperGzip/metrics=1-8 71.0 ± 0% 72.0 ± 0% +1.41% (p=0.000 n=50+50)
TargetScraperGzip/metrics=100-8 81.0 ± 0% 76.0 ± 0% -6.17% (p=0.000 n=50+50)
TargetScraperGzip/metrics=1000-8 92.0 ± 0% 83.0 ± 0% -9.78% (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8 93.0 ± 0% 91.0 ± 0% -2.15% (p=0.000 n=50+50)
TargetScraperGzip/metrics=100000-8 111 ± 0% 135 ± 1% +21.89% (p=0.000 n=40+50)
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:08:15 +00:00
92e381b8a3
Add a scrape benchmark with gzipped responses
...
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:07:22 +00:00
bf2658bec1
Merge pull request #13627 from aknuds1/arve/sync-otlp
...
otlptranslator: Upgrade to v0.95.0
2024-02-22 22:34:18 +05:30
9738e48a7d
Merge pull request #13629 from prometheus/release-2.50
...
merge back release 2.50
2024-02-22 11:07:52 +01:00
6ed56c9f04
WAL watcher: improve comments
...
Clarify in the first comment that it is `watch()` that waits, and reduce
verbiage.
The second comment was slightly contradictory to the first and otherwise
didn't seem to add much, since `currentSegment` was incremented just a
few lines later.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:32:46 +00:00
545c28d923
update v2.51 release date ( #13606 )
...
* update v2.51 release date
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* add Bryan as a next release shepherd
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
---------
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-22 10:23:26 +01:00
814b920e8a
Cut v2.50.0 ( #13612 )
...
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-22 10:21:59 +01:00
a975a83079
tsdb: clean up Watcher debug messages
...
Print lastSegment after it gets initialized.
Move variable declaration to first use.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:19:18 +00:00
78f46bccca
tsdb/wlog tests: remove unnecessary sleep check
...
Sleep() is documented to return immediately on negative or zero input.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-22 09:14:52 +00:00
bf5ca8cf38
otlptranslator: Upgrade to v0.95.0
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-02-22 09:12:07 +01:00
d55fe5064d
Merge branch 'prometheus:main' into chore/bump-to-go1.21
2024-02-22 12:42:07 +09:00
0c71230784
fix bug that would cause us to endlessly fall behind ( #13583 )
...
* fix bug that would cause us to only read from the WAL on the 15s
fallback timer if remote write had fallen behind and is no longer
reading from the WAL segment that is currently being written to
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* remove unintended logging, fix lint, plus allow test to take slightly
longer because cloud CI
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* address review feedback
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* fix watcher sleeps in test, flu brain is smooth
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* increase timeout, unfortunately cloud CI can require a longer timeout
Signed-off-by: Callum Styan <callumstyan@gmail.com>
---------
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2024-02-21 17:09:07 -08:00
e22b564049
Always align scrapes even if tolerance is bigger than 1% of scrape interval
...
When the scrape tolerance is bigger than 1% of the scrape interval, take
1% of the scrape interval as the tolerance instead of not aligning the
scrape at all.
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2024-02-21 15:09:21 +01:00
14e48cd3ce
Docs: update remote-write defaults
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-21 11:39:23 +00:00
841a133514
Move histogramsAppended to be more consistent
...
Signed-off-by: Fiona Liao <fiona.liao@grafana.com>
2024-02-21 11:15:04 +00:00
52389647b2
Add type label to outOfOrderSamplesAppended metric
...
Signed-off-by: Fiona Liao <fiona.liao@grafana.com>
2024-02-19 15:24:39 +00:00
93591ec5cc
Cut v2.50.0-rc.1 ( #13602 )
...
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-02-19 14:15:03 +01:00
39e858e524
go: bump go from 1.20 to 1.21
...
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-19 20:40:55 +09:00
ebb91bc111
ci: bump Go from 1.21 to 1.22
...
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
2024-02-19 20:40:00 +09:00
Bryan Boreham