prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	7c28521451	[TESTS] Truncate some long test names, for readability The strings produced by these tests can run to thousands of characters, which makes test logs difficult to read. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
carehabit	a672662073	all: fix some typos (#13863 ) Signed-off-by: carehabit <shenyuting@outlook.com>	8 months ago
Domantas	435f330d0b	[BUGFIX] labels: don't modify original labels in DropMetricName (#13845 ) Restrict the capacity of first argument to `append()` to force an allocation. This is for the slice implementation only. Signed-off-by: Domantas Jadenkus <djadenkus@gmail.com>	8 months ago
Bryan Boreham	48786ad4e8	Use slices insteda of exp/slices Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	080d440bf8	Merge remote-tracking branch 'origin/main' into pr/13461	8 months ago
Oleg Zaytsev	d12e785075	Improve readability As suggested by @bboreham Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	8 months ago
Oleg Zaytsev	9699598952	Improve Labels.Compare performance w/stringlabels I was bored on a train and I spent some amount of time trying to scratch some nanoseconds off the Labels.Compare when running with stringlabels. I would be ashamed to admit the real amount of time I spent on it. The worst thing is, I can't really explain why this is performing so much better, and someone should re-run the benchmarks on their machine to confirm that it's not something related to general relativity because the train is moving. I also added some extra real-life benchmark cases with longer labelsets (these aren't the longest we have in production, but kubernetes labelsets are fairly common in Prometheus so I thought it would be nice to have them). My benchmarks show this diff: goos: darwin goarch: arm64 pkg: github.com/prometheus/prometheus/model/labels │ old │ new │ │ sec/op │ sec/op vs base │ Labels_Compare/equal 5.898n ± 0% 5.875n ± 1% -0.40% (p=0.037 n=10) Labels_Compare/not_equal 11.78n ± 2% 11.01n ± 1% -6.54% (p=0.000 n=10) Labels_Compare/different_sizes 4.959n ± 1% 4.906n ± 2% -1.05% (p=0.050 n=10) Labels_Compare/lots 21.32n ± 0% 17.54n ± 5% -17.75% (p=0.000 n=10) Labels_Compare/real_long_equal 15.06n ± 1% 14.92n ± 0% -0.93% (p=0.000 n=10) Labels_Compare/real_long_different_end 25.20n ± 0% 24.43n ± 0% -3.04% (p=0.000 n=10) geomean 11.86n 11.25n -5.16% Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	8 months ago
Bryan Boreham	0bb5588386	labels: optimize String method (#13673 ) Use a stack buffer to reduce memory allocations. `Write(AppendQuote(AvailableBuffer` does not allocate or copy when the buffer has sufficient space. Also add a benchmark, with some refactoring. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Arthur Silva Sens	07355c9199	Bump client_golang to 1.19 Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	9 months ago
Ondrej Kokes	2e30f1231b	docs: textparse.Parser return type mismatch The docs suggest the Next method returns a bool, but that's not the case (`Entry` is an int). ``` // Next advances the parser to the next sample. It returns false if no // more samples were read or an error occurred. Next() (Entry, error) ``` The docs were first added in `d80a3de235` in 2017. Back then the signature was indeed `func (p *Parser) Next() bool`. But then it got refactored in `76a4a46cb0` and the signature changed with it, yet docs stayed the same - and eventually made their way into the `Parser` interface. However, the Protobuf parser does have the right wording: `5de2df752f` ``` // Next advances the parser to the next "sample" (emulating the behavior of a // text format parser). It returns (EntryInvalid, io.EOF) if no samples were // read. ``` Changing all other implementations (and the interface itself) to match this doc. Signed-off-by: Ondrej Kokes <ondrej.kokes@gmail.com>	9 months ago
machine424	f477e0539a	Move from golang.org/x/exp/slices into slices now that we only support Go >= 1.21 Prevent adding back golang.org/x/exp/slices. Signed-off-by: machine424 <ayoubmrini424@gmail.com>	9 months ago
Bryan Boreham	ba6aa95222	refactor: metrics parsing: extract common checking code Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	d2817e9c91	Merge pull request #12304 from bboreham/labels-symboltable Labels: reduce memory by de-duplicating strings in SymbolTables	9 months ago
Bryan Boreham	e1a741a0d7	labels: update copyright dates Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	55e7de04f8	model/labels (stringlabels): use strings.Clone Suggestion from @colega. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	d16ce3c9bd	model/labels (dedupelabels): small clarifications Suggestion from @colega. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	eff3a13e19	model/textparse: parsers take a labels SymbolTable This allows strings to be interned to save memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	93b72ec5dd	tsdb: create SymbolTables for labels as required Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
ismail simsek	2507469291	Fix: metadata API using wrong field names (#13633 ) Fix is to add json tags to `Metadata` struct. Absence of these tags causes Go to use the field name, which starts with an upper-case letter and breaks the protocol. Extend tests to verify the JSON response. Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>	9 months ago
Bryan Boreham	b39286fd1f	Add dedupelabels tag to not build regular labels Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	d51a5344cd	labels: new version de-duplicating strings in SymbolTables The individual strings for label names and values are held in a table, and each Labels value is a run of varint-encoded indexes into that table. When creating new labels, a sync.Mutex is locked around reads and writes. When reading labels, there is no locking because the table of strings used by those labels is immutable. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	28191109a8	Labels: add fake versions of SymbolTable apis So we can use them where necessary for internlabels implementation. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	d1af84f6ee	Labels: move Builder and Reset out of common New internstrings implementation is different. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	5aa4473894	labels tests: extend TestBuilder Start with empty base labels, also check new and re-used symbol tables Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	bb82a57e64	Labels: Call NewScratchBuilder in test_utils Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Bryan Boreham	cc5dc6a61b	labels: use Equal instead of DeepEqual This will work better with a different data structure. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	9 months ago
Łukasz Mierzwa	5597020a60	Use github.com/klauspost/compress for gzip and zlib klauspost/compress is a high quality drop-in replacement for common Go compression libraries. Since Prometheus sends out a lot of HTTP requests that often return compressed output having improved compression libraries helps to save cpu & memory resources. On a test Prometheus server I was able to see cpu reduction from 31 to 30 cores. Benchmark results: name old time/op new time/op delta TargetScraperGzip/metrics=1-8 69.4µs ± 4% 69.2µs ± 3% ~ (p=0.122 n=50+50) TargetScraperGzip/metrics=100-8 84.3µs ± 2% 80.9µs ± 2% -4.02% (p=0.000 n=48+46) TargetScraperGzip/metrics=1000-8 296µs ± 1% 274µs ±14% -7.35% (p=0.000 n=47+45) TargetScraperGzip/metrics=10000-8 2.06ms ± 1% 1.66ms ± 2% -19.34% (p=0.000 n=47+45) TargetScraperGzip/metrics=100000-8 20.9ms ± 2% 17.5ms ± 3% -16.50% (p=0.000 n=49+50) name old alloc/op new alloc/op delta TargetScraperGzip/metrics=1-8 6.06kB ± 0% 6.07kB ± 0% +0.24% (p=0.000 n=48+48) TargetScraperGzip/metrics=100-8 7.04kB ± 0% 6.89kB ± 0% -2.17% (p=0.000 n=49+50) TargetScraperGzip/metrics=1000-8 9.02kB ± 0% 8.35kB ± 1% -7.49% (p=0.000 n=50+50) TargetScraperGzip/metrics=10000-8 18.1kB ± 1% 16.1kB ± 2% -10.87% (p=0.000 n=47+47) TargetScraperGzip/metrics=100000-8 1.21MB ± 0% 1.01MB ± 2% -16.69% (p=0.000 n=36+50) name old allocs/op new allocs/op delta TargetScraperGzip/metrics=1-8 71.0 ± 0% 72.0 ± 0% +1.41% (p=0.000 n=50+50) TargetScraperGzip/metrics=100-8 81.0 ± 0% 76.0 ± 0% -6.17% (p=0.000 n=50+50) TargetScraperGzip/metrics=1000-8 92.0 ± 0% 83.0 ± 0% -9.78% (p=0.000 n=50+50) TargetScraperGzip/metrics=10000-8 93.0 ± 0% 91.0 ± 0% -2.15% (p=0.000 n=50+50) TargetScraperGzip/metrics=100000-8 111 ± 0% 135 ± 1% +21.89% (p=0.000 n=40+50) Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	9 months ago
Björn Rabenstein	ac10cd4d99	Merge pull request #13271 from ywwg/owilliams/utf8 UTF-8: Add partial support for parsing UTF-8 metric and label names	9 months ago
Owen Williams	a28d7865ad	UTF-8: Add support for parsing UTF8 metric and label names This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats. This grammar will also be valid for non-UTF-8 names. UTF-8 names will not be considered valid unless model.NameValidationScheme is changed. This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/. Part of https://github.com/prometheus/prometheus/issues/13095 Signed-off-by: Owen Williams <owen.williams@grafana.com>	9 months ago
Ziqi Zhao	a93859a52f	Prometheus support parse exemplars from native histogram (#13488 ) scrape: support parsing exemplars from native histogram --------- Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com> Signed-off-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Björn Rabenstein <github@rabenste.in>	9 months ago
Bryan Boreham	ff6c83269c	Merge pull request #13452 from bboreham/go-cmp Tests: Use DeepEqual replacement using go-cmp, which is more flexible	10 months ago
Bryan Boreham	17f48f2b3b	Tests: use replacement DeepEquals in more places Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	10 months ago
beorn7	553d92affd	model/labels: Fix new lint warning in test Signed-off-by: beorn7 <beorn@grafana.com>	10 months ago
Bryan Boreham	14b4fbc2ff	labels: make InternStrings a no-op for stringlabels version The current implementation of `InternStrings` will only save memory when the whole set of labels is identical to one already seen, and this cannot happen in the one place it is called from in Prometheus, remote-write, which already detects identical series. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	10 months ago
Marco Pracucci	501bc6419e	Add ShardedPostings() support to TSDB (#10421 ) This PR is a reference implementation of the proposal described in #10420. In addition to what described in #10420, in this PR I've introduced labels.StableHash(). The idea is to offer an hashing function which doesn't change over time, and that's used by query sharding in order to get a stable behaviour over time. The implementation of labels.StableHash() is the hashing function used by Prometheus before stringlabels, and what's used by Grafana Mimir for query sharding (because built before stringlabels was a thing). Follow up work As mentioned in #10420, if this PR is accepted I'm also open to upload another foundamental piece used by Grafana Mimir query sharding to accelerate the query execution: an optional, configurable and fast in-memory cache for the series hashes. Signed-off-by: Marco Pracucci <marco@pracucci.com>	10 months ago
Bryan Boreham	74b73d1e2c	Labels: Add DropMetricName function, used in PromQL (#13446 ) This function is called very frequently when executing PromQL functions, and we can do it much more efficiently inside Labels. In the common case that `__name__` comes first in the labels, we simply re-point to start at the next label, which is nearly free. `DropMetricName` is now so cheap I removed the cache - benchmarks show everything still goes faster. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	10 months ago
Marco Pracucci	bfec57bd2e	Further optimise FastRegexMatcher Signed-off-by: Marco Pracucci <marco@pracucci.com>	10 months ago
Filip Petkovski	583f3e587c	Optimize histogram iterators (#13340 ) Optimize histogram iterators Histogram iterators allocate new objects in the AtHistogram and AtFloatHistogram methods, which makes calculating rates over long ranges expensive. In #13215 we allowed an existing object to be reused when converting an integer histogram to a float histogram. This commit follows the same idea and allows injecting an existing object in the AtHistogram and AtFloatHistogram methods. When the injected value is nil, iterators allocate new histograms, otherwise they populate and return the injected object. The commit also adds a CopyTo method to Histogram and FloatHistogram which is used in the BufferedIterator to overwrite items in the ring instead of making new copies. Note that a specialized HPoint pool is needed for all of this to work (`matrixSelectorHPool`). --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	10 months ago
zenador	a3ddfbd1ee	Add warnings for histogramRate applied with isCounter not matching counter/gauge histogram (#13392 ) Add warnings for histogramRate applied with isCounter not matching counter/gauge histogram --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	10 months ago
tyltr	1fa131ee03	fix slice copy in 1.20 (#13389 ) The slices package is added to the standard library in Go 1.21; we need to import from the exp area to maintain compatibility with Go 1.20. Signed-off-by: tyltr <tylitianrui@126.com>	10 months ago
Björn Rabenstein	3db4596965	Merge pull request #13299 from vesari/add-unit-parser protobuf: add unit parser	11 months ago
Arianna Vespri	8f07f9dd90	Avoid creating string for suffix, consider counters without _total suffix Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	11 months ago
Bryan Boreham	4d44da2deb	Merge pull request #13176 from bboreham/move-metrictype Move metric type definitions to common/model	11 months ago
Arianna Vespri	9a664b515a	Correct order in error message Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	11 months ago
Arianna Vespri	51e78d9a32	Delete debugging lines, amend error message for unit Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>	11 months ago
Bryan Boreham	c83e1fc574	textparse: remove MetricType alias No backwards-compatibility; make a clean break. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago
Bryan Boreham	8065bef172	Move metric type definitions to common/model They are used in multiple repos, so common is a better place for them. Several packages now don't depend on `model/textparse`, e.g. `storage/remote`. Also remove `metadata` struct from `api.go`, since it was identical to a struct in the `metadata` package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago
Bryan Boreham	0289dd6157	relabel: blank replacement deletes label post-regexp If `cfg.TargetLabel` is a template like `$1`, it won't match any labels, so no point calling `lb.Del` with it. Similarly if `target` is not a valid label name, it won't match any labels, so don't call with that either. The intention seems to have been that a blank _value_ would delete the target, so change that code to use `target` instead of `cfg.TargetLabel`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago
Bryan Boreham	000182e4b8	relabel: check validity of all test cases Thought this would be a nice check on the `Validate()` function, but some of the test cases needed tweaking to pass. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago
Bryan Boreham	2d4c367d87	relabel: stricter check that target labels are valid For `Lowercase`, `KeepEqual`, etc., we do not expand a regexp, so the target label name must not contain anything like `${1}`. Also for the common case that the `Replace` target does not require any template expansion, check that the entire string passes label name validity rules. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago

1 2 3 4 5 ...

341 Commits (f07f4f293ff51ab64df23d99225b4cd6aba566ca)