prometheus

Commit Graph

Author	SHA1	Message	Date
Bartlomiej Plotka	76432aaf4e	Merge pull request #15220 from prometheus/nhcb-scrape-optimize perf(nhcb): scrape optimize	2 weeks ago
Matthieu MOREL	af1a19fc78	enable errorf rule from perfsprint linter Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	3 weeks ago
Bryan Boreham	6979b237b9	Merge branch 'release-2.55' into merge-2.55-into-main2	3 weeks ago
Ben Ye	b7aca45de7	fix round function ignoring enableDelayedNameRemoval feature flag Signed-off-by: Ben Ye <benye@amazon.com>	4 weeks ago
György Krajcsovits	eafe72a0d0	perf(nhcb): optimize away most allocations in convertnhcb In general aim for the happy case when the exposer lists the buckets in ascending order. Use Compact(2) to compact the result of nhcb convert. This is more in line with how client_golang optimizes spans vs buckets. `aef8aedb4b/prometheus/histogram.go (L1485)` Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	4 weeks ago
Jan Fajerski	38fd48e6b5	v2.55.0 -----BEGIN SSH SIGNATURE----- U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgX42TrpDUXJbbi9yZ3hs6cDg+kz G6d3nAlAb2XQInrEgAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5 AAAAQGoSEKIFT/BfavtG2qW9n7NYonNQk/9r6gCLvxln9elt1hiY0ZGcwRhm1QNx6FotxJ Y3LB9dt4s5akB3fOPkYwc= -----END SSH SIGNATURE----- Merge tag 'v2.55.0' into release-3.0.0-rc.0 v2.55.0	1 month ago
Björn Rabenstein	7c7116fea8	Merge pull request #15176 from jhesketh/jhesketh/round Round function should ignore native histograms	1 month ago
Alex Greenbank	421a3c22ea	scrape: provide a fallback format (#15136 ) scrape: Remove implicit fallback to the Prometheus text format Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config. --------- Signed-off-by: alexgreenbank <alex.greenbank@grafana.com> Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com> Co-authored-by: Björn Rabenstein <beorn@grafana.com>	1 month ago
Bryan Boreham	754c104a3e	Merge pull request #15173 from prometheus/merge-2.55-into-main-3 Merge release-2.55 into main (interim)	1 month ago
Joshua Hesketh	14ef1ce8ab	Round function should ignore native histograms As per the documentation, native histograms are skipped. This is in line with other simpleFunc's. Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	1 month ago
Bryan Boreham	a846bf9a5e	Merge branch 'release-2.55' into merge-2.55-into-main-3	1 month ago
Joshua Hesketh	5a4e4f6936	Fix stddev/stdvar when aggregating histograms, NaNs, and infinities (#14941 ) promql: Fix stddev/stdvar when aggregating histograms, NaNs, and Infs Native histograms are ignored when calculating stddev or stdvar. However, for the first series of each group, a `groupedAggregation` is always created. If the first series that was encountered is a histogram then it acts as the equivalent of a 0 point. This change creates the first `groupedAggregation` with the `seen` field set to `false` if the point is a histogram, thus ignoring it like the rest of the aggregation function does. A new `groupedAggregation` will then be created once an actual float value is encountered. This commit also sets the `floatValue` field of the `groupedAggregation` to `NaN`, if the first float value of a group is `NaN` or `±Inf`, so that the outcome is consistently `NaN` once those values are in the mix. (The added tests fail without this change). Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: beorn7 <beorn@grafana.com> --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org> Signed-off-by: beorn7 <beorn@grafana.com> Co-authored-by: beorn7 <beorn@grafana.com>	1 month ago
Arve Knudsen	de16f5e387	[FEATURE] PromQL: Add experimental info function MVP (#14495 ) The `info` function is an experiment to improve UX around including labels from info metrics. `info` has to be enabled via the feature flag `--enable-feature=promql-experimental-functions`. This MVP of info simplifies the implementation by assuming: * Only support for the target_info metric * That target_info's identifying labels are job and instance Also: * Encode info samples' original timestamp as sample value * Deduce info series select hints from top-most VectorSelector --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Ying WANG <ying.wang@grafana.com> Co-authored-by: Augustin Husson <augustin.husson@amadeus.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: Bryan Boreham <bjboreham@gmail.com>	1 month ago
Arve Knudsen	e05e97cdd7	evaluator.rangeEval: Split out gatherVector method Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 month ago
Arve Knudsen	f7b396a1dc	promql.Engine: Refactor vector selector evaluation into a method (#14900 ) New method is named `evalVectorSelector`. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 month ago
Neeraj Gartia	d4b1f9eb33	Corrects the behaviour of binary opperators between histogram and float (#14726 ) promql: corrects binary operators functioning for mixed sample with histogram and float For invalid pairings of sample types, an annotation is added now. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	1 month ago
George Krajcsovits	522149a2ae	model: move classic NHCB conversion into its own file (#15156 ) * model: move classic to NHCB conversion into its own file In preparation for #14978. Author: Jeanette Tan <jeanette.tan@grafana.com> 2024-07-03 11:56:48 Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Better naming from review comment Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Add doc strings. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>	1 month ago
Fiona Liao	8650d25804	Add additional basic nhcb unit tests (#15086 ) * Add additional basic nhcb unit tests * Update promql/promqltest/testdata/histograms.test Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 months ago
TJ Hoplock	6ebfbd2d54	chore!: adopt log/slog, remove go-kit/log For: #14355 This commit updates Prometheus to adopt stdlib's log/slog package in favor of go-kit/log. As part of converting to use slog, several other related changes are required to get prometheus working, including: - removed unused logging util func `RateLimit()` - forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger - move some of the json file logging functionality to use prom/common package functionality - refactored some of the new json file logging for scraping - changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers - updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition - added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>	2 months ago
Matthieu MOREL	ab64966e9d	fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094 ) * fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" --------- Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	2 months ago
Björn Rabenstein	199d2fb492	Merge pull request #14989 from NeerajGartia21/eval_info Adds eval_info command to PromQL testing framework	2 months ago
Björn Rabenstein	78f792135c	Merge pull request #15025 from prometheus/notreset-bug Fix bug in rate vs float and histogram mixup	2 months ago
György Krajcsovits	b5479831b8	Unit test for regression in rate vs float and histogram mixup Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 months ago
Manik Rana	f1c57a95ed	change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug (#14738 ) * chore: revert TypeRequiresCT to private Signed-off-by: Manik Rana <manikrana54@gmail.com> * feat: init NewOpenMetricsParser with skipCT true Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: allow opt-in to OM CT ingestion Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use textparse interface to set om options Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: set skipOMSeries in test Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * wip: add tests for OM CR parse Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: merge ct tests Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: add cases for OM text Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: check correct test cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: use both scrape protocols in config Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: fix inputs and output tests for OM Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: cleanup Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: rename skipOMSeries to skipOMCTSeries Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> * fix: finish refac Signed-off-by: Manik Rana <manikrana54@gmail.com> * refac: move setup code outside test Signed-off-by: Manik Rana <manikrana54@gmail.com> * tests: verify _created lines create new metric in certain cases Signed-off-by: Manik Rana <manikrana54@gmail.com> * fix: post merge fixes Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * manager: Fixed CT OMText conversion bug; Refactored tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * chore: lint Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: gofumpt Signed-off-by: Manik Rana <manikrana54@gmail.com> * chore: imports Signed-off-by: Manik Rana <manikrana54@gmail.com> --------- Signed-off-by: Manik Rana <manikrana54@gmail.com> Signed-off-by: Manik Rana <Manikrana54@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	2 months ago
Bryan Boreham	b8e5b7cda9	[REFACTOR] PromQL: remove label_join and label_replace stubs These functions operate on whole series, not on samples, so they do not fit into the table of functions that return a Vector. Remove the stub entries that were left to help downstream users of the code identify what changed. We cannot remove the entries from the `FunctionCalls` map without breaking `TestFunctionList`, so put some nils in to keep it happy. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 months ago
Neeraj Gartia	8aef821e10	eval_info command Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>	2 months ago
Bryan Boreham	7f99d2930d	[BUGFIX] PromQL: make sort_by_label stable Go's sorting functions can re-order equal elements, so the strategy of sorting by the fallback ordering first does not always work. Pulling the fallback into the main comparison function is more reliable and more efficient. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 months ago
Arve Knudsen	c2bbabb4a7	promql.Engine: Refactor vector selector evaluation into a method (#14900 ) * PromQL.Engine: Refactor Matrix expansion into a method Add utility method promql.evaluator.expandSeriesToMatrix, for expanding a slice of storage.Series into a promql.Matrix. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Rename to generateMatrix Rename evaluator.expandSeriesToMatrix into generateMatrix, while also dropping the start, end, interval arguments since they are evaluator fields. Write more extensive method documentation. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Rename to evalVectorSelector Rename to evalVectorSelector after discussing with @michahoffmann. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 months ago
Arthur Silva Sens	6bcb064d93	Merge pull request #14950 from Maniktherana/fuzz-om-minor-change chore: remove unused code	2 months ago
Manik Rana	3fba6bcdfa	chore: remove unused code Signed-off-by: Manik Rana <manikrana54@gmail.com>	2 months ago
Björn Rabenstein	5b9148e552	Merge pull request #14820 from charleskorn/promqltest-native-histogram-format promqltest: use test expression format for histograms in assertion failure messages and include reset hint in the test expression	2 months ago
Björn Rabenstein	df9916ef66	Merge pull request #14677 from prometheus/beorn7/histogram promql(native histograms): Introduce exponential interpolation	2 months ago
Jan Fajerski	96e5a94d29	promql: rename holt_winters to double_exponential_smoothing Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2 months ago
Björn Rabenstein	1639450172	Merge pull request #14821 from charleskorn/nh-negative-multiplication-division promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 months ago
beorn7	6fcd225aee	promql(native histograms): Introduce exponential interpolation The linear interpolation (assuming that observations are uniformly distributed within a bucket) is a solid and simple assumption in lack of any other information. However, the exponential bucketing used by standard schemas of native histograms has been chosen to cover the whole range of observations in a way that bucket populations are spread out over buckets in a reasonably way for typical distributions encountered in real-world scenarios. This is the origin of the idea implemented here: If we divide a given bucket into two (or more) smaller exponential buckets, we "most naturally" expect that the samples in the original buckets will split among those smaller buckets in a more or less uniform fashion. With this assumption, we end up with an "exponential interpolation", which therefore appears to be a better match for histograms with exponential bucketing. This commit leaves the linear interpolation in place for NHCB, but changes the interpolation for exponential native histograms to exponential. This affects `histogram_quantile` and `histogram_fraction` (because the latter is more or less the inverse of the former). The zero bucket has to be treated specially because the assumption above would lead to an "interpolation to zero" (the bucket density approaches infinity around zero, and with the postulated uniform usage of buckets, we would end up with an estimate of zero for all quantiles ending up in the zero bucket). We simply fall back to linear interpolation within the zero bucket. At the same time, this commit makes the call to stick with the assumption that the zero bucket only contains positive observations for native histograms without negative buckets (and vice versa). (This is an assumption relevant for interpolation. It is a mostly academic point, as the zero bucket is supposed to be very small anyway. However, in cases where it _is_ relevantly broad, the assumption helps a lot in practice.) This commit also updates and completes the documentation to match both details about interpolation. As a more high level note: The approach here attempts to strike a balance between a more simplistic approach without any assumption, and a more involved approach with more sophisticated assumptions. I will shortly describe both for reference: The "zero assumption" approach would be to not interpolate at all, but _always_ return the harmonic mean of the bucket boundaries of the bucket the quantile ends up in. This has the advantage of minimizing the maximum possible relative error of the quantile estimation. (Depending on the exact definition of the relative error of an estimation, there is also an argument to return the arithmetic mean of the bucket boundaries.) While limiting the maximum possible relative error is a good property, this approach would throw away the information if a quantile is closer to the upper or lower end of the population within a bucket. This can be valuable trending information in a dashboard. With any kind of interpolation, the maximum possible error of a quantile estimation increases to the full width of a bucket (i.e. it more than doubles for the harmonic mean approach, and precisely doubles for the arithmetic mean approach). However, in return the _expectation value_ of the error decreases. The increase of the theoretical maximum only has practical relevance for pathologic distributions. For example, if there are thousand observations within a bucket, they could _all_ be at the upper bound of the bucket. If the quantile calculation picks the 1st observation in the bucket as the relevant one, an interpolation will yield a value close to the lower bucket boundary, while the true quantile value is close to the upper boundary. The "fancy interpolation" approach would be one that analyses the _actual_ distribution of samples in the histogram. A lot of statistics could be applied based on the information we have available in the histogram. This would include the population of neighboring (or even all) buckets in the histogram. In general, the resolution of a native histogram should be quite high, and therefore, those "fancy" approaches would increase the computational cost quite a bit with very little practical benefits (i.e. just tiny corrections of the estimated quantile value). The results are also much harder to reason with. Signed-off-by: beorn7 <beorn@grafana.com>	2 months ago
Jan Fajerski	15cea39136	promql: put holt_winters behind experimental feature flag Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	2 months ago
Joshua Hesketh	b6107cc888	Make rate possible non-counter annotation consistent (#14910 ) * Make rate possible non-counter annotation consistent Previously a PossibleNonCounterInfo annotation would be left in cases where a range-vector selects 1 float data point, even if no more points are selected in order to calculate a rate. This change ensures an output float exists before emitting such an annotation. This fixes an inconsistency where a series with mixed data (ie, a float and a native histogram) would emit an annotation without any points. For example, ``` load 1m series{label="a"} 1 {{schema:1 sum:10 count:5 buckets:[1 2 3]}} eval instant at 1m rate(series[1m1s]) ``` Would have a PossibleNonCounterInfo annotation. Wheras ``` load 1m series{label="a"} {{schema:1 sum:10 count:5 buckets:[1 2 3]}} {{schema:1 sum:15 count:10 buckets:[1 2 3]}} eval instant at 1m rate(series[1m1s]) ``` Would not. --------- Signed-off-by: Joshua Hesketh <josh@nitrotech.org>	2 months ago
Mario Fernandez	5814920601	Fix: optimize .* regexp performance Shortcut for `.*` matches newlines as well. Add preamble change ^(?s: Add test dotAll flag por al regex Add and fix regex tests Signed-off-by: Mario Fernandez <mariofer@redhat.com>	2 months ago
Nathan Baulch	50cd453c8f	chore: Fix typos (#14868 ) * Fix typos --------- Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>	3 months ago
Jan Fajerski	91608c002f	Merge branch 'main' into release-3.0-beta.0 Conflicts: scrape/scrape_test.go Pick both changes.	3 months ago
Björn Rabenstein	8114f9a281	Merge pull request #14821 from charleskorn/nh-negative-multiplication-division promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars	3 months ago
Jan Fajerski	fa318711f4	Merge branch 'main' into 3.0-main-sync-24-09-09 Conflicts: cmd/prometheus/main.go docs/command-line/prometheus.md docs/feature_flags.md web/ui/build_ui.sh web/web.go Resolved by dropping the UTF-8 feature flag and adding the `auto-reload-config` feature flag. For the new web ui pick all changes from `main`.	3 months ago
Charles Korn	e8c7482137	Return negative counts when multiplied or divided by a negative value Signed-off-by: Charles Korn <charles.korn@grafana.com>	3 months ago
Charles Korn	2bdb3452d1	Modify parser for native histograms to allow negative values where supported Signed-off-by: Charles Korn <charles.korn@grafana.com>	3 months ago
Charles Korn	9852855084	Implement unary negation for native histograms Signed-off-by: Charles Korn <charles.korn@grafana.com>	3 months ago
Arve Knudsen	db5e48dc33	promql.Engine.Close: No-op if nil (#14861 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 months ago
Björn Rabenstein	d2b659540e	Merge pull request #14819 from prometheus/beorn7/doc promql: Improve the rate extrapolation explanation	3 months ago
Jan Fajerski	befcfadf78	Fix merge conflicts Fix call to newTestEngine(t) in promql/engine_test.go:3214. `agent` feature-flag it's own cmdline flag now. Remove `scrape.name-escaping-scheme` argument. Signed-off-by: Jan Fajerski <jfajersk@redhat.com>	3 months ago
Jan Fajerski	fe4289b502	Merge branch 'main' into HEAD	3 months ago
Bryan Boreham	485523eed2	Merge pull request #14816 from bboreham/improve-promql-tracing Improve promql tracing	3 months ago

1 2 3 4 5 ...

1146 Commits (140f4aa9aed6674c582dc603431a4aaa0c702cb7)