Commit Graph

1109 Commits (15cea39136bbdbcae3016b84bd646a3220bd7580)

Author SHA1 Message Date
Jan Fajerski 15cea39136 promql: put holt_winters behind experimental feature flag
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-09-18 15:39:58 +02:00
Joshua Hesketh b6107cc888
Make rate possible non-counter annotation consistent (#14910)
* Make rate possible non-counter annotation consistent

Previously a PossibleNonCounterInfo annotation would be left in cases
where a range-vector selects 1 float data point, even if no more points
are selected in order to calculate a rate.

This change ensures an output float exists before emitting such an
annotation.

This fixes an inconsistency where a series with mixed data (ie, a float
and a native histogram) would emit an annotation without any points.

For example,

```

load 1m
series{label="a"} 1 {{schema:1 sum:10 count:5 buckets:[1 2 3]}}

eval instant at 1m rate(series[1m1s])

```

Would have a PossibleNonCounterInfo annotation.

Wheras

```

load 1m
series{label="a"} {{schema:1 sum:10 count:5 buckets:[1 2 3]}} {{schema:1 sum:15 count:10 buckets:[1 2 3]}}

eval instant at 1m rate(series[1m1s])

```

Would not. 

---------

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
2024-09-18 10:21:25 +00:00
Mario Fernandez 5814920601
Fix: optimize .* regexp performance
Shortcut for `.*` matches newlines as well.
Add preamble change ^(?s:
Add test
dotAll flag por al regex
Add and fix regex tests

Signed-off-by: Mario Fernandez <mariofer@redhat.com>
2024-09-17 12:18:31 +02:00
Nathan Baulch 50cd453c8f
chore: Fix typos (#14868)
* Fix typos

---------

Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-10 22:32:03 +02:00
Jan Fajerski 91608c002f Merge branch 'main' into release-3.0-beta.0
Conflicts:
	scrape/scrape_test.go
          Pick both changes.
2024-09-10 20:51:20 +02:00
Björn Rabenstein 8114f9a281
Merge pull request #14821 from charleskorn/nh-negative-multiplication-division
promql: correctly handle unary negation of native histograms and add tests for multiplication and division of native histograms by negative scalars
2024-09-10 15:23:00 +02:00
Jan Fajerski fa318711f4 Merge branch 'main' into 3.0-main-sync-24-09-09
Conflicts:
	cmd/prometheus/main.go
	docs/command-line/prometheus.md
	docs/feature_flags.md
	web/ui/build_ui.sh
	web/web.go
    Resolved by dropping the UTF-8 feature flag and adding the
    `auto-reload-config` feature flag.
    For the new web ui pick all changes from `main`.
2024-09-09 15:44:22 +02:00
Charles Korn e8c7482137
Return negative counts when multiplied or divided by a negative value
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-09-09 14:37:59 +10:00
Charles Korn 2bdb3452d1
Modify parser for native histograms to allow negative values where supported
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-09-09 14:37:58 +10:00
Charles Korn 9852855084
Implement unary negation for native histograms
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-09-09 14:37:55 +10:00
Arve Knudsen db5e48dc33
promql.Engine.Close: No-op if nil (#14861)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-09-08 14:39:13 +02:00
Björn Rabenstein d2b659540e
Merge pull request #14819 from prometheus/beorn7/doc
promql: Improve the rate extrapolation explanation
2024-09-05 12:21:03 +02:00
Jan Fajerski befcfadf78 Fix merge conflicts
Fix call to newTestEngine(t) in promql/engine_test.go:3214.
`agent` feature-flag it's own cmdline flag now.
Remove `scrape.name-escaping-scheme` argument.

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-09-04 20:06:17 +02:00
Jan Fajerski fe4289b502 Merge branch 'main' into HEAD 2024-09-04 18:50:00 +02:00
Bryan Boreham 485523eed2
Merge pull request #14816 from bboreham/improve-promql-tracing
Improve promql tracing
2024-09-04 14:32:22 +01:00
Arve Knudsen 6ef1eb4e68
Merge pull request #14064 from aknuds1/arve/close-engine
Release PromQL engine resources on close, from tests
2024-09-04 12:07:35 +02:00
Charles Korn 113de6301c
Add failing test cases for unary negation and multiplication and division with negative scalars
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-09-04 16:20:28 +10:00
Charles Korn 9b451abec7
Make positive and negative bucket counts different in existing test cases
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-09-04 16:08:05 +10:00
beorn7 2304be9dd5 promql: Improve the rate extrapolation explanation
Signed-off-by: beorn7 <beorn@grafana.com>
2024-09-03 22:11:14 +02:00
Bryan Boreham abb0502685 [ENHANCEMENT] PromQL: Add detail to tracing spans
For aggregates, operators, calls, show what operation is performed.
Also add an event when series are expanded, typically time spent
accessing TSDB.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-03 14:15:58 +01:00
Bryan Boreham 54989ce6fd PromQL: add short string method
To be used in tracing or logging.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-03 14:15:58 +01:00
Bryan Boreham 8742077498 [BUGFIX] PromQL: pass Context so spans parent correctly
Assigning to `evaluator.ctx` in `eval()` broke the parent-child relationship.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-03 14:15:29 +01:00
Arve Knudsen 70bb219d33
Merge pull request #14680 from charleskorn/promqltest-scalar
promqltest: improve error messages when unexpected type is returned, and fix inverted expected and actual values in error message
2024-09-03 10:38:04 +02:00
Arve Knudsen 5dfbcc390e Merge remote-tracking branch 'prometheus/main' into arve/close-engine
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-09-02 16:26:59 +02:00
Jan Fajerski 956245b25b promqltest: adjust eval times and range selector
In order to fix new tests for changes added in
https://github.com/prometheus/prometheus/pull/13904.

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-09-02 11:27:39 +02:00
Jan Fajerski 00315ce15e Merge branch 'main' into 3.0-main-sync-24-08-30
using -Xours

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-09-02 11:27:18 +02:00
Neeraj Gartia 8c7bf39d96
Moves TestNativeHistogram_MulDivOperator to promql testing framework (#14688)
PromQL: add test for mul and div operator

Also, remove the converted test from the engine_test.go file.

This also includes an extension of the test framework to allow NaN/Inf in histogram buckets.

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-08-29 16:42:35 +02:00
Jorge Creixell e9e3d64b7c
PromQL engine: Delay deletion of __name__ label to the end of the query evaluation (#14477)
PromQL engine: Delay deletion of __name__ label to the end of the query evaluation

  - This change allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the dreaded "vector cannot contain metrics with the same labelset" error.
  - The implementation extends the `Series` and `Sample` structs with a boolean flag indicating whether the `__name__` label should be deleted at the end of the query evaluation.
  - The `label_replace` and `label_join` functions can still access the value of the `__name__` label, even if it has been previously marked for deletion. If  `__name__` is used as target label, it won't be dropped at the end of the query evaluation.
  - Fixes https://github.com/prometheus/prometheus/issues/11397
  - See https://github.com/jcreixell/prometheus/pull/2 for previous discussion, including the decision to create this PR and benchmark it before considering other alternatives (like refactoring `labels.Labels`).
  - See https://github.com/jcreixell/prometheus/pull/1 for an alternative implementation using a special label instead of boolean flags.
  - Note: a feature flag  `promql-delayed-name-removal` has been added as it changes the behavior of some "weird" queries (see https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792)

Example (this always fails, as `__name__` is being dropped by `count_over_time`):

```
count_over_time({__name__!=""}[1m])

=> Error executing query: vector cannot contain metrics with the same labelset
```

Before:

```
label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)")

=> Error executing query: vector cannot contain metrics with the same labelset
```

After:

```
label_replace(count_over_time({__name__!=""}[1m]), "__name__", "count_$1", "__name__", "(.+)")

=>
count_go_gc_cycles_automatic_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1
count_go_gc_cycles_forced_gc_cycles_total{instance="localhost:9090", job="prometheus"} 1
...
```

Signed-off-by: Jorge Creixell <jcreixell@gmail.com>

---------

Signed-off-by: Jorge Creixell <jcreixell@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
2024-08-29 15:50:39 +02:00
Arve Knudsen 99204f23ee Merge remote-tracking branch 'prometheus/main' into arve/close-engine
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-29 09:52:54 +02:00
Björn Rabenstein 849215d90c
Merge pull request #14585 from fatsheep9146/covert-TestNativeHistogram_Sum_Count_Add_AvgOperator-to-framework
convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework
2024-08-28 17:21:32 +02:00
Owen Williams 9da75328ea
fix(utf8): ensure correct validation when legacy mode turned on (#14736)
fix(utf8): ensure correct validation when legacy mode turned on

This depends on the included update of the prometheus/common dependency.

---------

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-08-28 17:15:42 +02:00
Arve Knudsen c9a460d570 Merge remote-tracking branch 'prometheus/main' into arve/close-engine
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-26 12:17:10 +02:00
Ziqi Zhao 87eab0aaad fix golang lint
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-08-23 09:32:04 +08:00
Ziqi Zhao c7c4a5c347 add helper function to compare native histograms in testing
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-08-23 09:28:31 +08:00
beorn7 0f760f63dd lint: Revamp our linting rules, mostly around doc comments
Several things done here:

- Set `max-issues-per-linter` to 0 so that we actually see all linter
  warnings and not just 50 per linter. (As we also set
  `max-same-issues` to 0, I assume this was the intention from the
  beginning.)

- Stop using the golangci-lint default excludes (by setting
  `exclude-use-default: false`. Those are too generous and don't match
  our style conventions. (I have re-added some of the excludes
  explicitly in this commit. See below.)

- Re-add the `errcheck` exclusion we have used so far via the
  defaults.

- Exclude the signature requirement `govet` has for `Seek` methods
  because we use non-standard `Seek` methods a lot. (But we keep other
  requirements, while the default excludes completely disabled the
  check for common method segnatures.)

- Exclude warnings about missing doc comments on exported symbols. (We
  used to be pretty adamant about doc comments, but stopped that at
  some point in the past. By now, we have about 500 missing doc
  comments. We may consider reintroducing this check, but that's
  outside of the scope of this commit. The default excludes of
  golangci-lint essentially ignore doc comments completely.)

- By stop using the default excludes, we now get warnings back on
  malformed doc comments. That's the most impactful change in this
  commit. It does not enforce doc comments (again), but _if_ there is
  a doc comment, it has to have the recommended form. (Most of the
  changes in this commit are fixing this form.)

- Improve wording/spelling of some comments in .golangci.yml, and
  remove an outdated comment.

- Leave `package-comments` inactive, but add a TODO asking if we
  should change that.

- Add a new sub-linter `comment-spacings` (and fix corresponding
  comments), which avoids missing spaces after the leading `//`.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-08-22 17:36:11 +02:00
Jan Fajerski 7c8c748399 promql tests: adjust range query intervals
Some test queries need their interval adjusted to account for
https://github.com/prometheus/prometheus/pull/13904. Otherwise the
queries don't return enough samples.
promql/engine_test.go:TestHistogramCopyFromIteratorRegression needed the
same, but also the result needed a fix since `increase` interpolates
over the full range.

Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
2024-08-21 12:33:52 +02:00
Björn Rabenstein 7fad1ec8ee
Merge pull request #14655 from suntala/suntala/sort-by-label-enhancement
promql: Fall back to full label sets when sorting by label
2024-08-21 12:28:55 +02:00
Jan Fajerski 5138922b0d Merge branch 'main' into 3.0-main-sync-24-08-21 2024-08-21 09:09:36 +02:00
Ziqi Zhao 8f828d45c1 convert TestNativeHistogram_Sum_Count_Add_AvgOperator into testing framework
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
2024-08-21 09:24:50 +08:00
suntala 8e82ac8d2b Add comments to the sort by label functions
Signed-off-by: suntala <arati.rana@grafana.com>
2024-08-20 20:41:55 +02:00
Arve Knudsen 3a78e76282 Upgrade golangci-lint to v1.60.1
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-18 12:13:25 +02:00
Charles Korn 6548421f61
promqltest: improve error messages when unexpected type is returned, and fix inverted expected and actual values in error message when scalar value does not match expected
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-08-16 13:37:05 +10:00
Björn Rabenstein 1daf7cdd62
Merge pull request #14626 from cuiweiyuan/main
chore: fix some function names
2024-08-15 11:46:21 +02:00
cuiweiyuan 1800af54f0 chore: fix some function names
Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>
2024-08-15 13:57:21 +08:00
Charles Korn 52818a97e2
Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms
# Conflicts:
#	promql/promqltest/testdata/native_histograms.test
2024-08-14 07:52:08 +10:00
Björn Rabenstein c2bc6cfe97
Merge pull request #14621 from charleskorn/panic-message
promql: clarify error message logged when panic occurs during query evaluation
2024-08-13 23:02:43 +02:00
George Krajcsovits fe5354df88
Merge pull request #14658 from krajorama/zero-rate-nh-quantile
Native histograms: define behavior when rate is null.
2024-08-13 16:05:43 +02:00
György Krajcsovits 386fc8b9f6 Update from review comments.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-13 15:26:07 +02:00
Arve Knudsen 0503d4f372 PromQL: Fix comment regarding non-nil histogram pointer
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-08-13 08:55:24 +02:00
György Krajcsovits 6aee5b4b38 fix typo
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-08-12 12:04:45 +02:00