Commit Graph

872 Commits (89ecb3a3f2e48a66260a9af1a834ed8617b86f27)

Author SHA1 Message Date
Bryan Boreham ff6c83269c
Merge pull request #13452 from bboreham/go-cmp
Tests: Use DeepEqual replacement using go-cmp, which is more flexible
2024-02-12 15:40:08 +01:00
Bryan Boreham 17f48f2b3b Tests: use replacement DeepEquals in more places
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-08 19:32:33 +00:00
Bryan Boreham 39af788dbd Tests: use replacement DeepEquals using go-cmp
Use DeepEqual replacement using go-cmp, which is more flexible.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-08 19:30:20 +00:00
beorn7 86d7618d84 promql: Fix wrongly scoped range vectors
Fixes #11708.

If a range vector is fixen in time with the @ modifier, it gets still
moved around for different steps in a range query. Since no additional
points are retrieved from the TSDB, this leads to steadily emptying
the range, leading to the weird behavior described in isse #11708.

This only happens for functions listed in `AtModifierUnsafeFunctions`,
and the only of those that takes a range vector is `predict_linear`,
which is the reason why we see it only for this particular function.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-02-07 18:07:51 +01:00
beorn7 384ab025e0 promql: Expose issue #11708
The shorter range in the test makes early points drop out of the range
in range queries, exposing the issue.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-02-07 18:06:05 +01:00
Bryan Boreham 98c4889029
Merge pull request #9298 from Creatone/creatone/use-testify
tests: Move from t.Errorf and others.
2024-02-04 16:27:57 +01:00
Faustas Butkus 6feffeb92e
promql: add histogram_avg function (#13467)
Add histogram_avg function

---------

Signed-off-by: Faustas Butkus <faustas.butkus@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2024-02-01 18:28:42 +01:00
Alan Protasio c006c57efc
Proposal to improve FPointSlice and HPointSlice allocation. (#13448)
* Reusing points slice from previous series when the slice is under utilized
* Adding comments on the bench test

Signed-off-by: Alan Protasio <alanprot@gmail.com>
2024-02-01 16:22:38 +00:00
Paweł Szulik 1a47c7d59b Refactor lexer tests to use testify.
Signed-off-by: Paweł Szulik <paul.szulik@gmail.com>
2024-02-01 13:51:31 +00:00
Filip Petkovski a577a0a542
Fix last_over_time for native histograms
The last_over_time retains a histogram sample without making a copy.
This sample is now coming from the buffered iterator used for windowing functions,
and can be reused for reading subsequent samples as the iterator progresses.

I would propose copying the sample in the last_over_time function, similar to
how it is done for rate, sum_over_time and others.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2024-01-26 15:02:40 +01:00
Bryan Boreham 74b73d1e2c
Labels: Add DropMetricName function, used in PromQL (#13446)
This function is called very frequently when executing PromQL functions,
and we can do it much more efficiently inside Labels.

In the common case that `__name__` comes first in the labels, we simply
re-point to start at the next label, which is nearly free.

`DropMetricName` is now so cheap I removed the cache - benchmarks show
everything still goes faster.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-01-25 11:48:49 +01:00
Filip Petkovski 583f3e587c
Optimize histogram iterators (#13340)
Optimize histogram iterators

Histogram iterators allocate new objects in the AtHistogram and
AtFloatHistogram methods, which makes calculating rates over long
ranges expensive.

In #13215 we allowed an existing object to be reused
when converting an integer histogram to a float histogram. This commit follows
the same idea and allows injecting an existing object in the AtHistogram and
AtFloatHistogram methods. When the injected value is nil, iterators allocate
new histograms, otherwise they populate and return the injected object.

The commit also adds a CopyTo method to Histogram and FloatHistogram which
is used in the BufferedIterator to overwrite items in the ring instead of making
new copies.

Note that a specialized HPoint pool is needed for all of this to work 
(`matrixSelectorHPool`).

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-01-23 17:02:14 +01:00
Ben Ye d778591fd3 add more context cancellation check at evaluation time
Signed-off-by: Ben Ye <benye@amazon.com>
2024-01-21 14:19:39 -08:00
Björn Rabenstein bfbb13cf36
Merge pull request #13267 from linasm/simplify-native-histogram-math
promql: simplify Native Histogram arithmetics
2024-01-18 13:50:59 +01:00
Julien Pivotto 4f941bbf69
Merge pull request #13416 from tylitianrui/feat/remove_obsolete_build_tag
remove  obsolete build tag
2024-01-17 18:12:21 +01:00
zenador a3ddfbd1ee
Add warnings for histogramRate applied with isCounter not matching counter/gauge histogram (#13392)
Add warnings for histogramRate applied with isCounter not matching counter/gauge histogram

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-01-17 17:06:35 +01:00
tyltr f97fa2736c remove obsolete build tag
Signed-off-by: tyltr <tylitianrui@126.com>
2024-01-17 22:26:32 +08:00
Ivan Babrou a6b35ff304
promql: use natural sort in sort_by_label and sort_by_label_desc (#13411)
These functions are intended for humans, as robots can already sort the results
however they please. Humans like things sorted "naturally":

* https://blog.codinghorror.com/sorting-for-humans-natural-sort-order/

A similar thing has been done to Grafana, which is also used by humans:

* https://github.com/grafana/grafana/pull/78024
* https://github.com/grafana/grafana/pull/78494

Signed-off-by: Ivan Babrou <github@ivan.computer>
2024-01-16 21:34:09 -03:00
zenador 72a8f1084b
Restore more efficient version of NewPossibleNonCounterInfo annotation (#13022)
Restore more efficient version of NewPossibleNonCounterInfo annotation

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-01-16 09:54:16 +01:00
Ayoub Mrini ace9c8a3da
promtool: allow setting multiple matchers to "promtool tsdb dump" command. (#13296)
Conditions are ANDed inside the same matcher but matchers are ORed

Including unit tests for "promtool tsdb dump".

Refactor some matchers scraping utils.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-01-15 10:29:53 +00:00
Bryan Boreham 252031c86f Revert "Adding small test update for temp dir using t.TempDir (#13293)"
This reverts commit 2ddb3596ef.

Various tests are failing in CI after this change; reverting to free up
other work.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-30 19:17:30 +00:00
Mile Druzijanic 2ddb3596ef
Adding small test update for temp dir using t.TempDir (#13293)
* Adding small test update for temp dir using t.TempDir

Signed-off-by: Mile Druzijanic <miledruz@gmail.com>
Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>

* removing not required cleanup

Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>

---------

Signed-off-by: Mile Druzijanic <miledruz@gmail.com>
Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>
2023-12-28 21:49:57 +01:00
Filip Petkovski 0e1ae1d1ca
Add comment
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-25 11:41:07 +01:00
Filip Petkovski 35f9620cd1
Expand benchmark
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-25 11:30:29 +01:00
Filip Petkovski 5df3820c7a
Copy last histogram point
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-25 11:20:51 +01:00
Filip Petkovski 1f69dcfa6b
Fix reusing float histograms
In https://github.com/prometheus/prometheus/pull/13276 we started reusing float histogram objects to reduce allocations in PromQL.
That PR introduces a bug where histogram pointers gets copied to the beginning of the histograms slice,
but are still kept in the end of the slice. When a new histogram is read into the last element,
it can overwrite a previous element because the pointer is the same.

This commit fixes the issue by moving outdated points to the end of the slice
so that we don't end up with duplicate pointers in the same buffer. In other words,
the slice gets rotated so that old objects can get reused.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-14 11:53:58 +01:00
Filip Petkovski bb8363dbb3
Add comment on SampleRingIterator
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-13 08:30:02 +01:00
Filip Petkovski e2a9f8ac0f
Reuse float histogram objects
This commit reduces the memory needed to query native histogram objects
by reusing existing HPoint instances.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-11 08:24:58 +01:00
Björn Rabenstein db915b07cb
Merge pull request #13215 from fpetkovski/float-histogram-reuse
Enable reusing memory when converting between histogram types
2023-12-09 22:44:46 +01:00
Bartlomiej Plotka 91a383f52c
Merge pull request #13059 from zenador/add-mad-function
Add mad_over_time function
2023-12-08 11:53:22 +00:00
Filip Petkovski 10a82f87fd
Enable reusing memory when converting between histogram types
The 'ToFloat' method on integer histograms currently allocates new memory
each time it is called.

This commit adds an optional *FloatHistogram parameter that can be used
to reuse span and bucket slices. It is up to the caller to make sure the
input float histogram is not used anymore after the call.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-08 10:22:59 +01:00
Linas Medziunas 7319ad6a0b promql: simplify Native Histogram arithmetics
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-12-08 10:59:00 +02:00
Matthieu MOREL 9c4782f1cc
golangci-lint: enable testifylint linter (#13254)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
Jeanette Tan 2910b48180 Make mad_over_time experimental and move tests
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-12-01 01:56:07 +08:00
Jeanette Tan 9bf4cc993e Add mad_over_time function
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-12-01 01:22:58 +08:00
Björn Rabenstein 5dbbadf598
Merge pull request #13216 from prometheus/beorn7/doc
Update “conventional histogram” → “classic histogram”
2023-11-30 10:35:27 +01:00
Oleksandr Redko 2a75604f8e
Enable default revive rules (#13068)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-11-29 17:23:34 +00:00
beorn7 0eb0ca42c5 Update “conventional histogram” → “classic histogram”
Signed-off-by: beorn7 <beorn@grafana.com>
2023-11-29 15:22:58 +01:00
Julien Pivotto c1ec6ae851 sort_by_label: Switch to feature flag
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-28 15:10:12 +01:00
Alexander Trost 5051a993ab promql: add sort_by_label and sort_by_label_desc functions
This adds functions to sort a vector by its label value.

Based on https://github.com/prometheus/prometheus/pull/1533

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2023-11-28 14:40:07 +01:00
zenador ccfe14d7e7
PromQL: ignore small errors for bucketQuantile (#13153)
promql: Improve histogram_quantile calculation for classic buckets

Tiny differences between classic buckets are most likely caused by floating point precision issues. With this commit, relative changes below a certain threshold are ignored. This makes the result of histogram_quantile more meaningful, and also avoids triggering the _input to histogram_quantile needed to be fixed for monotonicity_ annotations in unactionable cases.

This commit also adds explanation of the new adjustment and of the monotonicity annotation to the documentation of `histogram_quantile`.

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-25 00:05:38 +01:00
Filip Petkovski 35a15e8f04
Add benchmark for native histograms (#13160)
* Add benchmark for native histograms

This commit adds a PromQL benchmark for queries on native histograms.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-11-23 14:09:17 +00:00
Julien Pivotto c92fbf3fdf Add feature flag for PromQL experimental functions.
This PR adds an Experimental flag to the functions.

This can be used by https://github.com/prometheus/prometheus/pull/13059
but also xrate and other future functions.

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-11-14 17:48:58 +01:00
Linas Medziunas 1cd6c1cde5 ValidateHistogram: strict Count check in absence of NaNs
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:17:24 +02:00
Dimitar Dimitrov 9e3df532d8
Export `promql.Engine.FindMinMaxTime`
This function is useful to analyze promQL queries. We want to use this in Mimir to record the time range which the query touches.

I also chose to remove the `Engine` receiver because it was unnecessary, and it makes it easier to use, but happy to refactor that if you disagree.

The function is untested on its own. If you prefer to have unit tests now that its exported, I can look into adding some.

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
2023-11-02 13:17:35 +01:00
Björn Rabenstein a43669e611
Merge pull request #12928 from alexandear/ci-enable-godot
ci(lint): enable godot; append dot at the end of comments
2023-11-01 17:15:41 +01:00
Julien Pivotto f568221610
Merge pull request #13057 from prometheus/release-2.48
Merge release-2.48 back into main
2023-10-31 15:24:39 -04:00
Oleksandr Redko fa90ca46e5 ci(lint): enable godot; append dot at the end of comments
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 19:53:38 +02:00
Oleksandr Redko 8e5f0387a2
ci(lint): enable nolintlint and remove redundant comments (#12926)
Signed-off-by: Oleksandr Redko <Oleksandr_Redko@epam.com>
2023-10-31 12:35:13 +01:00
Bryan Boreham 49c5e7afe1 PromQL: reduce garbage in range-query evaluation
The temporary variable was allocated on the heap, and it is unnecessary.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-10-29 19:45:06 +00:00