Arve Knudsen
fbc9eddfaf
Refactor engine creation in tests
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-14 13:58:51 +02:00
Arve Knudsen
fec6adadcd
Merge remote-tracking branch 'prometheus/main' into arve/close-engine
2024-07-14 13:19:11 +02:00
Bryan Boreham
d116bf7b9f
Merge pull request #14109 from harry671003/pass_limit_to_querier
...
storage: pass limit param as hint in querier
2024-07-12 10:27:52 +01:00
JuanJo Ciarlante
c94c5b64c3
feat: add limitk() and limit_ratio() operators ( #12503 )
...
* rebase 2024-07-01, picks previous renaming to `limitk()` and `limit_ratio()`
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* gofumpt -d -extra
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* more lint fixes
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* more lint fixes+
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* put limitk() and limit_ratio() behind --enable-feature=promql-experimental-functions
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* EnableExperimentalFunctions for TestConcurrentRangeQueries() also
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* use testutil.RequireEqual to fix tests, WIP equivalent thingie for require.Contains
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* lint fix
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* moar linting
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* rebase 2024-06-19
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* re-add limit(2, metric) testing for N=2 common series subset
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* move `ratio = param` to default switch case, for better readability
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* gofumpt -d -extra util/testutil/cmp.go
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* early break when reaching k elems in limitk(), should have always been so (!)
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* small typo fix
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* no-change small break-loop rearrange for readability
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* remove IsNan(ratio) condition in switch-case, already handled as input validation
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* no-change adding some comments
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* no-change simplify fullMatrix() helper functions used for tests
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* add `limitk(-1, metric)` testcase, which is handled as any k < 1 case
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* engine_test.go: no-change create `requireCommonSeries() helper func (moving code into it) for readability
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* rebase 2024-06-21
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* engine_test.go: HAPPY NOW about its code -> reorg, create and use simpleRangeQuery() function, less lines and more readable ftW \o/
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* move limitk(), limit_ratio() testing to promql/promqltest/testdata/limit.test
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* remove stale leftover after moving tests from engine_test.go to testdata/
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* fix flaky `limit_ratio(0.5, ...)` test case
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* Update promql/engine.go
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* Update promql/engine.go
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* Update promql/engine.go
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* fix AddRatioSample() implementation to use a single conditional (instead of switch/case + fallback return)
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* docs/querying/operators.md: document r < 0
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* add negative limit_ratio() example to docs/querying/examples.md
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* move more extensive docu examples to docs/querying/operators.md
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* typo
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* small docu fix for poor-mans-normality-check, add it to limit.test ;)
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* limit.test: expand "Poor man's normality check" to whole eval range
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* restore mistakenly removed existing small comment
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* expand poors-man-normality-check case(s)
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* Revert "expand poors-man-normality-check case(s)"
This reverts commit f69e1603b2ebe69c0a100197cfbcf6f81644b564, indeed too
flaky 0:)
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* remove humor from docs/querying/operators.md
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* fix signoff
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* add web/ui missing changes
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* expand limit_ratio test cases, cross-fingering they'll not be flaky
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* remove flaky test
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* add missing warnings.Merge(ws) in instant-query return shortcut
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* add missing LimitK||LimitRatio case to codemirror-promql/src/parser/parser.ts
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* fix ui-lint
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
* actually fix returned warnings :]
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
---------
Signed-off-by: JuanJo Ciarlante <juanjosec@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
2024-07-03 22:18:57 +02:00
Arve Knudsen
493e196f10
Fix bad merge
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-03 15:12:40 +02:00
Arve Knudsen
e8ae8cf012
Merge remote-tracking branch 'prometheus/main' into arve/close-engine
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-07-01 10:47:21 +02:00
🌲 Harry 🌊 John 🏔
d5f6887294
Pass limit param as hint to storage.Querier
...
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2024-06-20 09:47:38 -07:00
Jeanette Tan
dda5f48c9e
Merge branch 'main' into nhcb-review-2
2024-06-20 22:50:00 +08:00
Charles Korn
aeec30f082
Convert `TestTimestampFunction_StepsMoreOftenThanSamples`
...
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:56:56 +10:00
Charles Korn
987fa5c6a2
Convert range query test cases to test scripting language
...
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-06-17 16:43:01 +10:00
Jeanette Tan
14f8dded39
Merge branch 'main' into nhcb
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-06-07 19:17:14 +08:00
Arve Knudsen
e57aac8084
Merge remote-tracking branch 'prometheus/main' into arve/close-engine
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-06-05 11:37:44 +02:00
Charles Korn
24ef000025
Merge branch 'main' into sort-matrix-series
...
# Conflicts:
# promql/engine_test.go
2024-05-31 12:44:27 +10:00
Arve Knudsen
0cc99e677a
promql.Engine: Add Close method
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-28 12:01:47 +02:00
Arve Knudsen
7b56353090
Merge remote-tracking branch 'prometheus/main' into arve/query-logger-munmap
2024-05-27 17:08:33 +02:00
Björn Rabenstein
114dc5c393
Merge pull request #13638 from NeerajGartia21/promql-test
...
Converts existing native histogram unit tests to the PromQL testing framework
2024-05-19 19:35:36 +02:00
Arve Knudsen
0f01d4b336
Fix flaky test
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-15 21:58:56 +02:00
Arve Knudsen
bf8d88f326
Merge remote-tracking branch 'origin/main' into arve/query-logger-munmap
2024-05-15 21:02:03 +02:00
Oleksandr Redko
f10c3454e9
Enable perfsprint linter and fix up code
...
Signed-off-by: Oleksandr Redko <oleksandr.red+github@gmail.com>
2024-05-15 17:51:05 +03:00
Jeanette Tan
f028496133
Merge branch 'main' into nhcb
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-05-14 16:20:15 +08:00
Neeraj Gartia
661856cb65
removes the added tests from engine_test.go
...
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:58:25 +05:30
Charles Korn
a4381608a0
Add failing test case
...
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-05-13 11:03:14 +10:00
Arve Knudsen
aabe4d6e4a
promql.ActiveQueryTracker: Unmap mmapped file when done
...
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-09 14:27:29 +02:00
Bryan Boreham
3fd24d1cd7
Merge pull request #13999 from bboreham/extract-promqltest
...
[Test] Extract most PromQL test code into separate packages
2024-05-09 13:23:11 +01:00
Bryan Boreham
786e0e706c
test: PromQL: stop using internal fields of engine
...
* set enablePerStepStats and lookback duration via
`NewTestEngine` parameters.
* check maxSamples by recreating query engine
* check lookback without modifying internals
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 19:58:41 +01:00
Bryan Boreham
0dbfd20b69
test: move most PromQL tests into separate test package
...
So that they can import promqltest which imports promql.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:28:56 +01:00
Bryan Boreham
4a72607c4a
refactor: extract some PromQL Engine tests which use unexported structs
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:27:17 +01:00
Bryan Boreham
babfcfdd91
refactor: Move NewTestEngine into promqltest
...
And export `DefaultMaxSamplesPerQuery` so callers can replicate previous
behaviour.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:07:28 +01:00
Bryan Boreham
a1af3c27d4
refactor: extract almost.Equal() to new package
...
To avoid a circular reference between promql and promqltest.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:10 +01:00
Jeanette Tan
796b1bbfde
Merge branch 'main' into nhcb
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-05-08 19:11:39 +08:00
Arve Knudsen
a25160e6a4
[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector ( #14021 )
...
The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing.
However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples.
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-08 10:39:44 +01:00
György Krajcsovits
bcafa5f1f9
Merge remote-tracking branch 'upstream/main' into update-nhcb
2024-04-24 11:06:59 +02:00
Bryan Boreham
3851b74db1
promql: aggregations: skip result vector in range queries
...
Adjust test to match the lower count, since samples in the vector
are no longer counted.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
György Krajcsovits
2a4aa085d2
Merge branch 'main' into nhcb
2024-03-27 18:42:10 +01:00
George Krajcsovits
dc7b282d39
engine_test: adjust and comment histogram sample counts ( #13841 )
...
The size of histogram points are now bigger by 24 bytes due to the
custom values slice.
When histograms are loaded into partial results in vector selectors
we use HPoint type where the size is calculated as
(size of histogram + 8 for timestamp)/16.
a3d1a46eda/promql/value.go (L176)
When histograms are put into Sample type in range evaluations, the
Sample has more overhead and the size is calculated differently:
(size of histogram / 16) + 1 for time stamp.
a3d1a46eda/promql/engine.go (L1928)
When the size of the histogram is 16k, then the first calculation gives k
but the second gives k+1 for the sample count.
If the histogram size is 16k+8, then both would give k+1.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-03-27 18:19:14 +01:00
Björn Rabenstein
b9a2a4e329
Merge pull request #13852 from prometheus/fix-hist-std-dev-var-negative
...
Fix hist std dev var negative
2024-03-27 17:58:03 +01:00
Jeanette Tan
22d0f4f114
improve handling of negative bounds in histogram std dev/var
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-03-27 17:06:12 +01:00
Domantas
435f330d0b
[BUGFIX] labels: don't modify original labels in DropMetricName ( #13845 )
...
Restrict the capacity of first argument to `append()` to force an allocation.
This is for the slice implementation only.
Signed-off-by: Domantas Jadenkus <djadenkus@gmail.com>
2024-03-27 10:35:17 +00:00
Jeanette Tan
9d32754bc0
add unit tests with all negative values for histogram_stddev and var
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-03-22 03:42:50 +08:00
beorn7
f48c7a5503
promql: Add histograms to TestQueryStatistics
...
Also, fix the bugs exposed by the tests.
Signed-off-by: beorn7 <beorn@grafana.com>
2024-02-29 19:02:40 +01:00
beorn7
f46dd34982
promql: Add code comment
...
Signed-off-by: beorn7 <beorn@grafana.com>
2024-02-29 19:02:40 +01:00
Bryan Boreham
22890b1eb3
PromQL: improve warning for mixed values in aggregations
...
Aggregations discard the metric name, so don't try to
include it in the error message.
Add a test that generates this warning.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-25 19:57:06 +00:00
Bryan Boreham
39af788dbd
Tests: use replacement DeepEquals using go-cmp
...
Use DeepEqual replacement using go-cmp, which is more flexible.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-08 19:30:20 +00:00
Bryan Boreham
252031c86f
Revert "Adding small test update for temp dir using t.TempDir ( #13293 )"
...
This reverts commit 2ddb3596ef
.
Various tests are failing in CI after this change; reverting to free up
other work.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-30 19:17:30 +00:00
Mile Druzijanic
2ddb3596ef
Adding small test update for temp dir using t.TempDir ( #13293 )
...
* Adding small test update for temp dir using t.TempDir
Signed-off-by: Mile Druzijanic <miledruz@gmail.com>
Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>
* removing not required cleanup
Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>
---------
Signed-off-by: Mile Druzijanic <miledruz@gmail.com>
Signed-off-by: Mile Druzijanic <zedsprogramms@gmail.com>
2023-12-28 21:49:57 +01:00
Filip Petkovski
1f69dcfa6b
Fix reusing float histograms
...
In https://github.com/prometheus/prometheus/pull/13276 we started reusing float histogram objects to reduce allocations in PromQL.
That PR introduces a bug where histogram pointers gets copied to the beginning of the histograms slice,
but are still kept in the end of the slice. When a new histogram is read into the last element,
it can overwrite a previous element because the pointer is the same.
This commit fixes the issue by moving outdated points to the end of the slice
so that we don't end up with duplicate pointers in the same buffer. In other words,
the slice gets rotated so that old objects can get reused.
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-14 11:53:58 +01:00
Filip Petkovski
10a82f87fd
Enable reusing memory when converting between histogram types
...
The 'ToFloat' method on integer histograms currently allocates new memory
each time it is called.
This commit adds an optional *FloatHistogram parameter that can be used
to reuse span and bucket slices. It is up to the caller to make sure the
input float histogram is not used anymore after the call.
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
2023-12-08 10:22:59 +01:00
Matthieu MOREL
9c4782f1cc
golangci-lint: enable testifylint linter ( #13254 )
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
zenador
ccfe14d7e7
PromQL: ignore small errors for bucketQuantile ( #13153 )
...
promql: Improve histogram_quantile calculation for classic buckets
Tiny differences between classic buckets are most likely caused by floating point precision issues. With this commit, relative changes below a certain threshold are ignored. This makes the result of histogram_quantile more meaningful, and also avoids triggering the _input to histogram_quantile needed to be fixed for monotonicity_ annotations in unactionable cases.
This commit also adds explanation of the new adjustment and of the monotonicity annotation to the documentation of `histogram_quantile`.
---------
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2023-11-25 00:05:38 +01:00
Linas Medziunas
1cd6c1cde5
ValidateHistogram: strict Count check in absence of NaNs
...
Signed-off-by: Linas Medziunas <linas.medziunas@gmail.com>
2023-11-03 16:17:24 +02:00