Bartlomiej Plotka
fe802f29c9
storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response.
...
This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in
Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet
which rely on sorting.
I found during work on https://github.com/prometheus/prometheus/pull/5882 that
we do so many repetitions because of this, for not good reason. I think
I found a good balance between convenience and readability with just one method.
Smaller the interface = better.
Also I don't know what TestSelectSorted was testing, but now it's testing sorting.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-03-13 13:06:25 +00:00
Björn Rabenstein
bc703b6456
Use `struct{}` as underlying type for context keys ( #6965 )
...
This is an alternative to #6963 .
Signed-off-by: beorn7 <beorn@grafana.com>
2020-03-11 15:05:35 +01:00
Julien Pivotto
5ddd1dcf0f
Fix panic when parsing varags ( #6940 )
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-08 13:09:24 +01:00
Tobias Guggenmos
1dbd799354
PromQL: Fix regression tests ( #6935 )
...
This PR fixes the regression tests for the issue fixed in #6931 .
The reason for that is that all of the invalid queries that triggered the regression have become more or less valid syntax in #6933 (they might still fail typechecking).
Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-06 08:17:01 +00:00
Brian Brazil
44ad28dd5e
PromQL: Allow more keywords as metric names ( #6933 )
...
* Allow more keywords as metric names
* Add documentation about forbidden keywords
Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-05 13:20:53 +00:00
Brian Brazil
7164b58945
PromQL: Fix parser panic ( #6931 )
...
Signed-off-by: Tobias Guggenmos <tobias.guggenmos@uni-ulm.de>
2020-03-05 08:03:38 +00:00
李国忠
2bf4952049
remove Unused parameter 'sf' in calcTrendValue function ( #6900 )
...
Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>
2020-03-03 08:23:27 +00:00
LongKB
82f7ed208b
Remove some duplicated words ( #6882 )
...
Signed-off-by: Pham Duc Hanh <hanhpd@fujitsu.com>
2020-02-27 07:08:31 +01:00
Tobias Guggenmos
3d74fcfa6a
Bartek's suggestions
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-25 13:57:30 +01:00
Tobias Guggenmos
f9db320e5a
Look up function call in all cases
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-24 13:45:03 +01:00
Tobias Guggenmos
9ebf6bd1e6
Remove superfluous blank lines
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-21 13:36:57 +01:00
Tobias Guggenmos
7143d64fc1
Julien's suggestion
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
2020-02-21 13:34:15 +01:00
Tobias Guggenmos
4124828c00
Add test to check that promql.FunctionCalls and parser.Functions contain the same functions
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-21 12:43:30 +01:00
Tobias Guggenmos
4835bbf376
Merge branch 'master' into split_parser
2020-02-19 15:18:13 +01:00
Bartlomiej Plotka
849faa407b
Minor fixes.
...
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka
2cf637fbf5
Addressed comments.
...
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka
34426766d8
Unify Iterator interfaces. All point to storage now.
...
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things.
All todos I added will be fixed in follow up PRs.
* querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged
with storage interface.go. All imports that.
* querier.SeriesIterator replaced by chunkenc.Iterator
* Added chunkenc.Iterator.Seek method and tests for xor implementation (?)
* Since we properly handle SelectParams for Select methods I adjusted min max
based on that. This should help in terms of performance for queries with functions like offset.
* added Seek to deletedIterator and test.
* storage/tsdb was removed as it was only a unnecessary glue with incompatible structs.
No logic was changed, only different source of abstractions, so no need for benchmarks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:54 +00:00
Tobias Guggenmos
2d77611f73
Unbump year number
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
bf8542a99e
Add missing imports to fuzz tests
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
ec3df53f78
Add nolint comment
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
b93845d6bf
Remove unused function
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
43d206f0a1
Add missing file
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
6c00f2ffcb
Comment fixes
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
273ea9093a
Fix promql tests
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
1360f9ff12
Fix all build errors in promql package
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:09:23 +01:00
Tobias Guggenmos
2164e366ea
Fix more identifiers
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos
5caf7ed6db
Fix more identifiers
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos
9a1366775e
Store function implementations independently of their signatures
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos
ff0ea1c1ac
Fix more identifiers
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos
6b1b323558
Export sequenceValue
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:53 +01:00
Tobias Guggenmos
228967a507
Fix usages of more things that have moved the package
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:07:27 +01:00
Tobias Guggenmos
4a4817a444
Fix usages of parser.Statement
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos
2f1113479f
Fix usages of ValueType
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos
65f5b9827d
Rename package
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos
98a9b7c90c
Remove engine specific types
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos
fab2373752
Add everything the parser needs to build
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Tobias Guggenmos
e7c2716f55
Move files
...
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
2020-02-17 16:05:21 +01:00
Zhou Hao
e628fd7735
fix comments spelling ( #6829 )
...
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
2020-02-17 12:45:11 +01:00
Björn Rabenstein
af04cb22c8
Merge pull request #6821 from prometheus/release-2.16
...
Release 2.16
2020-02-14 13:10:14 +01:00
Julien Pivotto
ff0003e072
Make lookbackDelta a option of QueryEngine ( #6746 )
...
* Make lookbackDelta a option of QueryEngine
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* julius' suggestion
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* remove trivial getter
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Assume lookback delta is always > 0
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* add debug log
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* don't expose loopback delta
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Specify that lookack delta is also used in federation
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Fix federation test
While we have added some logic to the promql engine to keep it backwards
compatible and have a 5 minute loopback by default, the web/ package is
likely to really be internal to Prometheus and we should not add the
same kind of heuritstics here.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* loopback delta: Fix debug log
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-10 00:58:23 +01:00
Julien Pivotto
cbd0eec9fc
Avoid /1000
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-08 16:30:09 +01:00
Julien Pivotto
881dde505a
promql: fix promql query log step unit
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-08 16:26:56 +01:00
Julien Pivotto
9adad8ad30
Remove MaxConcurrent from the PromQL engine opts ( #6712 )
...
Since we use ActiveQueryTracker to check for concurrency in
d992c36b3a
it does not make sense to keep
the MaxConcurrent value as an option of the PromQL engine.
This pull request removes it from the PromQL engine options, sets the
max concurrent metric to -1 if there is no active query tracker, and use
the value of the active query tracker otherwise.
It removes dead code and also will inform people who import the promql
package that we made that change, as it breaks the EngineOpts struct.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-28 20:38:49 +00:00
Brian Brazil
61262159c4
Simplify benchmark given the new API
...
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-01-28 14:38:09 +00:00
Brian Brazil
38d32e0686
Don't sort postings if we only have one block.
...
Sorting the heads postings can be quite slow.
We only need sorted series when merging with another
querier, so only sort then.
This will make big queries that only touch the head faster,
though queries that touch both the head and a block will still
be the same speed. This probably won't help much with graphing
unless the range is under an hour, however it should make most
recording rules faster.
Add gaurantee that remote read streaming produces sorted series.
PromQL benchmarks for histograms show only 2-3% improvement, but
they're only over 1k series.
benchmark old ns/op new ns/op delta
BenchmarkQuerierSelect/Head/1of1000000-4 1375486282 507657736 -63.09%
BenchmarkQuerierSelect/Head/10of1000000-4 1387859004 507769850 -63.41%
BenchmarkQuerierSelect/Head/100of1000000-4 1387087935 506029110 -63.52%
BenchmarkQuerierSelect/Head/1000of1000000-4 1386869064 504521986 -63.62%
BenchmarkQuerierSelect/Head/10000of1000000-4 1386213685 505210422 -63.55%
BenchmarkQuerierSelect/Head/100000of1000000-4 1392754988 529842406 -61.96%
BenchmarkQuerierSelect/Head/1000000of1000000-4 1569414722 725059506 -53.80%
BenchmarkQuerierSelect/SortedHead/1of1000000-4 1381019902 1370495863 -0.76%
BenchmarkQuerierSelect/SortedHead/10of1000000-4 1375696209 1366789468 -0.65%
BenchmarkQuerierSelect/SortedHead/100of1000000-4 1386009422 1364519297 -1.55%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4 1377700532 1364486191 -0.96%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4 1383539536 1369545314 -1.01%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4 1410089163 1394731339 -1.09%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4 1634744148 1581554956 -3.25%
BenchmarkQuerierSelect/Block/1of1000000-4 881741242 879839470 -0.22%
BenchmarkQuerierSelect/Block/10of1000000-4 880381562 882846038 +0.28%
BenchmarkQuerierSelect/Block/100of1000000-4 887519357 881016916 -0.73%
BenchmarkQuerierSelect/Block/1000of1000000-4 902194205 883433524 -2.08%
BenchmarkQuerierSelect/Block/10000of1000000-4 892321964 885130170 -0.81%
BenchmarkQuerierSelect/Block/100000of1000000-4 938604466 933527150 -0.54%
BenchmarkQuerierSelect/Block/1000000of1000000-4 1313510845 1295881124 -1.34%
benchmark old allocs new allocs delta
BenchmarkQuerierSelect/Head/1of1000000-4 4000056 4000018 -0.00%
BenchmarkQuerierSelect/Head/10of1000000-4 4000074 4000036 -0.00%
BenchmarkQuerierSelect/Head/100of1000000-4 4000254 4000216 -0.00%
BenchmarkQuerierSelect/Head/1000of1000000-4 4002054 4002016 -0.00%
BenchmarkQuerierSelect/Head/10000of1000000-4 4020054 4020016 -0.00%
BenchmarkQuerierSelect/Head/100000of1000000-4 4200054 4200016 -0.00%
BenchmarkQuerierSelect/Head/1000000of1000000-4 6000054 6000016 -0.00%
BenchmarkQuerierSelect/SortedHead/1of1000000-4 4000071 4000071 +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4 4000089 4000089 +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4 4000269 4000269 +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4 4002069 4002069 +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4 4020069 4020069 +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4 4200069 4200069 +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4 6000069 6000069 +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4 6000023 6000022 -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4 6000059 6000058 -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4 6000419 6000418 -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4 6004019 6004018 -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4 6040019 6040018 -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4 6400019 6400018 -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4 10000020 10000019 -0.00%
benchmark old bytes new bytes delta
BenchmarkQuerierSelect/Head/1of1000000-4 229192200 176001176 -23.21%
BenchmarkQuerierSelect/Head/10of1000000-4 229193352 176002328 -23.21%
BenchmarkQuerierSelect/Head/100of1000000-4 229204872 176013848 -23.21%
BenchmarkQuerierSelect/Head/1000of1000000-4 229320072 176129048 -23.20%
BenchmarkQuerierSelect/Head/10000of1000000-4 230472072 177281048 -23.08%
BenchmarkQuerierSelect/Head/100000of1000000-4 241992072 188801048 -21.98%
BenchmarkQuerierSelect/Head/1000000of1000000-4 357192072 304001048 -14.89%
BenchmarkQuerierSelect/SortedHead/1of1000000-4 229193928 229193928 +0.00%
BenchmarkQuerierSelect/SortedHead/10of1000000-4 229195080 229195080 +0.00%
BenchmarkQuerierSelect/SortedHead/100of1000000-4 229206600 229206600 +0.00%
BenchmarkQuerierSelect/SortedHead/1000of1000000-4 229321800 229321800 +0.00%
BenchmarkQuerierSelect/SortedHead/10000of1000000-4 230473800 230473800 +0.00%
BenchmarkQuerierSelect/SortedHead/100000of1000000-4 241993800 241993800 +0.00%
BenchmarkQuerierSelect/SortedHead/1000000of1000000-4 357193800 357193800 +0.00%
BenchmarkQuerierSelect/Block/1of1000000-4 227201516 227201500 -0.00%
BenchmarkQuerierSelect/Block/10of1000000-4 227202924 227202908 -0.00%
BenchmarkQuerierSelect/Block/100of1000000-4 227217036 227217020 -0.00%
BenchmarkQuerierSelect/Block/1000of1000000-4 227358156 227358140 -0.00%
BenchmarkQuerierSelect/Block/10000of1000000-4 228769356 228769340 -0.00%
BenchmarkQuerierSelect/Block/100000of1000000-4 242881356 242881340 -0.00%
BenchmarkQuerierSelect/Block/1000000of1000000-4 384001616 384001600 -0.00%
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-01-28 09:14:56 +00:00
Julien Pivotto
d992c36b3a
promql: make active query tracker context-aware ( #6701 )
...
* promql: make query logger context-aware
* Remove gate
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-27 22:29:44 +00:00
Julien Pivotto
cf42888e4d
Fix order of testutil.Equals ( #6695 )
...
Equals takes the expected value as first parameter, and the actual value
as second parameter.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-27 12:21:59 +00:00
Julien Pivotto
5f27ac3583
Refactor query log fields ( #6694 )
...
* Refactor query log fields
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-27 09:53:10 +00:00
Julien Pivotto
3fbeee07fa
Fix flaky windows test ( #6677 )
...
The windows clock is sometime off by 25ms, and as precise as 15ms.
Let's give it more time to avoid flaky tests.
Fix #6672
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-22 10:09:31 +00:00
Julien Pivotto
2b2eb79e8b
Add windows tests for query logger ( #6653 )
...
* Add windows tests
* Do not rely on time.Time in timer
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-20 13:17:11 +00:00