prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	d21229b27a	Merge pull request #12101 from bboreham/disable-slow-promql-tests promql: disable some slow cases in TestConcurrentRangeQueries	2 years ago
Julien Pivotto	1fd59791e1	Update tests Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Bryan Boreham	be4a9c25f0	promql: disable some slow cases in TestConcurrentRangeQueries TestConcurrentRangeQueries runs many queries, up to 4 at the same time, to try to expose any race conditions. This change stops four of them from running with a thousand or more steps: `holt_winters(a_X[1d], 0.3, 0.3)` `changes(a_X[1d])` `rate(a_X[1d])` `absent_over_time(a_X[1d])` Particularly when the test runs with `-race` in CI, this reduces the time and resources required. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
tyltr	24a9678dcc	typo 'efficcient' (#12090 ) Signed-off-by: tylitianrui <tylitianrui@126.com>	2 years ago
Justin Lei	af1d9e01c7	Refactor tsdbutil for tests/native histograms (#11948 ) * Add float histograms to ChunkFromSamplesGeneric Signed-off-by: Justin Lei <justin.lei@grafana.com> * Add GenerateSamples functions to tsdbutil Signed-off-by: Justin Lei <justin.lei@grafana.com> PR responses Signed-off-by: Justin Lei <justin.lei@grafana.com> --------- Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Björn Rabenstein	60d763282e	Merge pull request #11864 from prometheus/beorn7/histogram2 histograms: Return actually useful counter reset hints	2 years ago
beorn7	1cfc8f65a3	histograms: Return actually useful counter reset hints This is a bit more conservative than we could be. As long as a chunk isn't the first in a block, we can be pretty sure that the previous chunk won't disappear. However, the incremental gain of returning NotCounterReset in these cases is probably very small and might not be worth the code complications. Wwith this, we now also pay attention to an explicitly set counter reset during ingestion. While the case doesn't show up in practice yet, there could be scenarios where the metric source knows there was a counter reset even if it might not be visible from the values in the histogram. It is also useful for testing. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Bryan Boreham	9ae3572d24	TestConcurrentRangeQueries: log query with error We've seen some timeouts in CI, and wanted to know what queries are involved. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	57bcbf1888	Merge pull request #11783 from codesome/gauge-histogram tsdb: Add gauge histogram support	2 years ago
Ganesh Vernekar	3c2ea91a83	tsdb: Test gauge float histograms Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	fd89d7892c	Merge pull request #11809 from bboreham/dont-sort-postings-values tsdb: sort values for Postings only when required	2 years ago
Bryan Boreham	80ac0d7c82	promql: add benchmark for match against blank string Blank strings are not handled efficiently by tsdb. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Marc Tudurí	49f775d8a0	histograms: Add missing float histograms tests for PromQL (#11780 ) * test: TestSparseHistogramRate * test: TestSparseHistogram_HistogramQuantile * test: TestSparseHistogram_HistogramFraction * test: TestSparseHistogram_HistogramFraction * test: TestSparseHistogram_Sum_Count_AddOperator * test: TestSparseHistogram_HistogramCountAndSum * tests: fix TestSparseHistogram_HistogramCountAndSum * linter * refactor TestSparseHistogram_HistogramCountAndSum * wrap TestSparseHistogram_HistogramCountAndSum Signed-off-by: Marc Tuduri <marctc@protonmail.com>	2 years ago
Marc Tudurí	9474610baf	Support FloatHistogram in TSDB (#11522 ) Extends Appender.AppendHistogram function to accept the FloatHistogram. TSDB supports appending, querying, WAL replay, for this new type of histogram. Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	1b0a29701b	promql: optimise aggregation with no labels For a query like 'sum (foo)', we can quickly skip to the empty labels that its result needs. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	aafef011b7	Promql: reuse LabelBuilder in aggregations We have a LabelBuilder in EvalNodeHelper; use it instead of creating a new one at every step. Need to take some care that different uses of enh.lb do not overlap. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	2c382f5e24	promql: extract function to initialize LabelBuilder Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ccea61c7bf	Merge pull request #11717 from bboreham/labels-abstraction Add and use abstractions over labels.Labels	2 years ago
Bryan Boreham	dbd7021cc2	promql: add test for race conditions in query engine (#11743 ) * promql: refactor BenchmarkRangeQuery so we can re-use test cases Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * promql: add test for race conditions in query engine Note we skip large count_values queries - `count_values` allocates a slice per unique value in the output, and this test has unique values on every step of every series so it adds up to a lot of slices. Add Go runtime overhead for checking `-race`, and it chews up many gigabytes. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * TestConcurrentRangeQueries: wait before starting goroutine Instead of starting 100 goroutines which just wait for the semaphore. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	aa634e0b7e	Update package promql tests for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	56fefcd812	Update package promql for new labels.Labels type We use `labels.Builder` to parse metrics, to avoid depending on the internal implementation. This is not efficient, but the feature is only used in tests. It wasn't efficient previously either - calling `Sort()` after adding each label. `createLabelsForAbsentFunction` also uses a Builder now, and gets an extra `map` to replace the previous `Has()` usage. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Fix up promql to compile with changes to Labels	2 years ago
Bryan Boreham	1695a7ee2f	promql: refactor BenchmarkRangeQuery so we can re-use test cases Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	463f5cafdd	storage: re-use iterators to save garbage Re-use previous memory if it is already of the correct type. In `NewListSeries` we hoist the conversion to an interface value out so it only allocates once. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	3c7de69059	storage: allow re-use of iterators Patterned after `Chunk.Iterator()`: pass the old iterator in so it can be re-used to avoid allocating a new object. (This commit does not do any re-use; it is just changing all the method signatures so re-use is possible in later commits.) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Alan Protasio	8460807475	fix blank lines Signed-off-by: Alan Protasio <approtas@amazon.com>	2 years ago
Alan Protasio	f8f4ac14a8	Finishing evalSpanTimer always before return Signed-off-by: Alan Protasio <approtas@amazon.com>	2 years ago
Bryan Boreham	6bdecf377c	Switch from 'sanity' to more inclusive lanuage (#9376 ) * Switch from 'sanity' to more inclusive lanuage "Removing ableist language in code is important; it helps to create and maintain an environment that welcomes all developers of all backgrounds, while emphasizing that we as developers select the most articulate, precise, descriptive language we can rather than relying on metaphors. The phrase sanity check is ableist, and unnecessarily references mental health in our code bases. It denotes that people with mental illnesses are inferior, wrong, or incorrect, and the phrase sanity continues to be used by employers and other individuals to discriminate against these people." From https://gist.github.com/seanmhanson/fe370c2d8bd2b3228680e38899baf5cc Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	bde500e690	Merge pull request #11420 from jesusvazquez/jvp/update-sparsehistogram-with-main-v2 Merge branch 'main' into sparsehistogram	2 years ago
beorn7	bf0847073d	histogram: Modify getBound to deal properly with infinity The bucket receiving math.MaxFloat64 observations now has math.MaxFloat64 as upper bound, while the bucket after it (the last possible bucket) has +Inf. This also adds a test for getBound and moves the getBound code to generic.go (where it should have been in the first place). Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Jesus Vazquez	e934d0f011	Merge 'main' into sparsehistogram Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>	2 years ago
Björn Rabenstein	dccfb9db4e	histogram: Remove code replication via generics (#11361 ) * histogram: Simplify iterators We don't really need currLower and currUpper and can calculate it when needed (as already done for the floatBucketIterator). The calculation is cheap, while keeping those extra variables around costs RAM (potentially a lot with many iterators). * histogram: Convert Bucket/FloatBucket to one generic type * histogram: Move some bucket iterator code into generic base iterator * histogram: Remove cumulative iterator for FloatHistogram We added it in the past for completeness (Histogram has one), but it has never been used. Plus, even the cumulative iterator for Histogram is only there for test reasons. We can always add it back, and then maybe even using generics. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Giedrius Statkevičius	a1d6ba59ac	promql: pass down subquery interval (#11163 ) If we are populating series for a subquery then set the interval parameter accordingly so that downstream users could use that information. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>	2 years ago
Bryan Boreham	3330d85ba8	Replace sort.Strings and sort.Ints with faster slices.Sort (#11318 ) Use new experimental package `golang.org/x/exp/slices`. slices.Sort works on values that are directly comparable, like ints, so avoids the overhad of an interface call to `.Less()`. Left tests unchanged, because they don't need the speed and it may be a cross-check that slices.Sort gives the same answer. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
beorn7	a7c519930e	histograms: Add Compact method to the normal integer Histogram And use the new method to call to compact Histograms during parsing. This happens for both `Histogram` and `FloatHistogram`. In this way, if targets decide to optimize the exposition size by merging spans with empty buckets in between, we still get a normalized results. It will also normalize away any valid but weird representations like empty spans, spans with offset zero, and empty buckets at the start or end of a span. The implementation seemed easy at first as it just turns the `compactBuckets` helper into a generic function (which now got its own file). However, the integer Histograms have delta buckets instead of absolute buckets, which had to be treated specially in the generic `compactBuckets` function. To make sure it works, I have added plenty of explicit tests for `Histogram` in addition to the `FloatHistogram` tests. I have also updated the doc comment for the `Compact` method. Based on the insights now expressed in the doc comment, compacting with a maxEmptyBuckets > 0 is rarely useful. Therefore, this commit also sets the value to 0 in the two cases we were using 3 so far. We might still want to reconsider, so I don't want to remove the maxEmptyBuckets parameter right now. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Bryan Boreham	b01d29cf9e	promql: in tests use labels.FromStrings And a few cases of `EmptyLabels()`. Replacing code which assumes the internal structure of `Labels`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Julien Pivotto	96d5a32659	Update go to 1.19, set min version to 1.18 (#11279 ) * Update go to 1.19, set min version to 1.18 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> * Update golangci-lint Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Ganesh Vernekar	71489d0e3d	Fix count() for histograms and add test case Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	8b863c42dd	Optimise relabeling by re-using memory (#11147 ) * model/relabel: Add benchmark Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * model/relabel: re-use Builder across relabels Saves memory allocations. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * labels.Builder: allow re-use of result slice This reduces memory allocations where the caller has a suitable slice available. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * model/relabel: re-use source values slice To reduce memory allocations. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Unwind one change causing test failures Restore original behaviour in PopulateLabels, where we must not overwrite the input set. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * relabel: simplify values optimisation Use a stack-based array for up to 16 source labels, which will be the vast majority of cases. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * lint Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	9325caa41c	Remove a TODO that is no longer valid (#11186 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Cosrider	af79dce8ea	Close file descriptor in query logging (#11148 ) Signed-off-by: Cosrider <cosrider7@gmail.com> Signed-off-by: Cosrider <cosrider7@gmail.com>	2 years ago
beorn7	c9fd3c235d	Merge branch 'main' into sparsehistogram	2 years ago
Vilius Pranckaitis	4660656312	Allow setting custom lookback delta for instant queries (#9946 ) * Allow setting custom lookback delta for instant queries Signed-off-by: Vilius Pranckaitis <vpranckaitis@gmail.com>	2 years ago
Levi Harrison	77a7af4461	Add histogram validation (#11052 ) * Add histogram validation Signed-off-by: Levi Harrison <git@leviharrison.dev> * Correct negative offset validation Signed-off-by: Levi Harrison <git@leviharrison.dev> * Address review comments Signed-off-by: Levi Harrison <git@leviharrison.dev> * Validation benchmark Signed-off-by: Levi Harrison <git@leviharrison.dev> * Add more checks Signed-off-by: Levi Harrison <git@leviharrison.dev> * Attempt to fix tests Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fix stuff Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Julien Pivotto	d41e5a5582	Prettifier: Add spaces with non-callable keywords (#11005 ) * Prettifier: Add spaces with non-callable keywords I prefer to have a difference between, on one side: functions calls, end(), start(), and on the other side with, without, ignoring, by and group_rrigt, group_left. The reasoning is that the former ones are not calls, while other are functions. Additionally, it matches the examples in our documentation. Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> * Fix tests Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Łukasz Mierzwa	54a3c3ba3f	Print query that caused a panic (#10995 ) We print the stacktrace of a panic when query causes one, but there's no information about the query itself, which makes it harder to debug and reproduce the issue. This adds the 'expr' string to the logged panic. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	2 years ago
beorn7	53982c3562	Merge branch 'main' into sparsehistogram	2 years ago
Oleg Zaytsev	d2abe9a58a	Shortcut `ContainsSameLabelset()` (#11000 ) * Shortcut Matrix.ContainsSameLabelset() It's quite often to execute this check on a Matrix that has zero or only one series. There's no need to allocate a map for those cases. There's also a one-liner for two-series case, so why not using it? Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> * Add license header Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> * Optimize Vector.ContainsSameLabelset Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2 years ago
beorn7	28f028e938	Merge branch 'main' into sparsehistogram	2 years ago
Harkishen Singh	44fcf876ca	Adds support for prettifying PromQL expression (#10544 ) * Implement Pretty() function for AST nodes. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> This commit adds .Pretty() for all nodes of PromQL AST. Each .Pretty() prettifies the node it belongs to, and under no circustance, the parent or child node is touch/prettified. Read more in the "Approach" part in `prettier.go` * Refactor functions between printer.go & prettier.go Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> This commit removes redundancy between printer.go and prettier.go by taking out the common code into separate private functions. * Add more unit tests for Prettier. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Add support for spliting function calls with 1 arg & unary expressions. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> This commit does 2 things: 1. It adds support to split function calls that have 1 arg and exceeds the max_characters_per_line to multiple lines. 2. Splits Unary expressions that exceed the max_characters_per_line. This is done by formatting the child node and then removing the prefix indent, which is already applied before the unary operator.	2 years ago
beorn7	9eafed0f79	promql: Add `histogram_count` and `histogram_sum` This follow a simple function-based approach to access the count and sum fields of a native Histogram. It might be more elegant to implement “accessors” via the dot operator, as considered in the brainstorming doc [1]. However, that would require the introduction of a whole new concept in PromQL. For the PoC, we should be fine with the function-based approch. Even the obvious inefficiencies (rate'ing a whole histogram twice when we only want to rate each the count and the sum once) could be optimized behind the scenes. Note that the function-based approach elegantly solves the problem of detecting counter resets in the sum of observations in the case of negative observations. (Since the whole native Histogram is rate'd, the counter reset is detected for the Histogram as a whole.) We will decide later if an “accessor” approach is really needed. It would change the example expression for average duration in functions.md from histogram_sum(rate(http_request_duration_seconds[10m])) / histogram_count(rate(http_request_duration_seconds[10m])) to rate(http_request_duration_seconds.sum[10m]) / rate(http_request_duration_seconds.count[10m]) [1]: https://docs.google.com/document/d/1ch6ru8GKg03N02jRjYriurt-CZqUVY09evPg6yKTA1s/edit Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago

1 2 3 4 5 ...

741 Commits (742979a3e55b4427f2407a76c48379a646158e02)