prometheus

Commit Graph

Author	SHA1	Message	Date
AllenZMC	ead0933dd9	fix word 'substracting' to 'subtracting' (#5822 ) Signed-off-by: czm <zhongming.chang@daocloud.io>	5 years ago
Advait Bhatwadekar	5d401f1e1b	Added query logging for prometheus. Issue #1315 (#5794 ) * Added query logging for prometheus. Options added: 1) active.queries.filepath: Filename where queries will be recorded 2) active.queries.filesize: Size of the file where queries will be recorded. Functionality added: All active queries are now logged in a file. If prometheus crashes unexpectedly, these queries are also printed out on stdout in the rerun. Queries are written concurrently to an mmaped file, and removed once they are done. Their positions in the file are reused. They are written in json format. However, due to dynamic nature of application, the json has an extra comma after the last query, and is missing an ending ']'. There may also null bytes in the tail of file. Signed-off-by: Advait Bhatwadekar <advait123@ymail.com>	5 years ago
Ganesh Vernekar	588eb20018	Efficient iteration and search in HashForLabels and HashWithoutLabels (#5707 ) * Efficient iteration and search in Labels.HashForLabels Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Better names for variables Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * HashWithoutLabels optimizations Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor HashForLabels and HashWithoutLabels to take sorted names Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
beorn7	dd81912554	Add objectives to Summaries With the next release of client_golang, Summaries will not have objectives by default. To not lose the objectives we have right now, explicitly state the current default objectives. Signed-off-by: beorn7 <beorn@grafana.com>	6 years ago
Thomas Jackson	a000cec011	Re-use label builder in promql aggregation (#5641 ) For my benchmarks on aggregation this reduces allocations by ~5% (~10% time improvement): ``` benchmark old ns/op new ns/op delta BenchmarkEvaluations/benchdata/aggregators.test/promxy-4 727692 649626 -10.73% benchmark old allocs new allocs delta BenchmarkEvaluations/benchdata/aggregators.test/promxy-4 2566 2434 -5.14% benchmark old bytes new bytes delta BenchmarkEvaluations/benchdata/aggregators.test/promxy-4 162760 148854 -8.54% ``` Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>	6 years ago
Goutham Veeramachaneni	3cc5f9d880	Make sure subquery range is taken into account for selection (#5467 ) * Make sure subquery range is taken into account for selection Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Julius Volz	bc1c7f1809	Fix scalar-vector comparisons (#5454 ) * Fix scalar-vector comparisons Fixes https://github.com/prometheus/prometheus/issues/5452 Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
Bryan Boreham	69dd090880	Check for cancellation on every step of a range evaluation Signed-off-by: Bryan Boreham <bryan@weave.works>	6 years ago
Bryan Boreham	e4a37d0986	Replace select with simpler error check The documentation for Context states that this is just as good: // If Done is not yet closed, Err returns nil. // If Done is closed, Err returns a non-nil error Signed-off-by: Bryan Boreham <bryan@weave.works>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Julius Volz	8155cc4992	Expose lexer item types (#5358 ) * Expose lexer item types We have generally agreed to expose AST types / values that are necessary to make sense of the AST outside of the promql package. Currently the `UnaryExpr`, `BinaryExpr`, and `AggregateExpr` AST nodes store the lexer item type to indicate the operator type, but since the individual item types aren't exposed, an external user of the package cannot determine the operator type. So this PR exposes them. Although not all item types are required to make sense of the AST (some are really only used in the lexer), I decided to expose them all here to be somewhat more consistent. Another option would be to not use lexer item types at all in AST nodes. The concrete motivation is my work on the PromQL->Flux transpiler, but this ought to be useful for other cases as well. Signed-off-by: Julius Volz <julius.volz@gmail.com> * Fix item type names in tests Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
Daisy T	683fbc59ec	exponentation operator to drop metric name in result of op operation (#5329 ) Signed-off-by: Daisy T <daisyts@gmx.com>	6 years ago
Brian Brazil	858c363e94	Fix panic when aggregator param is not a literal. The return value for checkForSeriesSetExpansion is always nil, simplify. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Tariq Ibrahim	a2a6e24f9f	show list of offending labels in the error message in many-to-many scenarios (#5189 ) Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Bryan Boreham	8841692a63	Use the context associated with the inner evaluation span (#5130 ) Signed-off-by: Bryan Boreham <bryan@weave.works>	6 years ago
Matt Layher	43c9d9e91f	promql: apply golint suggestions (#5066 ) Signed-off-by: Matt Layher <mdlayher@gmail.com>	6 years ago
Simon Pasquier	f678e27eb6	: use latest release of staticcheck (#5057 ) : use latest release of staticcheck It also fixes a couple of things in the code flagged by the additional checks. Signed-off-by: Simon Pasquier <spasquie@redhat.com> Use official release of staticcheck Also run 'go list' before staticcheck to avoid failures when downloading packages. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Tom Wilkie	6e08029b56	Move err to be the last return value from storage.Select. (#5054 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Ganesh Vernekar	dbe55c1352	Subquery (#4831 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Tom Wilkie	e1d9bf77f1	Export the error field in ErrStorage, so we can 'throw' it outside the package. (#4954 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
mknapphrt	f0e9196dca	Return warnings on a remote read fail (#4832 ) Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>	6 years ago
Ben Kochie	c6399296dc	Fix spelling/typos (#4921 ) * Fix spelling/typos Fix spelling/typos reported by codespell/misspell. * UK -> US spelling changes. Signed-off-by: Ben Kochie <superq@gmail.com>	6 years ago
Bryan Boreham	9a956872a3	Make ErrorStorage a concrete type not an interface Since it is used in a type assertion, having it as an alias to the error interface is the same as saying 'error', i.e. it succeeds for all types of error. Change to a struct which is a concrete type and the type assertion will only succeed if the type is identical. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	6 years ago
Callum Styan	9bca041285	WIP: keep track of samples per query, set a max # of samples (#4513 ) * keep track of samples per query, set a max # of samples that can be in memory at once Signed-off-by: Callum Styan <callumstyan@gmail.com>	6 years ago
Tom Wilkie	4c52400708	Limit concurrent remote reads. (#4656 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Harsh Agarwal	18a9a390b5	Add duplicate-labelset check for range/instant vectors (#4589 ) Signed-off-by: Harsh Agarwal <cs15btech11019@iith.ac.in>	6 years ago
Ganesh Vernekar	576ee4d309	Label name check for 'count_values' (#4585 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Dan Cech	9f4cb06a37	use Welford/Knuth method to compute standard deviation and variance (#4533 ) * use Welford/Knuth method to compute standard deviation and variance, avoids float precision issues * use better method for calculating avg and avg_over_time Signed-off-by: Dan Cech <dcech@grafana.com>	6 years ago
Goutham Veeramachaneni	71855a22a4	Add tracing spans to promql (#4436 ) * Add spans to promql Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Simplify timer and span tracking. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Thomas Jackson	56daa1f28a	Only add LookbackDelta to vector selectors (#4399 ) Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Related to #4226	6 years ago
Alin Sinpalean	372e7652b7	Reuse (copy) overlapping matrix samples between range evaluation steps (#4315 ) * Reuse (copy) overlapping matrix samples between range evaluation steps. Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>	6 years ago
Tony Lee	bcdaf8e2d2	add unused pointslices to the pool (#4363 ) Signed-off-by: Tony Lee <tl@hudson-trading.com>	6 years ago
Alin Sinpalean	e3b775b78b	Simplify BufferedSeriesIterator usage (#4294 ) * Allow for BufferedSeriesIterator instances to be created without an underlying iterator, to simplify their usage. Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>	6 years ago
Julius Volz	219e477272	Fix some (valid) lint errors (#4287 ) Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
Thomas Jackson	92c6f0c92e	Add offset to selectParams (#4226 ) * Add Start/End to SelectParams * Make remote read use the new selectParams for start/end This commit will continue sending the start/end time of the remote read query as the overarching promql time and the specific range of data that the query is intersted in receiving a response to is now part of the ReadHints (upstream discussion in #4226). * Remove unused vendored code The genproto.sh script was updated, but the code wasn't regenerated. This simply removes the vendored deps that are no longer part of the codegen output. Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>	6 years ago
Alin Sinpalean	96fb0b2155	Optimize PromQL aggregations (#4248 ) * Compute hash of label subsets without creating a LabelSet first. Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>	6 years ago
Tom Wilkie	3228814456	Don't forget to register query_duration_seconds{slice="queue_time"} (#4381 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Thomas Jackson	a6dace8829	Check for timeout in each iteration of matrixSelector (#4300 ) Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4288	7 years ago
Thomas Jackson	630f42fcf1	Timeout if populating iterators takes too long (#4291 ) Right now promql won't time out a request if populating the iterators takes a long time. Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4289	7 years ago
Thomas Jackson	404abe0f1c	Bubble up errors to promql from populating iterators (#4136 ) This changes the Walk/Inspect API inside the promql package to bubble up errors. This is done by having the inspector return an error (instead of a bool) and then bubbling that up in the Walk. This way if any error is encountered in the Walk() the walk will stop and return the error. This avoids issues where errors from the Querier where being ignored (causing incorrect promql evaluation). Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4136	7 years ago
Mario Trangoni	0e2aa35771	promql: fix unconvert issues (#4040 ) See, $ gometalinter --vendor --disable-all --enable=unconvert --deadline 6m ./... promql/engine.go:1396:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1396:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1505:50⚠️ unnecessary conversion (unconvert) promql/engine.go:1573:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1578:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1591:80⚠️ unnecessary conversion (unconvert) promql/engine.go:1602:94⚠️ unnecessary conversion (unconvert) promql/engine.go:1630:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1631:24⚠️ unnecessary conversion (unconvert) promql/engine.go:1634:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1635:34⚠️ unnecessary conversion (unconvert) promql/functions.go:302:42⚠️ unnecessary conversion (unconvert) promql/functions.go:315:42⚠️ unnecessary conversion (unconvert) promql/functions.go:334:26⚠️ unnecessary conversion (unconvert) promql/functions.go:395:31⚠️ unnecessary conversion (unconvert) promql/functions.go:406:31⚠️ unnecessary conversion (unconvert) promql/functions.go:454:27⚠️ unnecessary conversion (unconvert) promql/functions.go:701:46⚠️ unnecessary conversion (unconvert) promql/functions.go:701:78⚠️ unnecessary conversion (unconvert) promql/functions.go:730:43⚠️ unnecessary conversion (unconvert) promql/functions.go:1220:23⚠️ unnecessary conversion (unconvert) promql/functions.go:1249:23⚠️ unnecessary conversion (unconvert) promql/quantile.go:107:54⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:16⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:64⚠️ unnecessary conversion (unconvert) Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>	7 years ago
Brian Brazil	dd6781add2	Optimise PromQL (#3966 ) * Move range logic to 'eval' Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make aggregegate range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * PromQL is statically typed, so don't eval to find the type. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Extend rangewrapper to multiple exprs Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Start making function evaluation ranged Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make instant queries a special case of range queries Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Eliminate evalString Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Evaluate range vector functions one series at a time Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make unary operators range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make binops range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Pass time to range-aware functions. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple _over_time functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce allocs when working with matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add basic benchmark for range evaluation Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse objects for function arguments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Do dropmetricname and allocating output vector only once. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add range-aware support for range vector functions with params Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise holt_winters, cut cpu and allocs by ~25% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make rate&friends range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware. Document calling convention. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make date functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple math functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Convert more functions to be range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Specialcase timestamp() with vector selector arg for range awareness Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove transition code for functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the rest of the engine transition code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove more obselete code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the last uses of the eval* functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove engine finalizers to prevent corruption The finalizers set by matrixSelector were being called just before the value they were retruning to the pool was then being provided to the caller. Thus a concurrent query could corrupt the data that the user has just been returned. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add new benchmark suite for range functinos Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Migrate existing benchmarks to new system Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand promql benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simply test by removing unused range code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * When testing instant queries, check range queries too. To protect against subsequent steps in a range query being affected by the previous steps, add a test that evaluates an instant query that we know works again as a range query with the tiimestamp we care about not being the first step. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse ring for matrix iters. Put query results back in pool. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse buffer when iterating over matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Unary minus should remove metric name Cut down benchmarks for faster runs. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce repetition in benchmark test cases Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Work series by series when doing normal vectorSelectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise benchmark setup, cuts time by 60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Have rangeWrapper use an evalNodeHelper to cache across steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use evalNodeHelper with functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cache dropMetricName within a node evaluation. This saves both the calculations and allocs done by dropMetricName across steps. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse input vectors in rangewrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse the point slices in the matrixes input/output by rangeWrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make benchmark setup faster using AddFast Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simplify benchmark code. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add caching in VectorBinop Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use xor to have one-level resultMetric hash key Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add more benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Call Query.Close in apiv1 This allows point slices allocated for the response data to be reused by later queries, saving allocations. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise histogram_quantile It's now 5-10% faster with 97% less garbage generated for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make the input collection in rangeVector linear rather than quadratic Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_join, 1.8x faster and 11x less memory for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand benchmarks, cleanup comments, simplify numSteps logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Fabian's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Comments from Alin. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address jrv's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove dead code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Simon's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Rename populateIterators, pre-init some sizes Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Handle case where function has non-matrix args first Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Split rangeWrapper out to rangeEval function, improve comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cleanup and make things more consistent Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make EvalNodeHelper public Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Fabian's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
David King	6286c10df0	Fix OOM when a large K is used in topk queries (#4087 ) This attempts to close #3973. Handles cases where the length of the input vector to an aggregate topk / bottomk function is less than the K paramater. The change updates Prometheus to allocate a result vector the same length as the input vector in these cases. Previously Prometheus would out-of-memory panic for large K values. This change makes that unlikely unless the size of the input vector is equally large. Signed-off-by: David King <dave@davbo.org>	7 years ago
Tony Lee	7cd56f56df	add queue_time slice to query_duration_seconds (#4050 )	7 years ago
Anton Tereshchenkov	18bbec050c	promql: propagate storage errors	7 years ago
Nikunj Aggarwal	998dfcbac6	Expose itemtype outside the package (#3933 )	7 years ago
Fabian Reinartz	309c666426	Merge pull request #3671 from prometheus/queryparams *: implement query params	7 years ago
Fabian Reinartz	7ccd4b39b8	*: implement query params This adds a parameter to the storage selection interface which allows query engine(s) to pass information about the operations surrounding a data selection. This can for example be used by remote storage backends to infer the correct downsampling aggregates that need to be provided.	7 years ago
Krasi Georgiev	a53d4ed197	drop metric name for bool modifier (#3821 ) fixes #3820	7 years ago
Fabian Reinartz	f8fccc73d8	promql: remove global metrics	7 years ago

1 2 3 4

158 Commits (160186da18d8daa39b2318f9bb7263fc106847b9)