prometheus

Commit Graph

Author	SHA1	Message	Date
Darshan Chaudhary	c4f2e9eec5	Add present_over_time (#9097 ) * Add present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Add tests for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Address PR comments Signed-off-by: darshanime <deathbullet@gmail.com> * Add documentation for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation comment Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
songjiayang	776c2b8f42	Speed delta value without loop to calculate resultValue Signed-off-by: songjiayang <songjiayang1@gmail.com>	4 years ago
pschou	aff3c702ab	promql: Add sgn, clamp and last_over_time functions (#8457 ) * Add sgn, clamp and last_over_time functions Signed-off-by: schou <pschou@users.noreply.github.com>	4 years ago
Ganesh Vernekar	9199fcb8d1	'@ <timestamp>' modifier (#8121 ) This commit adds `@ <timestamp>` modifier as per this design doc: https://docs.google.com/document/d/1uSbD3T2beM-iX4-Hp7V074bzBRiRNlqUdcWP6JTDQSs/edit. An example query: ``` rate(process_cpu_seconds_total[1m]) and topk(7, rate(process_cpu_seconds_total[1h] @ 1234)) ``` which ranks based on last 1h rate and w.r.t. unix timestamp 1234 but actually plots the 1m rate. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Vijay Samuel	00ee73ef91	Export members of EvalNodeHelper to facilitate usage in external functions (#7860 ) Signed-off-by: Vijay Samuel <vjsamuel@ebay.com>	4 years ago
Julien Pivotto	d77b56e88e	Fix avg_over_time for nan and float64 overflows (#7346 ) * Fix avg_over_time with Inf and NaN values Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Callum Styan	5bb7f00d00	change labelset comparison in promql engine to avoid false positive during detection of duplicates (#7058 ) * Use go1.14 new hash/maphash to hash both RHS and LHS instead of XOR'ing which has been resulting in hash collisions. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Refactor engine labelset signature generation, just use labels.Labels instead of hashes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments; function comments + store result of lhs.String+rhs.String as key. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Replace all signatureFunc usage with signatureFuncString. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Make optimizations to labels String function and generation of rhs+lhs as string in resultMetric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use separate string functions that don't use strconv just for engine maps. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use a byte invalid separator instead of quoting and have a buffer attached to EvalNodeHelper instead of using a global pool in the labels package. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address more review comments, labels has a function that now builds a byte slice without turning it into a string. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use two different non-ascii hex codes as byte separators between labels and between sets of labels when building bytes of a Labels struct. Signed-off-by: Callum Styan <callumstyan@gmail.com> * We only need the 2nd byte invalid sep. at the beginning of a labels.Bytes Signed-off-by: Callum Styan <callumstyan@gmail.com>	5 years ago
Ben Kochie	269e7c8091	Fix golint issues. Signed-off-by: Ben Kochie <superq@gmail.com>	5 years ago
李国忠	2bf4952049	remove Unused parameter 'sf' in calcTrendValue function (#6900 ) Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	5 years ago
Tobias Guggenmos	b93845d6bf	Remove unused function Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	1360f9ff12	Fix all build errors in promql package Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	9a1366775e	Store function implementations independently of their signatures Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	ff0ea1c1ac	Fix more identifiers Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	6b1b323558	Export sequenceValue Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	228967a507	Fix usages of more things that have moved the package Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	2f1113479f	Fix usages of ValueType Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	2aacd807b3	PromQL: Various small improvements in the parser (#6652 ) * Move check for empty VectorSelector to typeChecking * Move check for twice set metric name to typeChecking * Make child of MatrixSelector a general Node * rename checkType to checkAST * Rename fail to addParseErr * Remove trailing whitespace Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	64194f7d45	PromQL: AST: Make VectorSelector Children of MatrixSelector (#6590 ) Make Vector selectors children of Matrix Selectors Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Julien Pivotto	577e738986	Cleanup PromQL functions (#6551 ) * Cleanup PromQL functions The engine ensures, for Matrix functions, that functions are called with exactly one series at the time. Therefore a lot of code can be inlined and we can directly assume the first element of the arguments exists and contains all the samples needed. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	e0afec906f	add absent_over_time (#6490 ) * Implement absent_over_time Fixes #2882 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	f363a71ef7	promql: function: fix typo (#6501 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	4b3bab890d	Fix absent output where two label matchers are identical (#6493 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Tobias Guggenmos	cd39ebe7c6	PromQL: export function list (#6434 ) This PR exports the list of supported PromQL functions and their signatures. The reason for that is that the PromQL language server likes to use that list. Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Ben Kochie	c6399296dc	Fix spelling/typos (#4921 ) * Fix spelling/typos Fix spelling/typos reported by codespell/misspell. * UK -> US spelling changes. Signed-off-by: Ben Kochie <superq@gmail.com>	6 years ago
Thomas Jackson	abf6fe0a98	Change max/min over_time to handle NaNs properly (#4386 ) We only want to return a NaN if the NaN is the only value Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4385	6 years ago
Harsh Agarwal	18a9a390b5	Add duplicate-labelset check for range/instant vectors (#4589 ) Signed-off-by: Harsh Agarwal <cs15btech11019@iith.ac.in>	6 years ago
Dan Cech	9f4cb06a37	use Welford/Knuth method to compute standard deviation and variance (#4533 ) * use Welford/Knuth method to compute standard deviation and variance, avoids float precision issues * use better method for calculating avg and avg_over_time Signed-off-by: Dan Cech <dcech@grafana.com>	6 years ago
Benjamin Raskin	9353696d77	Fix spelling and holt-winters check (#4424 ) Signed-off-by: Benjamin Raskin <braskin@uber.com>	6 years ago
Mario Trangoni	0e2aa35771	promql: fix unconvert issues (#4040 ) See, $ gometalinter --vendor --disable-all --enable=unconvert --deadline 6m ./... promql/engine.go:1396:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1396:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1505:50⚠️ unnecessary conversion (unconvert) promql/engine.go:1573:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1578:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1591:80⚠️ unnecessary conversion (unconvert) promql/engine.go:1602:94⚠️ unnecessary conversion (unconvert) promql/engine.go:1630:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1631:24⚠️ unnecessary conversion (unconvert) promql/engine.go:1634:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1635:34⚠️ unnecessary conversion (unconvert) promql/functions.go:302:42⚠️ unnecessary conversion (unconvert) promql/functions.go:315:42⚠️ unnecessary conversion (unconvert) promql/functions.go:334:26⚠️ unnecessary conversion (unconvert) promql/functions.go:395:31⚠️ unnecessary conversion (unconvert) promql/functions.go:406:31⚠️ unnecessary conversion (unconvert) promql/functions.go:454:27⚠️ unnecessary conversion (unconvert) promql/functions.go:701:46⚠️ unnecessary conversion (unconvert) promql/functions.go:701:78⚠️ unnecessary conversion (unconvert) promql/functions.go:730:43⚠️ unnecessary conversion (unconvert) promql/functions.go:1220:23⚠️ unnecessary conversion (unconvert) promql/functions.go:1249:23⚠️ unnecessary conversion (unconvert) promql/quantile.go:107:54⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:16⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:64⚠️ unnecessary conversion (unconvert) Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>	7 years ago
Brian Brazil	dd6781add2	Optimise PromQL (#3966 ) * Move range logic to 'eval' Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make aggregegate range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * PromQL is statically typed, so don't eval to find the type. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Extend rangewrapper to multiple exprs Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Start making function evaluation ranged Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make instant queries a special case of range queries Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Eliminate evalString Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Evaluate range vector functions one series at a time Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make unary operators range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make binops range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Pass time to range-aware functions. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple _over_time functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce allocs when working with matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add basic benchmark for range evaluation Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse objects for function arguments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Do dropmetricname and allocating output vector only once. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add range-aware support for range vector functions with params Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise holt_winters, cut cpu and allocs by ~25% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make rate&friends range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware. Document calling convention. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make date functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple math functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Convert more functions to be range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Specialcase timestamp() with vector selector arg for range awareness Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove transition code for functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the rest of the engine transition code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove more obselete code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the last uses of the eval* functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove engine finalizers to prevent corruption The finalizers set by matrixSelector were being called just before the value they were retruning to the pool was then being provided to the caller. Thus a concurrent query could corrupt the data that the user has just been returned. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add new benchmark suite for range functinos Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Migrate existing benchmarks to new system Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand promql benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simply test by removing unused range code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * When testing instant queries, check range queries too. To protect against subsequent steps in a range query being affected by the previous steps, add a test that evaluates an instant query that we know works again as a range query with the tiimestamp we care about not being the first step. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse ring for matrix iters. Put query results back in pool. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse buffer when iterating over matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Unary minus should remove metric name Cut down benchmarks for faster runs. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce repetition in benchmark test cases Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Work series by series when doing normal vectorSelectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise benchmark setup, cuts time by 60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Have rangeWrapper use an evalNodeHelper to cache across steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use evalNodeHelper with functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cache dropMetricName within a node evaluation. This saves both the calculations and allocs done by dropMetricName across steps. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse input vectors in rangewrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse the point slices in the matrixes input/output by rangeWrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make benchmark setup faster using AddFast Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simplify benchmark code. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add caching in VectorBinop Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use xor to have one-level resultMetric hash key Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add more benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Call Query.Close in apiv1 This allows point slices allocated for the response data to be reused by later queries, saving allocations. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise histogram_quantile It's now 5-10% faster with 97% less garbage generated for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make the input collection in rangeVector linear rather than quadratic Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_join, 1.8x faster and 11x less memory for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand benchmarks, cleanup comments, simplify numSteps logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Fabian's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Comments from Alin. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address jrv's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove dead code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Simon's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Rename populateIterators, pre-init some sizes Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Handle case where function has non-matrix args first Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Split rangeWrapper out to rangeEval function, improve comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cleanup and make things more consistent Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make EvalNodeHelper public Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Fabian's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Krasi Georgiev	4801573b64	time() return milliseconds (#3811 )	7 years ago
Brian Brazil	7158675aa8	Add back continue. Accidentally removed in `15a931dbdb`	7 years ago
Brian Brazil	b2ac3d2d86	Remove count_scalar and drop_common_labels. For #3060	7 years ago
Fabian Reinartz	87918f3097	Merge branch 'master' into dev-2.0	7 years ago
Brian Brazil	2354c2544b	Set timestamp for date functions (#3070 )	7 years ago
Fabian Reinartz	25f3e1c424	Merge branch 'master' into mergemaster	7 years ago
Brian Brazil	4c8173acac	Use timestamp of a sample in deriv() to avoid FP issues (#2958 ) With the squaring of the timestamp, we run into the limitations of the 53bit mantissa for a 64bit float. By subtracting away a timestamp of one of the samples (which is how the intercept is used) we avoid this issue in practice as it's unlikely that it is used over a very long time range. Fixes #2674	7 years ago
Alexey Palazhchenko	695ec0b981	Fix few typos. (#2962 )	7 years ago
Fabian Reinartz	ca2b68889b	Merge branch 'master' into dev-2.0	8 years ago
Harsh Agarwal	16867c89a7	implement label_join issue 1147 (#2806 ) Replace OptionalArgs int with Variadic int.	8 years ago
Brian Brazil	517b81f927	Add timestamp() function. Make the timestamp of instant vectors be the timestamp of the sample rather than the evaluation. We were not using this anywhere, so this is safe. Add a function to return the timestamp of samples in an instant vector. Fixes #1557	8 years ago
Fabian Reinartz	035976b275	retrieval: handle not found error correctly	8 years ago
Fabian Reinartz	ad9bc62e4c	storage: extend appender and adapt it	8 years ago
Fabian Reinartz	71fe0c58a8	promql: misc fixes	8 years ago
Fabian Reinartz	fecf9532b9	*: fix misc compile errors	8 years ago
Fabian Reinartz	9ea10d5265	promql: use labels.Builder to modify labels	8 years ago
Fabian Reinartz	c6cd998905	promql: use local labels, add conversion	8 years ago
Fabian Reinartz	ff504af2aa	promql: undo accidental exports	8 years ago
Fabian Reinartz	c5f225b920	promql: export Sample	8 years ago
Fabian Reinartz	65581a3d46	promql: export SmapleStream	8 years ago
Fabian Reinartz	6315d00942	promql: export String value	8 years ago
Fabian Reinartz	ac5d3bc05e	promql: scalar T/V and Point	8 years ago
Fabian Reinartz	09666e2e2a	promql: make scalar public	8 years ago
Fabian Reinartz	b3f71df350	promql: make matrix exported	8 years ago
Fabian Reinartz	a62df87022	promql: rename vector	8 years ago
Fabian Reinartz	15a931dbdb	promql: migrate model types, use tsdb interfaces	8 years ago
Fabian Reinartz	ccbce0c51f	promql: handle NaN in changes() correctly	8 years ago
Matt Bostock	9628eb5998	PromQL: Add minute() function Returns the minutes from the current time in UTC. Related to the `hour()` function. Fixes #1983.	8 years ago
Fabian Reinartz	ab88057063	Merge pull request #1908 from prometheus/on-dates Add various time and date functions	8 years ago
Brian Brazil	4680daf237	Default date functions to current time.	8 years ago
beorn7	71571a8ec4	promql: Fix (and simplify) populating iterators This was only relevant so far for the benchmark suite as it would recycle Expr for repetitions. However, the append is unnecessary as each node is only inspected once when populating iterators, and population must always start from scratch. This also introduces error checking during benchmarks and fixes the so far undetected test errors during benchmarking. Also, remove a style nit (two golint warnings less…).	8 years ago
Brian Brazil	ea1318f38b	Short names of some date related functions	8 years ago
Brian Brazil	d2ca2b496a	Add days_in_month function.	8 years ago
Brian Brazil	0ed31c8c47	Sort list of functions.	8 years ago
Brian Brazil	fd7822829c	Add date related functions. Add day_of_month, day_of_week, hour_of_day, month_of_year and year. This only work for UTC, and ignore leap seconds the same as Go.	8 years ago
Fabian Stäber	08b6556ee6	Assume counters start at zero after reset.	8 years ago
Frederic Branczyk	f02df4138c	refactor duplication of irate and idelta functions implementations	8 years ago
Frederic Branczyk	dbf83666bb	add idelta function similar to the irate function the idelta function calculates the delta function with the last two values	8 years ago
Brian Brazil	0303ccc6a7	Add quantile aggregator.	8 years ago
Brian Brazil	15f9fe0a45	Factor out quantile fucntion.	8 years ago
Brian Brazil	b0342ba9ec	Add quantile_over_time function	8 years ago
Brian Brazil	1edd6875f5	Add stddev_over_time and stdvar_over_time.	8 years ago
Brian Brazil	3e5136e36d	Make topk/bottomk aggregators.	9 years ago
Brian Brazil	74094947ea	effect -> affect	9 years ago
eliothedeman	1543ef92b2	Adds holt-winters query function	9 years ago
beorn7	dad302144d	Make a naked return less naked	9 years ago
beorn7	c740789ce3	Improve predict_linear Fixes https://github.com/prometheus/prometheus/issues/1401 This remove the last (and in fact bogus) use of BoundaryValues. Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can be removed here, too.	9 years ago
beorn7	0e202dacb4	Streamline series iterator creation This will fix issue #1035 and will also help to make issue #1264 less bad. The fundamental problem in the current code: In the preload phase, we quite accurately determine which chunks will be used for the query being executed. However, in the subsequent step of creating series iterators, the created iterators are referencing _all_ in-memory chunks in their series, even the un-pinned ones. In iterator creation, we copy a pointer to each in-memory chunk of a series into the iterator. While this creates a certain amount of allocation churn, the worst thing about it is that copying the chunk pointer out of the chunkDesc requires a mutex acquisition. (Remember that the iterator will also reference un-pinned chunks, so we need to acquire the mutex to protect against concurrent eviction.) The worst case happens if a series doesn't even contain any relevant samples for the query time range. We notice that during preloading but then we will still create a series iterator for it. But even for series that do contain relevant samples, the overhead is quite bad for instant queries that retrieve a single sample from each series, but still go through all the effort of series iterator creation. All of that is particularly bad if a series has many in-memory chunks. This commit addresses the problem from two sides: First, it merges preloading and iterator creation into one step, i.e. the preload call returns an iterator for exactly the preloaded chunks. Second, the required mutex acquisition in chunkDesc has been greatly reduced. That was enabled by a side effect of the first step, which is that the iterator is only referencing pinned chunks, so there is no risk of concurrent eviction anymore, and chunks can be accessed without mutex acquisition. To simplify the code changes for the above, the long-planned change of ValueAtTime to ValueAtOrBefore time was performed at the same time. (It should have been done first, but it kind of accidentally happened while I was in the middle of writing the series iterator changes. Sorry for that.) So far, we actively filtered the up to two values that were returned by ValueAtTime, i.e. we invested work to retrieve up to two values, and then we invested more work to throw one of them away. The SeriesIterator.BoundaryValues method can be removed once #1401 is fixed. But I really didn't want to load even more changes into this PR. Benchmarks: The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times faster) and allocate 95% fewer bytes. The reason for that is that the benchmark reads one sample after another from the time series and creates a new series iterator for each sample read. To find out how much these improvements matter in practice, I have mirrored a beefy Prometheus server at SoundCloud that suffers from both issues #1035 and #1264. To reach steady state that would be comparable, the server needs to run for 15d. So far, it has run for 1d. The test server currently has only half as many memory time series and 60% of the memory chunks the main server has. The 90th percentile rule evaluation cycle time is ~11s on the main server and only ~3s on the test server. However, these numbers might get much closer over time. In addition to performance improvements, this commit removes about 150 LOC.	9 years ago
Brian Brazil	c77c3a8c56	promql: Limit extrapolation of delta/rate/increase The new implementation detects the start and end of a series by looking at the average sample interval within the range. If the first (last) sample in the range is more than 1.1*interval distant from the beginning (end) of the range, it is considered the first (last) sample of the series as a whole, and extrapolation is limited to half the interval (rather than all the way to the beginning (end) of the range). In addition, if the extrapolated starting point of a counter (where it is zero) is within the range, it is used as the starting point of the series. Fixes #581	9 years ago
Brian Brazil	bac1f28cad	Similar to topk/bottomk, have sort/sort_desc put NaN at end. This makes topk and bottomk consistent with the sorting functions, as per #1271.	9 years ago
Brian Brazil	88ca82304c	Make topk/bottomk prefer returning real numbers over NaN.	9 years ago
James Sanford	5b53262b7a	promql: Add clamp_max/clamp_min functions.	9 years ago
Brian Brazil	5740a8fade	promql: Remove deprecated 2nd argument to delta() This change is breaking, use increase() instead. I'm not cleaning up the function in this PR, as my solution to #581 will rewrite and simplify increase/rate/delta.	9 years ago
Brian Brazil	f08abdb48b	promql: Add irate() function irate is a rate function that only looks at the most recent two data points, and calucaltes a per-second value from that. This produces much more granular graphs for fast moving data, and works sanely across many scrape intervals. It doesn't do so well for slowly moving data.	9 years ago
Fabian Reinartz	e005f939fd	Fix scalar construction in function	9 years ago
Fabian Reinartz	eca41f5319	Run gofmt	9 years ago
Brian Brazil	9b382647b5	Remove optional vector() 2nd argument	9 years ago
Brian Brazil	69f5fa0c1e	promql: Add vector function. Currently the only way to convert a scalar to a vector is to use absent(), which isn't very clean. This adds a vector() function that's the inverse of scalar() and lets your optionally set labels. Example usage would be vector(time() % 86400) < 3600 to filter to only the first hour of the day.	9 years ago
Julius Volz	61c42c8da0	Change relabel_replace() to do full-string matches. THIS IS A BREAKING CHANGE. Fixes part of https://github.com/prometheus/prometheus/issues/996	9 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	9 years ago
Fabian Reinartz	1535ef1457	Replace metric.SamplePair with model.SamplePair	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
Julius Volz	27ed874358	Implement label_replace() Implements part of https://github.com/prometheus/prometheus/issues/959.	9 years ago
Brian Brazil	d6a80c2b76	promql: Add support for predict_linear(my_timeseries[1h], 7200) This will give a prediction for the value of my_timeseries in 2 hours, based on the last hour of data.	9 years ago
Brian Brazil	f34de493d5	Add increase() function, to replace delta(..., 1). This calculates how much a counter increases over a given period of time, which is the area under the curve of it's rate. increase(x[5m]) is equivilent to rate(x[5m]) * 300.	10 years ago
Julius Volz	d44a89c6e8	Implement changes() function. changes() takes a range vector and returns the number of times a value has changed in the given time window for each time series as an instant vector.	10 years ago
Julius Volz	6f33ed9e59	Add resets() function to count counter resets. resets() returns for every range vector element how many counter resets there have been in the specified range.	10 years ago
Fabian Reinartz	5602328c7c	Refactor query evaluation. This copies the evaluation logic from the current rules/ package. The new engine handles the execution process from query string to final result. It provides query timeout and cancellation and general flexibility for future changes. functions.go: Add evaluation implementation. Slight changes to in/out data but not to the processing logic. quantile.go: No changes. analyzer.go: No changes. engine.go: Actually new part. Mainly consists of evaluation methods which were not changed. setup_test.go: Copy of rules/helpers_test.go to setup test storage. promql_test.go: Copy of rules/rules_test.go.	10 years ago
Fabian Reinartz	32b7595c47	Create promql package with lexer/parser. This commit creates a (so far unused) package. It contains the a custom lexer/parser for the query language. ast.go: New AST that interacts well with the parser. lex.go: Custom lexer (new). lex_test.go: Lexer tests (new). parse.go: Custom parser (new). parse_test.go: Parser tests (new). functions.go: Changed function type, dummies for parser testing (barely changed/dummies). printer.go: Adapted from rules/ and adjusted to new AST (mostly unchanged, few additions).	10 years ago

1 2 3 4

200 Commits (a425dbfa72490c599ea235a0ef2278e56b40c005)