237 Commits (7622c2bc5fe92c3f33fc74898292247f91b4b987)

Author	SHA1	Message	Date
Brian Brazil	3e5136e36d	Make topk/bottomk aggregators.	9 years ago
Fabian Reinartz	4d1985e405	Merge pull request #1778 from mattbostock/fix_annotations ... promql: Fix annotations conflated with labels	9 years ago
Matt Bostock	cc98e164d3	promql: Fix annotations conflated with labels ... When converting `AlertStmt` to a string, the alert rule labels were printed as `ANNOTATIONS` instead of the annotations themselves. Fix and add a test to catch future regressions.	9 years ago
Brian Brazil	3b89616d82	Allow on, ignoring, by and without wit empty laberls. ... This offers new semantics in allowing on() for matching two single-element vectors with no known common labels. Previosuly this was often done using on(dummy). This also allows making it explict that you meant to do an aggregation without labels via by(). Fixes #1597.	9 years ago
Brian Brazil	246a817300	Flip vector matching to be ignoring by default. ... This is a noop semantically.	9 years ago
Julius Volz	b7b6717438	Separate query interface out of local.Storage. ... PromQL only requires a much narrower interface than local.Storage in order to run queries. Narrower interfaces are easier to replace and test, too. We could also change the web interface to use local.Querier, except that we'll probably use appending functions from there in the future.	9 years ago
Fabian Reinartz	0e281f5500	Merge pull request #1687 from royels/issue-1629 ... Added power binop	9 years ago
royels	2fdc5717a3	promql: add power binary operation	9 years ago
Fatih Arslan	362e44501a	promql: fix printing annotations of an *AlertStmt ... Currently the printer doesn't print the annotations of an `*AlertStmt` declaration. I've added a test case as well, which fails for the current master.	9 years ago
beorn7	e3ec8fa83b	Merge branch 'release-0.19'	9 years ago
beorn7	5408666387	Correctly stringify GROUP_x modifiers without labels ... Since rule evaluations work via String(), this fixes evaluation of rules containing GROUP_x modifiers without labels. This change is the minimal bugfix (so that we can release a fixed version without risk). It does not intend to implement any additional features (like allowing `GROUP_LEFT()` or `ON()` or even `ON` - see discussion in https://github.com/prometheus/prometheus/issues/1597 ).	9 years ago
Ali Reza	e7eba75690	remove keeping_extra because it's replaced with keep_common ... change all keepExtra label into keepCommon, and move action into removed list change incorrect token list	9 years ago
Brian Brazil	74094947ea	effect -> affect	9 years ago
Brian Brazil	68aaea618a	Merge pull request #1624 from dmitris/golint ... (trivial) fix several minor golint style issues	9 years ago
Fabian Reinartz	bbc4f11bcc	Merge pull request #945 from msiebuhr/fuzz ... Fuzz parsers	9 years ago
Dmitry Savintsev	7fdb62c253	fix several minor golint style issues	9 years ago
Morten Siebuhr	ffc8cab39a	Updates fuzzers to discard less interesting data	9 years ago
Brian Brazil	ef55fd6176	Add unittest for using a metric for thresholds with group_left.	9 years ago
Morten Siebuhr	981b636004	Bring fuzzer error handling in line.	9 years ago
Morten Siebuhr	9eb2e98509	Fix up documentation + go fmt.	9 years ago
Morten Siebuhr	7371dcc787	Fuzzing corpus for ParseMetric.	9 years ago
Morten Siebuhr	5fec020b27	Initial fuzzing corpus for ParseExpr.	9 years ago
Morten Siebuhr	0ebcca5eb7	Add basic fuzzer of the parser.	9 years ago
Brian Brazil	68e70d992a	Clarify error message around on(x) group_left(x)	9 years ago
Brian Brazil	7201c010c4	Rename On to MatchingLabels	9 years ago
Brian Brazil	d991f0cf47	For many-to-one matches, always copy label from one side. ... This is a breaking change for everyone using the machine roles labeling approach.	9 years ago
Brian Brazil	768d09fd2a	Change on+group_* to take copy from the one side. ... If the label doesn't exist on the one side, it's not copied. All labels on the many inside are included, this is a breaking change but likely low impact.	9 years ago
Brian Brazil	d1edfb25b3	Add support for OneToMany with IGNORING. ... The labels listed in the group_ modifier will be copied from the one side to the many side. It will be valid to specify no labels. This is intended to replace the existing ON/GROUP_* support.,	9 years ago
Brian Brazil	1d08c4fef0	Add 'ignoring' as modifier for binops. ... Where 'on' uses the given labels to match, 'ignoring' uses all other labels to match. group_left/right is not supported yet.	9 years ago
Brian Brazil	f5084ab1c5	Add tests for group_left/group_right	9 years ago
Fabian Reinartz	fceedfa807	Add error message if old alert rule tokens are read	9 years ago
Julius Volz	6ac39700ea	Fix missing printed keep_common without grouping.	9 years ago
Jonathan Boulle	38098f8c95	Add missing license headers ... Prometheus is Apache 2 licensed, and most source files have the appropriate copyright license header, but some were missing it without apparent reason. Correct that by adding it.	9 years ago
Fabian Reinartz	9ee91062c4	Merge pull request #1522 from prometheus/unless-operator ... Implement relative complement set operator "unless"	9 years ago
Tobias Schmidt	8cc86f25c0	Implement relative complement set operator "unless" ... The `unless` set operator can be used to return all vector elements from the LHS which do not match the elements on the RHS. A use case is to return all metrics for nodes which do not have a specific role: node_load1 unless on(instance) chef_role{role="app"}	9 years ago
Tobias Schmidt	e82ef154ee	Remove unused code leftovers	9 years ago
Tobias Schmidt	4c3dc25e35	Fix whitespace in promql test data	9 years ago
Fabian Reinartz	235e6c554b	Use ContainsRune	9 years ago
Brian Brazil	24a3ad3d16	Merge pull request #1485 from eliothedeman/master ... Adds holt-winters query function	9 years ago
eliothedeman	1543ef92b2	Adds holt-winters query function	9 years ago
beorn7	507f550cd4	Merge branch 'master' into beorn7/storage7	9 years ago
Brian Brazil	070d663948	Merge pull request #1501 from prometheus/and-dummy ... Pull in fix for and with empty labelsets	9 years ago
Fabian Reinartz	ab3d7a0ec0	Remove old alerting syntax	9 years ago
beorn7	4b574e8a61	Switch chunk encoding to type 2 where it was hardcoded type 1 before ... The chunk encoding was hardcoded there because it mostly doesn't matter what encoding is chosen in that test. Since type 1 is battle-hardened enough, I'm switching to type 2 here so that we can catch unexpected problems as a byproduct. My expectation is that the chunk encoding doesn't matter anyway, as said, but then "unexpected problems" contains the word "unexpected".	9 years ago
Brian Brazil	8788701ce7	Add test for incorrect behaviour	9 years ago
Brian Brazil	39d556f0d5	Move all the operator tests into one file	9 years ago
beorn7	99854a84d7	Merge branch 'beorn7/storage6' into beorn7/storage7	9 years ago
beorn7	d0a4477446	Merge branch 'beorn7/storage3' into beorn7/storage4 ... Conflicts: storage/local/preload.go storage/local/storage.go storage/local/storage_test.go	9 years ago
beorn7	dad302144d	Make a naked return less naked	9 years ago
beorn7	836f1db04c	Improve MetricsForLabelMatchers ... WIP: This needs more tests. It now gets a from and through value, which it may opportunistically use to optimize the retrieval. With possible future range indices, this could be used in a very efficient way. This change merely applies some easy checks, which should nevertheless solve the use case of heavy rule evaluations on servers with a lot of series churn. Idea is the following: - Only archive series that are at least as old as the headChunkTimeout (which was already extremely unlikely to happen). - Then maintain a high watermark for the last archival, i.e. no archived series has a sample more recent than that watermark. - Any query that doesn't reach to a time before that watermark doesn't have to touch the archive index at all. (A production server at Soundcloud with the aforementioned series churn and heavy rule evaluations spends 50% of its CPU time in archive index lookups. Since rule evaluations usually only touch very recent values, most of those lookup should disappear with this change.) - Federation with a very broad label matcher will profit from this, too. As a byproduct, the un-needed MetricForFingerprint method was removed from the Storage interface.	9 years ago
beorn7	f7fc542db6	Merge branch 'master' into beorn7/storage4 ... Conflicts: storage/local/persistence.go	9 years ago
beorn7	3d86130d8c	Merge branch 'master' into beorn7/storage3	9 years ago
Björn Rabenstein	2a2cc52828	Merge pull request #1405 from prometheus/beorn7/storage ... Streamline series iterator creation	9 years ago
Patrick Bogen	250344b344	use short variable assignment	9 years ago
Patrick Bogen	2062fbae0f	rewrite operator balancing to be recursive	9 years ago
beorn7	0ea5801e47	Handle errors caused by data corruption more gracefully ... This requires all the panic calls upon unexpected data to be converted into errors returned. This pollute the function signatures quite lot. Well, this is Go... The ideas behind this are the following: - panic only if it's a programming error. Data corruptions happen, and they are not programming errors. - If we detect a data corruption, we "quarantine" the series, essentially removing it from the database and putting its data into a separate directory for forensics. - Failure during writing to a series file is not considered corruption automatically. It will call setDirty, though, so that a crashrecovery upon the next restart will commence and check for that. - Series quarantining and setDirty calls are logged and counted in metrics, but are hidden from the user of the interfaces in interface.go, whith the notable exception of Append(). The reasoning is that we treat corruption by removing the corrupted series, i.e. a query for it will return no results on its next call anyway, so return no results right now. In the case of Append(), we want to tell the user that no data has been appended, though. Minor side effects: - Now consistently using filepath.* instead of path.*. - Introduced structured logging where I touched it. This makes things less consistent, but a complete change to structured logging would be out of scope for this PR.	9 years ago
beorn7	8766f99085	Merge branch 'beorn7/storage2' into beorn7/storage3	9 years ago
beorn7	162f6fa6f6	Merge branch 'beorn7/storage' into beorn7/storage2	9 years ago
beorn7	79a2ae2d2e	Add missing test file	9 years ago
beorn7	b6840997a7	Merge branch 'beorn7/storage2' into beorn7/storage3	9 years ago
beorn7	ce58fd357b	Merge branch 'beorn7/storage' into beorn7/storage2 ... Conflicts: storage/local/chunk.go storage/local/interface.go	9 years ago
beorn7	2581648f70	Separate iterators by offset ... Add test that exposes the problem.	9 years ago
Fabian Reinartz	95c9706d2d	Fix missing comment period.	9 years ago
Julius Volz	9ea2465b99	Fix typo in lexer test.	9 years ago
Tobias Schmidt	907b1380a7	Add tests to specify the string escaping behavior	9 years ago
beorn7	c740789ce3	Improve predict_linear ... Fixes https://github.com/prometheus/prometheus/issues/1401 This remove the last (and in fact bogus) use of BoundaryValues. Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can be removed here, too.	9 years ago
beorn7	454ecf3f52	Rework the way ranges and instants are handled ... In a way, our instants were also ranges, just with the staleness delta as range length. They are no treated equally, just that in one case, the range length is set as range, in the other the staleness delta. However, there are "real" instants where start and and time of a query is the same. In those cases, we only want to return a single value (the one closest before or at the equal start and end time). If that value is the last sample in the series, odds are we have it already in the series object. In that case, there is no need to pin or load any chunks. A special singleSampleSeriesIterator is created for that. This should greatly speed up instant queries as they happen frequently for rule evaluations.	9 years ago
beorn7	0e202dacb4	Streamline series iterator creation ... This will fix issue #1035 and will also help to make issue #1264 less bad. The fundamental problem in the current code: In the preload phase, we quite accurately determine which chunks will be used for the query being executed. However, in the subsequent step of creating series iterators, the created iterators are referencing _all_ in-memory chunks in their series, even the un-pinned ones. In iterator creation, we copy a pointer to each in-memory chunk of a series into the iterator. While this creates a certain amount of allocation churn, the worst thing about it is that copying the chunk pointer out of the chunkDesc requires a mutex acquisition. (Remember that the iterator will also reference un-pinned chunks, so we need to acquire the mutex to protect against concurrent eviction.) The worst case happens if a series doesn't even contain any relevant samples for the query time range. We notice that during preloading but then we will still create a series iterator for it. But even for series that do contain relevant samples, the overhead is quite bad for instant queries that retrieve a single sample from each series, but still go through all the effort of series iterator creation. All of that is particularly bad if a series has many in-memory chunks. This commit addresses the problem from two sides: First, it merges preloading and iterator creation into one step, i.e. the preload call returns an iterator for exactly the preloaded chunks. Second, the required mutex acquisition in chunkDesc has been greatly reduced. That was enabled by a side effect of the first step, which is that the iterator is only referencing pinned chunks, so there is no risk of concurrent eviction anymore, and chunks can be accessed without mutex acquisition. To simplify the code changes for the above, the long-planned change of ValueAtTime to ValueAtOrBefore time was performed at the same time. (It should have been done first, but it kind of accidentally happened while I was in the middle of writing the series iterator changes. Sorry for that.) So far, we actively filtered the up to two values that were returned by ValueAtTime, i.e. we invested work to retrieve up to two values, and then we invested more work to throw one of them away. The SeriesIterator.BoundaryValues method can be removed once #1401 is fixed. But I really didn't want to load even more changes into this PR. Benchmarks: The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times faster) and allocate 95% fewer bytes. The reason for that is that the benchmark reads one sample after another from the time series and creates a new series iterator for each sample read. To find out how much these improvements matter in practice, I have mirrored a beefy Prometheus server at SoundCloud that suffers from both issues #1035 and #1264. To reach steady state that would be comparable, the server needs to run for 15d. So far, it has run for 1d. The test server currently has only half as many memory time series and 60% of the memory chunks the main server has. The 90th percentile rule evaluation cycle time is ~11s on the main server and only ~3s on the test server. However, these numbers might get much closer over time. In addition to performance improvements, this commit removes about 150 LOC.	9 years ago
Julius Volz	9b6d69610a	Fix various typos in comments. ... Helpfully reported by https://goreportcard.com/report/github.com/prometheus/prometheus :)	9 years ago
Brian Brazil	9d0112d7cf	Add without aggregator modifier. ... This has the advantage that the user doesn't need to list all labels they want to keep (as with "by") but without having to worry about inconsistent labels as when there's only one time series (as with "keeping_common"). Almost all aggregation should use this rather than the existing two options as it's much less error prone and easier to maintain due to not having to always add in "job" plus whatever other common job-level labels you have like "region".	9 years ago
Brian Brazil	b7ef0b45e8	Break aggregation tests out. Add missing tests.	9 years ago
beorn7	a7408bfb47	Unify duration parsing ... It's actually happening in several places (and for flags, we use the standard Go time.Duration...). This at least reduces all our home-grown parsing to one place (in model).	9 years ago
Fabian Reinartz	a6935024e1	Remove old WITH clause in alert printing	9 years ago
Tobias Schmidt	1a91cd6e09	Rename matrix to range selector in external error messages ... The documentation speaks about range vectors and range vector selectors. This change does not fix all issues, we might still expose the term "Matrix" in error messages using %T.	9 years ago
Tobias Schmidt	411ca4dba1	Consolidate offset modifier parsing ... Remove duplicated offset modifier parsing and ensure offset can only appear at the end of a selector statement.	9 years ago
Fabian Reinartz	6b4a6962d2	Support old alerting rule syntax	9 years ago
Brian Brazil	c77c3a8c56	promql: Limit extrapolation of delta/rate/increase ... The new implementation detects the start and end of a series by looking at the average sample interval within the range. If the first (last) sample in the range is more than 1.1*interval distant from the beginning (end) of the range, it is considered the first (last) sample of the series as a whole, and extrapolation is limited to half the interval (rather than all the way to the beginning (end) of the range). In addition, if the extrapolated starting point of a counter (where it is zero) is within the range, it is used as the starting point of the series. Fixes #581	9 years ago
Brian Brazil	89760dd77d	Handle NaN for min/max. ... Similar to topk and sort, prefer not returning NaN where possible.	9 years ago
Brian Brazil	bac1f28cad	Similar to topk/bottomk, have sort/sort_desc put NaN at end. ... This makes topk and bottomk consistent with the sorting functions, as per #1271.	9 years ago
Fabian Reinartz	4209ec6864	Change WITH keyword to LABELS	9 years ago
Brian Brazil	88ca82304c	Make topk/bottomk prefer returning real numbers over NaN.	9 years ago
Brian Brazil	edf3e123f5	Move topk/bottomk tests from legacy.	9 years ago
Fabian Reinartz	af3a6661ed	Implement new alerting rule syntax	9 years ago
James Sanford	5b53262b7a	promql: Add clamp_max/clamp_min functions.	9 years ago
Brian Brazil	a287264989	Print offsets in promql.	9 years ago
Fabian Reinartz	33aab4169c	Anchor regexes in vector matching ... This commit makes the regex behavior of vector matching consistent with configuration and label_replace() by anchoring it. Fixes #1200	9 years ago
Fabian Reinartz	51e8badc7f	Merge pull request #1159 from prometheus/scalar-bool ... promql: Remove scalar/scalar comparisons.	9 years ago
Brian Brazil	c36961130b	promql: Remove scalar/scalar comparisons. ... This change is breaking, use the 'bool' modifier for such comprisons. After this change all comparisons without 'bool' will filter, and all comparisons with 'bool' will return 0/1. This makes the language more consistent and orthogonal, and ultimately easier to learn and use. If we ever figure out sane semantics for filtering scalar/scalar comparisons we can add them in, which will most likely come out of how the new vector() function is used.	9 years ago
Brian Brazil	5740a8fade	promql: Remove deprecated 2nd argument to delta() ... This change is breaking, use increase() instead. I'm not cleaning up the function in this PR, as my solution to #581 will rewrite and simplify increase/rate/delta.	9 years ago
Brian Brazil	965a71dc4d	Merge pull request #1155 from prometheus/irate ... promql: Add irate() function	9 years ago
Brian Brazil	f08abdb48b	promql: Add irate() function ... irate is a rate function that only looks at the most recent two data points, and calucaltes a per-second value from that. This produces much more granular graphs for fast moving data, and works sanely across many scrape intervals. It doesn't do so well for slowly moving data.	9 years ago
Julius Volz	0088aa4d45	Merge pull request #1132 from prometheus/fix-quoting-and-escaping ... Support escape sequences in strings and add raw strings	9 years ago
Julius Volz	46c5260761	Support escape sequences in strings and add raw strings. ... This adapts some functionality from the Go standard library for string literal lexing and unquoting/unescaping. The following string types are now supported: Double- or single-quoted strings: These support all escape sequences that Go supports in double-quoted string literals. The difference is that Prometheus also has single-quoted strings (instead of single-quoted runes in Go). Raw newlines are not allowed. Backtick-quoted raw strings: Strings quoted in backticks are treated as raw strings just like in Go and may contain raw newlines and other special characters directly. Fixes https://github.com/prometheus/prometheus/issues/1122 Fixes https://github.com/prometheus/prometheus/issues/1121	9 years ago
Fabian Reinartz	e3b6ec9784	Switch to common/log	9 years ago
Brian Brazil	653ff71f1f	promql: Reduce flakiness of concurrency test	9 years ago
Fabian Reinartz	171f50706a	Fix unkeyed field errors.	9 years ago
Fabian Reinartz	36ec8ba460	Fix missing return on error	9 years ago
Fabian Reinartz	e005f939fd	Fix scalar construction in function	9 years ago
Fabian Reinartz	eca41f5319	Run gofmt	9 years ago
Brian Brazil	fa793d917e	Merge pull request #1080 from prometheus/query-timeout-test ... promql: Bump sleep in query timeout test	9 years ago