prometheus

Commit Graph

Author	SHA1	Message	Date
Morten Siebuhr	0ebcca5eb7	Add basic fuzzer of the parser.	9 years ago
Brian Brazil	68e70d992a	Clarify error message around on(x) group_left(x)	9 years ago
Brian Brazil	7201c010c4	Rename On to MatchingLabels	9 years ago
Brian Brazil	d991f0cf47	For many-to-one matches, always copy label from one side. This is a breaking change for everyone using the machine roles labeling approach.	9 years ago
Brian Brazil	768d09fd2a	Change on+group_* to take copy from the one side. If the label doesn't exist on the one side, it's not copied. All labels on the many inside are included, this is a breaking change but likely low impact.	9 years ago
Brian Brazil	d1edfb25b3	Add support for OneToMany with IGNORING. The labels listed in the group_ modifier will be copied from the one side to the many side. It will be valid to specify no labels. This is intended to replace the existing ON/GROUP_* support.,	9 years ago
Brian Brazil	1d08c4fef0	Add 'ignoring' as modifier for binops. Where 'on' uses the given labels to match, 'ignoring' uses all other labels to match. group_left/right is not supported yet.	9 years ago
Brian Brazil	f5084ab1c5	Add tests for group_left/group_right	9 years ago
Fabian Reinartz	fceedfa807	Add error message if old alert rule tokens are read	9 years ago
Julius Volz	6ac39700ea	Fix missing printed keep_common without grouping.	9 years ago
Jonathan Boulle	38098f8c95	Add missing license headers Prometheus is Apache 2 licensed, and most source files have the appropriate copyright license header, but some were missing it without apparent reason. Correct that by adding it.	9 years ago
Fabian Reinartz	9ee91062c4	Merge pull request #1522 from prometheus/unless-operator Implement relative complement set operator "unless"	9 years ago
Tobias Schmidt	8cc86f25c0	Implement relative complement set operator "unless" The `unless` set operator can be used to return all vector elements from the LHS which do not match the elements on the RHS. A use case is to return all metrics for nodes which do not have a specific role: node_load1 unless on(instance) chef_role{role="app"}	9 years ago
Tobias Schmidt	e82ef154ee	Remove unused code leftovers	9 years ago
Tobias Schmidt	4c3dc25e35	Fix whitespace in promql test data	9 years ago
Fabian Reinartz	235e6c554b	Use ContainsRune	9 years ago
Brian Brazil	24a3ad3d16	Merge pull request #1485 from eliothedeman/master Adds holt-winters query function	9 years ago
eliothedeman	1543ef92b2	Adds holt-winters query function	9 years ago
beorn7	507f550cd4	Merge branch 'master' into beorn7/storage7	9 years ago
Brian Brazil	070d663948	Merge pull request #1501 from prometheus/and-dummy Pull in fix for and with empty labelsets	9 years ago
Fabian Reinartz	ab3d7a0ec0	Remove old alerting syntax	9 years ago
beorn7	4b574e8a61	Switch chunk encoding to type 2 where it was hardcoded type 1 before The chunk encoding was hardcoded there because it mostly doesn't matter what encoding is chosen in that test. Since type 1 is battle-hardened enough, I'm switching to type 2 here so that we can catch unexpected problems as a byproduct. My expectation is that the chunk encoding doesn't matter anyway, as said, but then "unexpected problems" contains the word "unexpected".	9 years ago
Brian Brazil	8788701ce7	Add test for incorrect behaviour	9 years ago
Brian Brazil	39d556f0d5	Move all the operator tests into one file	9 years ago
beorn7	99854a84d7	Merge branch 'beorn7/storage6' into beorn7/storage7	9 years ago
beorn7	d0a4477446	Merge branch 'beorn7/storage3' into beorn7/storage4 Conflicts: storage/local/preload.go storage/local/storage.go storage/local/storage_test.go	9 years ago
beorn7	dad302144d	Make a naked return less naked	9 years ago
beorn7	836f1db04c	Improve MetricsForLabelMatchers WIP: This needs more tests. It now gets a from and through value, which it may opportunistically use to optimize the retrieval. With possible future range indices, this could be used in a very efficient way. This change merely applies some easy checks, which should nevertheless solve the use case of heavy rule evaluations on servers with a lot of series churn. Idea is the following: - Only archive series that are at least as old as the headChunkTimeout (which was already extremely unlikely to happen). - Then maintain a high watermark for the last archival, i.e. no archived series has a sample more recent than that watermark. - Any query that doesn't reach to a time before that watermark doesn't have to touch the archive index at all. (A production server at Soundcloud with the aforementioned series churn and heavy rule evaluations spends 50% of its CPU time in archive index lookups. Since rule evaluations usually only touch very recent values, most of those lookup should disappear with this change.) - Federation with a very broad label matcher will profit from this, too. As a byproduct, the un-needed MetricForFingerprint method was removed from the Storage interface.	9 years ago
beorn7	f7fc542db6	Merge branch 'master' into beorn7/storage4 Conflicts: storage/local/persistence.go	9 years ago
beorn7	3d86130d8c	Merge branch 'master' into beorn7/storage3	9 years ago
Björn Rabenstein	2a2cc52828	Merge pull request #1405 from prometheus/beorn7/storage Streamline series iterator creation	9 years ago
Patrick Bogen	250344b344	use short variable assignment	9 years ago
Patrick Bogen	2062fbae0f	rewrite operator balancing to be recursive	9 years ago
beorn7	0ea5801e47	Handle errors caused by data corruption more gracefully This requires all the panic calls upon unexpected data to be converted into errors returned. This pollute the function signatures quite lot. Well, this is Go... The ideas behind this are the following: - panic only if it's a programming error. Data corruptions happen, and they are not programming errors. - If we detect a data corruption, we "quarantine" the series, essentially removing it from the database and putting its data into a separate directory for forensics. - Failure during writing to a series file is not considered corruption automatically. It will call setDirty, though, so that a crashrecovery upon the next restart will commence and check for that. - Series quarantining and setDirty calls are logged and counted in metrics, but are hidden from the user of the interfaces in interface.go, whith the notable exception of Append(). The reasoning is that we treat corruption by removing the corrupted series, i.e. a query for it will return no results on its next call anyway, so return no results right now. In the case of Append(), we want to tell the user that no data has been appended, though. Minor side effects: - Now consistently using filepath.* instead of path.*. - Introduced structured logging where I touched it. This makes things less consistent, but a complete change to structured logging would be out of scope for this PR.	9 years ago
beorn7	8766f99085	Merge branch 'beorn7/storage2' into beorn7/storage3	9 years ago
beorn7	162f6fa6f6	Merge branch 'beorn7/storage' into beorn7/storage2	9 years ago
beorn7	79a2ae2d2e	Add missing test file	9 years ago
beorn7	b6840997a7	Merge branch 'beorn7/storage2' into beorn7/storage3	9 years ago
beorn7	ce58fd357b	Merge branch 'beorn7/storage' into beorn7/storage2 Conflicts: storage/local/chunk.go storage/local/interface.go	9 years ago
beorn7	2581648f70	Separate iterators by offset Add test that exposes the problem.	9 years ago
Fabian Reinartz	95c9706d2d	Fix missing comment period.	9 years ago
Julius Volz	9ea2465b99	Fix typo in lexer test.	9 years ago
Tobias Schmidt	907b1380a7	Add tests to specify the string escaping behavior	9 years ago
beorn7	c740789ce3	Improve predict_linear Fixes https://github.com/prometheus/prometheus/issues/1401 This remove the last (and in fact bogus) use of BoundaryValues. Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can be removed here, too.	9 years ago
beorn7	454ecf3f52	Rework the way ranges and instants are handled In a way, our instants were also ranges, just with the staleness delta as range length. They are no treated equally, just that in one case, the range length is set as range, in the other the staleness delta. However, there are "real" instants where start and and time of a query is the same. In those cases, we only want to return a single value (the one closest before or at the equal start and end time). If that value is the last sample in the series, odds are we have it already in the series object. In that case, there is no need to pin or load any chunks. A special singleSampleSeriesIterator is created for that. This should greatly speed up instant queries as they happen frequently for rule evaluations.	9 years ago
beorn7	0e202dacb4	Streamline series iterator creation This will fix issue #1035 and will also help to make issue #1264 less bad. The fundamental problem in the current code: In the preload phase, we quite accurately determine which chunks will be used for the query being executed. However, in the subsequent step of creating series iterators, the created iterators are referencing _all_ in-memory chunks in their series, even the un-pinned ones. In iterator creation, we copy a pointer to each in-memory chunk of a series into the iterator. While this creates a certain amount of allocation churn, the worst thing about it is that copying the chunk pointer out of the chunkDesc requires a mutex acquisition. (Remember that the iterator will also reference un-pinned chunks, so we need to acquire the mutex to protect against concurrent eviction.) The worst case happens if a series doesn't even contain any relevant samples for the query time range. We notice that during preloading but then we will still create a series iterator for it. But even for series that do contain relevant samples, the overhead is quite bad for instant queries that retrieve a single sample from each series, but still go through all the effort of series iterator creation. All of that is particularly bad if a series has many in-memory chunks. This commit addresses the problem from two sides: First, it merges preloading and iterator creation into one step, i.e. the preload call returns an iterator for exactly the preloaded chunks. Second, the required mutex acquisition in chunkDesc has been greatly reduced. That was enabled by a side effect of the first step, which is that the iterator is only referencing pinned chunks, so there is no risk of concurrent eviction anymore, and chunks can be accessed without mutex acquisition. To simplify the code changes for the above, the long-planned change of ValueAtTime to ValueAtOrBefore time was performed at the same time. (It should have been done first, but it kind of accidentally happened while I was in the middle of writing the series iterator changes. Sorry for that.) So far, we actively filtered the up to two values that were returned by ValueAtTime, i.e. we invested work to retrieve up to two values, and then we invested more work to throw one of them away. The SeriesIterator.BoundaryValues method can be removed once #1401 is fixed. But I really didn't want to load even more changes into this PR. Benchmarks: The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times faster) and allocate 95% fewer bytes. The reason for that is that the benchmark reads one sample after another from the time series and creates a new series iterator for each sample read. To find out how much these improvements matter in practice, I have mirrored a beefy Prometheus server at SoundCloud that suffers from both issues #1035 and #1264. To reach steady state that would be comparable, the server needs to run for 15d. So far, it has run for 1d. The test server currently has only half as many memory time series and 60% of the memory chunks the main server has. The 90th percentile rule evaluation cycle time is ~11s on the main server and only ~3s on the test server. However, these numbers might get much closer over time. In addition to performance improvements, this commit removes about 150 LOC.	9 years ago
Julius Volz	9b6d69610a	Fix various typos in comments. Helpfully reported by https://goreportcard.com/report/github.com/prometheus/prometheus :)	9 years ago
Brian Brazil	9d0112d7cf	Add without aggregator modifier. This has the advantage that the user doesn't need to list all labels they want to keep (as with "by") but without having to worry about inconsistent labels as when there's only one time series (as with "keeping_common"). Almost all aggregation should use this rather than the existing two options as it's much less error prone and easier to maintain due to not having to always add in "job" plus whatever other common job-level labels you have like "region".	9 years ago
Brian Brazil	b7ef0b45e8	Break aggregation tests out. Add missing tests.	9 years ago
beorn7	a7408bfb47	Unify duration parsing It's actually happening in several places (and for flags, we use the standard Go time.Duration...). This at least reduces all our home-grown parsing to one place (in model).	9 years ago
Fabian Reinartz	a6935024e1	Remove old WITH clause in alert printing	9 years ago
Tobias Schmidt	1a91cd6e09	Rename matrix to range selector in external error messages The documentation speaks about range vectors and range vector selectors. This change does not fix all issues, we might still expose the term "Matrix" in error messages using %T.	9 years ago
Tobias Schmidt	411ca4dba1	Consolidate offset modifier parsing Remove duplicated offset modifier parsing and ensure offset can only appear at the end of a selector statement.	9 years ago
Fabian Reinartz	6b4a6962d2	Support old alerting rule syntax	9 years ago
Brian Brazil	c77c3a8c56	promql: Limit extrapolation of delta/rate/increase The new implementation detects the start and end of a series by looking at the average sample interval within the range. If the first (last) sample in the range is more than 1.1*interval distant from the beginning (end) of the range, it is considered the first (last) sample of the series as a whole, and extrapolation is limited to half the interval (rather than all the way to the beginning (end) of the range). In addition, if the extrapolated starting point of a counter (where it is zero) is within the range, it is used as the starting point of the series. Fixes #581	9 years ago
Brian Brazil	89760dd77d	Handle NaN for min/max. Similar to topk and sort, prefer not returning NaN where possible.	9 years ago
Brian Brazil	bac1f28cad	Similar to topk/bottomk, have sort/sort_desc put NaN at end. This makes topk and bottomk consistent with the sorting functions, as per #1271.	9 years ago
Fabian Reinartz	4209ec6864	Change WITH keyword to LABELS	9 years ago
Brian Brazil	88ca82304c	Make topk/bottomk prefer returning real numbers over NaN.	9 years ago
Brian Brazil	edf3e123f5	Move topk/bottomk tests from legacy.	9 years ago
Fabian Reinartz	af3a6661ed	Implement new alerting rule syntax	9 years ago
James Sanford	5b53262b7a	promql: Add clamp_max/clamp_min functions.	9 years ago
Brian Brazil	a287264989	Print offsets in promql.	9 years ago
Fabian Reinartz	33aab4169c	Anchor regexes in vector matching This commit makes the regex behavior of vector matching consistent with configuration and label_replace() by anchoring it. Fixes #1200	9 years ago
Fabian Reinartz	51e8badc7f	Merge pull request #1159 from prometheus/scalar-bool promql: Remove scalar/scalar comparisons.	9 years ago
Brian Brazil	c36961130b	promql: Remove scalar/scalar comparisons. This change is breaking, use the 'bool' modifier for such comprisons. After this change all comparisons without 'bool' will filter, and all comparisons with 'bool' will return 0/1. This makes the language more consistent and orthogonal, and ultimately easier to learn and use. If we ever figure out sane semantics for filtering scalar/scalar comparisons we can add them in, which will most likely come out of how the new vector() function is used.	9 years ago
Brian Brazil	5740a8fade	promql: Remove deprecated 2nd argument to delta() This change is breaking, use increase() instead. I'm not cleaning up the function in this PR, as my solution to #581 will rewrite and simplify increase/rate/delta.	9 years ago
Brian Brazil	965a71dc4d	Merge pull request #1155 from prometheus/irate promql: Add irate() function	9 years ago
Brian Brazil	f08abdb48b	promql: Add irate() function irate is a rate function that only looks at the most recent two data points, and calucaltes a per-second value from that. This produces much more granular graphs for fast moving data, and works sanely across many scrape intervals. It doesn't do so well for slowly moving data.	9 years ago
Julius Volz	0088aa4d45	Merge pull request #1132 from prometheus/fix-quoting-and-escaping Support escape sequences in strings and add raw strings	9 years ago
Julius Volz	46c5260761	Support escape sequences in strings and add raw strings. This adapts some functionality from the Go standard library for string literal lexing and unquoting/unescaping. The following string types are now supported: Double- or single-quoted strings: These support all escape sequences that Go supports in double-quoted string literals. The difference is that Prometheus also has single-quoted strings (instead of single-quoted runes in Go). Raw newlines are not allowed. Backtick-quoted raw strings: Strings quoted in backticks are treated as raw strings just like in Go and may contain raw newlines and other special characters directly. Fixes https://github.com/prometheus/prometheus/issues/1122 Fixes https://github.com/prometheus/prometheus/issues/1121	9 years ago
Fabian Reinartz	e3b6ec9784	Switch to common/log	9 years ago
Brian Brazil	653ff71f1f	promql: Reduce flakiness of concurrency test	9 years ago
Fabian Reinartz	171f50706a	Fix unkeyed field errors.	9 years ago
Fabian Reinartz	36ec8ba460	Fix missing return on error	9 years ago
Fabian Reinartz	e005f939fd	Fix scalar construction in function	9 years ago
Fabian Reinartz	eca41f5319	Run gofmt	9 years ago
Brian Brazil	fa793d917e	Merge pull request #1080 from prometheus/query-timeout-test promql: Bump sleep in query timeout test	9 years ago
Brian Brazil	ce7f31e03c	promql: Bump sleep in query timeout test This test is flaky, I'm presuming the time.AfterFunc call is being delayed so the evaluation isn't getting cancelled.	9 years ago
Julius Volz	347630431c	Merge pull request #1077 from prometheus/cleanups Fix some dead code, missing error checks, shadowings.	9 years ago
Julius Volz	af513468eb	Fix some dead code, missing error checks, shadowings. I applied https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567 and was greeted with a deluge of warnings, most of which were not applicable or really fixable realistically. These are some of the first ones I decided to fix.	9 years ago
Brian Brazil	29de4ee2b0	Merge pull request #1078 from prometheus/whats-our-vector-victor Remove optional vector() 2nd argument	9 years ago
Brian Brazil	9b382647b5	Remove optional vector() 2nd argument	9 years ago
Fabian Reinartz	a1617d90f4	Merge pull request #1073 from prometheus/whats-our-vector-victor promql: Add vector function.	9 years ago
Brian Brazil	69f5fa0c1e	promql: Add vector function. Currently the only way to convert a scalar to a vector is to use absent(), which isn't very clean. This adds a vector() function that's the inverse of scalar() and lets your optionally set labels. Example usage would be vector(time() % 86400) < 3600 to filter to only the first hour of the day.	9 years ago
Julius Volz	6d3e054692	Fix bool modifier in recording rules and printing. Fixes https://github.com/prometheus/prometheus/issues/1065	9 years ago
Brian Brazil	9ec11b1847	Merge pull request #1049 from prometheus/bool-nofilter promql: Add 'bool' modifier to comparison functions	9 years ago
Brian Brazil	29e8dc2c49	promql: Add 'bool' modifier to comparison functions When doing comparison operations on vectors, filtering sometimes gets in the way and you have to go to a fair bit of effort to workaround it in order to always return a result. The 'bool' modifier instead of filtering returns 0/1 depending on the result of the compairson. This is also a prerequisite to removing plain scalar/scalar comparisons, as it maintains the current behaviour under a new syntax.	9 years ago
Julius Volz	61c42c8da0	Change relabel_replace() to do full-string matches. THIS IS A BREAKING CHANGE. Fixes part of https://github.com/prometheus/prometheus/issues/996	9 years ago
Julius Volz	744d5d5a7a	Merge pull request #1029 from prometheus/vet-fixes Fix "go vet" errors.	9 years ago
Julius Volz	995d3b831d	Fix most golint warnings. This is with `golint -min_confidence=0.5`. I left several lint warnings untouched because they were either incorrect or I felt it was better not to change them at the moment.	9 years ago
Julius Volz	963ad82dcb	Fix "go vet" errors. I ignored all errors of the type "composite literal uses unkeyed fields". Most of them are wrong because of https://github.com/golang/go/issues/9171.	9 years ago
Julius Volz	077a753e6b	Merge pull request #1006 from prometheus/true-values promql: Remove interpolation of vector values.	9 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	9 years ago
Brian Brazil	fb585e4591	promql: Remove interpolation of vector values. The current behaviour produces values that are not from rules or scrapes. So if for example I have a boolean 0/1 it can be returned as 0.2344589. This prevents a number of advanced use cases, introduces race conditions and can produce misleading graphs.	9 years ago
Fabian Reinartz	1535ef1457	Replace metric.SamplePair with model.SamplePair	9 years ago
Fabian Reinartz	438e232c9b	Fix grouping of import blocks	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
Brian Brazil	296f551418	Merge pull request #1014 from prometheus/scalar-rules rules: Allow recorded rules expressions to be scalars.	9 years ago
Brian Brazil	e6a67476c2	rules: Allow recorded rules expressions to be scalars. This is useful if you want to build up a constant metric, such as a set of alert thresholds that vary by label value.	9 years ago

1 2 3 4 5

215 Commits (0a4d31477a75ff229ddfd95f610919c493a5a9c7)