prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	2581648f70	Separate iterators by offset Add test that exposes the problem.	9 years ago
Fabian Reinartz	95c9706d2d	Fix missing comment period.	9 years ago
Julius Volz	9ea2465b99	Fix typo in lexer test.	9 years ago
Tobias Schmidt	907b1380a7	Add tests to specify the string escaping behavior	9 years ago
beorn7	c740789ce3	Improve predict_linear Fixes https://github.com/prometheus/prometheus/issues/1401 This remove the last (and in fact bogus) use of BoundaryValues. Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can be removed here, too.	9 years ago
beorn7	454ecf3f52	Rework the way ranges and instants are handled In a way, our instants were also ranges, just with the staleness delta as range length. They are no treated equally, just that in one case, the range length is set as range, in the other the staleness delta. However, there are "real" instants where start and and time of a query is the same. In those cases, we only want to return a single value (the one closest before or at the equal start and end time). If that value is the last sample in the series, odds are we have it already in the series object. In that case, there is no need to pin or load any chunks. A special singleSampleSeriesIterator is created for that. This should greatly speed up instant queries as they happen frequently for rule evaluations.	9 years ago
beorn7	0e202dacb4	Streamline series iterator creation This will fix issue #1035 and will also help to make issue #1264 less bad. The fundamental problem in the current code: In the preload phase, we quite accurately determine which chunks will be used for the query being executed. However, in the subsequent step of creating series iterators, the created iterators are referencing _all_ in-memory chunks in their series, even the un-pinned ones. In iterator creation, we copy a pointer to each in-memory chunk of a series into the iterator. While this creates a certain amount of allocation churn, the worst thing about it is that copying the chunk pointer out of the chunkDesc requires a mutex acquisition. (Remember that the iterator will also reference un-pinned chunks, so we need to acquire the mutex to protect against concurrent eviction.) The worst case happens if a series doesn't even contain any relevant samples for the query time range. We notice that during preloading but then we will still create a series iterator for it. But even for series that do contain relevant samples, the overhead is quite bad for instant queries that retrieve a single sample from each series, but still go through all the effort of series iterator creation. All of that is particularly bad if a series has many in-memory chunks. This commit addresses the problem from two sides: First, it merges preloading and iterator creation into one step, i.e. the preload call returns an iterator for exactly the preloaded chunks. Second, the required mutex acquisition in chunkDesc has been greatly reduced. That was enabled by a side effect of the first step, which is that the iterator is only referencing pinned chunks, so there is no risk of concurrent eviction anymore, and chunks can be accessed without mutex acquisition. To simplify the code changes for the above, the long-planned change of ValueAtTime to ValueAtOrBefore time was performed at the same time. (It should have been done first, but it kind of accidentally happened while I was in the middle of writing the series iterator changes. Sorry for that.) So far, we actively filtered the up to two values that were returned by ValueAtTime, i.e. we invested work to retrieve up to two values, and then we invested more work to throw one of them away. The SeriesIterator.BoundaryValues method can be removed once #1401 is fixed. But I really didn't want to load even more changes into this PR. Benchmarks: The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times faster) and allocate 95% fewer bytes. The reason for that is that the benchmark reads one sample after another from the time series and creates a new series iterator for each sample read. To find out how much these improvements matter in practice, I have mirrored a beefy Prometheus server at SoundCloud that suffers from both issues #1035 and #1264. To reach steady state that would be comparable, the server needs to run for 15d. So far, it has run for 1d. The test server currently has only half as many memory time series and 60% of the memory chunks the main server has. The 90th percentile rule evaluation cycle time is ~11s on the main server and only ~3s on the test server. However, these numbers might get much closer over time. In addition to performance improvements, this commit removes about 150 LOC.	9 years ago
Julius Volz	9b6d69610a	Fix various typos in comments. Helpfully reported by https://goreportcard.com/report/github.com/prometheus/prometheus :)	9 years ago
Brian Brazil	9d0112d7cf	Add without aggregator modifier. This has the advantage that the user doesn't need to list all labels they want to keep (as with "by") but without having to worry about inconsistent labels as when there's only one time series (as with "keeping_common"). Almost all aggregation should use this rather than the existing two options as it's much less error prone and easier to maintain due to not having to always add in "job" plus whatever other common job-level labels you have like "region".	9 years ago
Brian Brazil	b7ef0b45e8	Break aggregation tests out. Add missing tests.	9 years ago
beorn7	a7408bfb47	Unify duration parsing It's actually happening in several places (and for flags, we use the standard Go time.Duration...). This at least reduces all our home-grown parsing to one place (in model).	9 years ago
Fabian Reinartz	a6935024e1	Remove old WITH clause in alert printing	9 years ago
Tobias Schmidt	1a91cd6e09	Rename matrix to range selector in external error messages The documentation speaks about range vectors and range vector selectors. This change does not fix all issues, we might still expose the term "Matrix" in error messages using %T.	9 years ago
Tobias Schmidt	411ca4dba1	Consolidate offset modifier parsing Remove duplicated offset modifier parsing and ensure offset can only appear at the end of a selector statement.	9 years ago
Fabian Reinartz	6b4a6962d2	Support old alerting rule syntax	9 years ago
Brian Brazil	c77c3a8c56	promql: Limit extrapolation of delta/rate/increase The new implementation detects the start and end of a series by looking at the average sample interval within the range. If the first (last) sample in the range is more than 1.1*interval distant from the beginning (end) of the range, it is considered the first (last) sample of the series as a whole, and extrapolation is limited to half the interval (rather than all the way to the beginning (end) of the range). In addition, if the extrapolated starting point of a counter (where it is zero) is within the range, it is used as the starting point of the series. Fixes #581	9 years ago
Brian Brazil	89760dd77d	Handle NaN for min/max. Similar to topk and sort, prefer not returning NaN where possible.	9 years ago
Brian Brazil	bac1f28cad	Similar to topk/bottomk, have sort/sort_desc put NaN at end. This makes topk and bottomk consistent with the sorting functions, as per #1271.	9 years ago
Fabian Reinartz	4209ec6864	Change WITH keyword to LABELS	9 years ago
Brian Brazil	88ca82304c	Make topk/bottomk prefer returning real numbers over NaN.	9 years ago
Brian Brazil	edf3e123f5	Move topk/bottomk tests from legacy.	9 years ago
Fabian Reinartz	af3a6661ed	Implement new alerting rule syntax	9 years ago
James Sanford	5b53262b7a	promql: Add clamp_max/clamp_min functions.	9 years ago
Brian Brazil	a287264989	Print offsets in promql.	9 years ago
Fabian Reinartz	33aab4169c	Anchor regexes in vector matching This commit makes the regex behavior of vector matching consistent with configuration and label_replace() by anchoring it. Fixes #1200	9 years ago
Fabian Reinartz	51e8badc7f	Merge pull request #1159 from prometheus/scalar-bool promql: Remove scalar/scalar comparisons.	9 years ago
Brian Brazil	c36961130b	promql: Remove scalar/scalar comparisons. This change is breaking, use the 'bool' modifier for such comprisons. After this change all comparisons without 'bool' will filter, and all comparisons with 'bool' will return 0/1. This makes the language more consistent and orthogonal, and ultimately easier to learn and use. If we ever figure out sane semantics for filtering scalar/scalar comparisons we can add them in, which will most likely come out of how the new vector() function is used.	9 years ago
Brian Brazil	5740a8fade	promql: Remove deprecated 2nd argument to delta() This change is breaking, use increase() instead. I'm not cleaning up the function in this PR, as my solution to #581 will rewrite and simplify increase/rate/delta.	9 years ago
Brian Brazil	965a71dc4d	Merge pull request #1155 from prometheus/irate promql: Add irate() function	9 years ago
Brian Brazil	f08abdb48b	promql: Add irate() function irate is a rate function that only looks at the most recent two data points, and calucaltes a per-second value from that. This produces much more granular graphs for fast moving data, and works sanely across many scrape intervals. It doesn't do so well for slowly moving data.	9 years ago
Julius Volz	0088aa4d45	Merge pull request #1132 from prometheus/fix-quoting-and-escaping Support escape sequences in strings and add raw strings	9 years ago
Julius Volz	46c5260761	Support escape sequences in strings and add raw strings. This adapts some functionality from the Go standard library for string literal lexing and unquoting/unescaping. The following string types are now supported: Double- or single-quoted strings: These support all escape sequences that Go supports in double-quoted string literals. The difference is that Prometheus also has single-quoted strings (instead of single-quoted runes in Go). Raw newlines are not allowed. Backtick-quoted raw strings: Strings quoted in backticks are treated as raw strings just like in Go and may contain raw newlines and other special characters directly. Fixes https://github.com/prometheus/prometheus/issues/1122 Fixes https://github.com/prometheus/prometheus/issues/1121	9 years ago
Fabian Reinartz	e3b6ec9784	Switch to common/log	9 years ago
Brian Brazil	653ff71f1f	promql: Reduce flakiness of concurrency test	9 years ago
Fabian Reinartz	171f50706a	Fix unkeyed field errors.	9 years ago
Fabian Reinartz	36ec8ba460	Fix missing return on error	9 years ago
Fabian Reinartz	e005f939fd	Fix scalar construction in function	9 years ago
Fabian Reinartz	eca41f5319	Run gofmt	9 years ago
Brian Brazil	fa793d917e	Merge pull request #1080 from prometheus/query-timeout-test promql: Bump sleep in query timeout test	9 years ago
Brian Brazil	ce7f31e03c	promql: Bump sleep in query timeout test This test is flaky, I'm presuming the time.AfterFunc call is being delayed so the evaluation isn't getting cancelled.	9 years ago
Julius Volz	347630431c	Merge pull request #1077 from prometheus/cleanups Fix some dead code, missing error checks, shadowings.	9 years ago
Julius Volz	af513468eb	Fix some dead code, missing error checks, shadowings. I applied https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567 and was greeted with a deluge of warnings, most of which were not applicable or really fixable realistically. These are some of the first ones I decided to fix.	9 years ago
Brian Brazil	29de4ee2b0	Merge pull request #1078 from prometheus/whats-our-vector-victor Remove optional vector() 2nd argument	9 years ago
Brian Brazil	9b382647b5	Remove optional vector() 2nd argument	9 years ago
Fabian Reinartz	a1617d90f4	Merge pull request #1073 from prometheus/whats-our-vector-victor promql: Add vector function.	9 years ago
Brian Brazil	69f5fa0c1e	promql: Add vector function. Currently the only way to convert a scalar to a vector is to use absent(), which isn't very clean. This adds a vector() function that's the inverse of scalar() and lets your optionally set labels. Example usage would be vector(time() % 86400) < 3600 to filter to only the first hour of the day.	9 years ago
Julius Volz	6d3e054692	Fix bool modifier in recording rules and printing. Fixes https://github.com/prometheus/prometheus/issues/1065	9 years ago
Brian Brazil	9ec11b1847	Merge pull request #1049 from prometheus/bool-nofilter promql: Add 'bool' modifier to comparison functions	9 years ago
Brian Brazil	29e8dc2c49	promql: Add 'bool' modifier to comparison functions When doing comparison operations on vectors, filtering sometimes gets in the way and you have to go to a fair bit of effort to workaround it in order to always return a result. The 'bool' modifier instead of filtering returns 0/1 depending on the result of the compairson. This is also a prerequisite to removing plain scalar/scalar comparisons, as it maintains the current behaviour under a new syntax.	9 years ago
Julius Volz	61c42c8da0	Change relabel_replace() to do full-string matches. THIS IS A BREAKING CHANGE. Fixes part of https://github.com/prometheus/prometheus/issues/996	9 years ago
Julius Volz	744d5d5a7a	Merge pull request #1029 from prometheus/vet-fixes Fix "go vet" errors.	9 years ago
Julius Volz	995d3b831d	Fix most golint warnings. This is with `golint -min_confidence=0.5`. I left several lint warnings untouched because they were either incorrect or I felt it was better not to change them at the moment.	9 years ago
Julius Volz	963ad82dcb	Fix "go vet" errors. I ignored all errors of the type "composite literal uses unkeyed fields". Most of them are wrong because of https://github.com/golang/go/issues/9171.	9 years ago
Julius Volz	077a753e6b	Merge pull request #1006 from prometheus/true-values promql: Remove interpolation of vector values.	9 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	9 years ago
Brian Brazil	fb585e4591	promql: Remove interpolation of vector values. The current behaviour produces values that are not from rules or scrapes. So if for example I have a boolean 0/1 it can be returned as 0.2344589. This prevents a number of advanced use cases, introduces race conditions and can produce misleading graphs.	9 years ago
Fabian Reinartz	1535ef1457	Replace metric.SamplePair with model.SamplePair	9 years ago
Fabian Reinartz	438e232c9b	Fix grouping of import blocks	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
Brian Brazil	296f551418	Merge pull request #1014 from prometheus/scalar-rules rules: Allow recorded rules expressions to be scalars.	9 years ago
Brian Brazil	e6a67476c2	rules: Allow recorded rules expressions to be scalars. This is useful if you want to build up a constant metric, such as a set of alert thresholds that vary by label value.	9 years ago
Laurie Malau	cdf38ab93a	Log runtime errors during query evaluation instead of panicking.	9 years ago
Julius Volz	27ed874358	Implement label_replace() Implements part of https://github.com/prometheus/prometheus/issues/959.	9 years ago
Fabian Reinartz	690b5f1575	Remove multi-statement queries This commit removes the possibility to have multi-statement queries which had no full support anyway. This makes the caller responsible for multi-statement semantics. Multiple tests are no longer timing-dependent.	9 years ago
Julius Volz	e324910ff2	Merge pull request #936 from prometheus/predict promql: Add support for predict(my_timeseries[1h], 2h)	9 years ago
Brian Brazil	d6a80c2b76	promql: Add support for predict_linear(my_timeseries[1h], 7200) This will give a prediction for the value of my_timeseries in 2 hours, based on the last hour of data.	9 years ago
Fabian Reinartz	579fdf65e2	Implement unary expression for vector types. Closes #956	9 years ago
Fabian Reinartz	c322422412	Merge pull request #954 from prometheus/fabxc/fuzz-fix Add missing check for nil expression	9 years ago
Fabian Reinartz	adf109795c	forbid unexpected (runtime) errors in parse tests	9 years ago
Fabian Reinartz	c20e25f718	Add missing check for nil expression	9 years ago
Brian Brazil	a0f0b82348	promql: Test errors aren't always ParseErr	9 years ago
Fabian Reinartz	5279d50d92	Handle parser runtime panics gracefully	9 years ago
Julius Volz	4e4b468fba	Fix lexer bug treating non-Latin Unicode digits as digits. Fixes https://github.com/prometheus/prometheus/issues/939	9 years ago
Fabian Reinartz	3d67d75935	promql: implement JSON array format for scalar and string	10 years ago
Fabian Reinartz	77e8983221	promql: add MarshalJSON method for SamplePair	10 years ago
Fabian Reinartz	c1d37bc55b	Merge pull request #843 from prometheus/fabxc/runbook promql: add runbook to alert statement.	10 years ago
Fabian Reinartz	70d7a987a7	promql: add json tags, fix query constructor.	10 years ago
Fabian Reinartz	749ae450c5	promql: add runbook to alert statement. This commit adds the RUNBOOK keyword to alert statements. The field is optional and expected to be a link.	10 years ago
Fabian Reinartz	7f85b9b215	promql: add MarshalJSON method for ExprType.	10 years ago
Fabian Reinartz	1eff186555	Merge pull request #810 from prometheus/fabxc/lmatch Match empty labels.	10 years ago
Fabian Reinartz	5b91ea9b36	storage: improve label matching and allow unset matching. Matching of empty labels now also matches metrics where the label was not explicitly set to the empty string.	10 years ago
Fabian Reinartz	94cd321be1	promql: error if all label matchers are empty.	10 years ago
Fabian Reinartz	fe301d7946	promql: remove global flags	10 years ago
Julius Volz	5e2d1c1464	Deprecate `keeping_extra`, rename it to `keep_common`. `keep_common` is more in line with the function name `drop_common_labels()` terminology-wise, and also more in line with `group_left`/`group_right` (no `...ing` verb suffix). We could also go the full way and call it `keep_common_labels`. That would have the benefit of being even more consistent with the function `drop_common_labels()` and would be more explanatory, but it also seems quite long.	10 years ago
Fabian Reinartz	e7659f908c	promql: remove DotGraph methods from nodes.	10 years ago
Fabian Reinartz	c716d8a47b	promql: fix aggregation expression String() method. Fixes #794.	10 years ago
Fabian Reinartz	c32ae22119	promql: fix missing metric in range results.	10 years ago
Fabian Reinartz	0acd44b0e3	promql: expose ParseMetric and ParseMetricSelector	10 years ago
Fabian Reinartz	cb10ceac18	promql: allow scalar expressions in range queries, improve errors. These changes allow to do range queries over scalar expressions. Errors on bad types for range queries are now raised on query creation rather than evaluation.	10 years ago
Fabian Reinartz	ab9c98acac	web/api: add initial API v1 implementation.	10 years ago
Fabian Reinartz	03b737322e	promql: expose storage and query engine for tests	10 years ago
Fabian Reinartz	319068a7a6	promql: fix parsing ambiguity for serial repetitions	10 years ago
Fabian Reinartz	c60e3387ef	promql: fix lexing of \r as whitespaces	10 years ago
Fabian Reinartz	0de6edbdfc	Move pkg/ to util/	10 years ago
Fabian Reinartz	dbc0d30e3e	Move string functionality to pkg/strutil	10 years ago
Fabian Reinartz	ccf51b132e	Move stats package to pkg/stats	10 years ago
Fabian Reinartz	3c8fbf1e15	Move test package to pkg/testutil	10 years ago
Brian Brazil	f34de493d5	Add increase() function, to replace delta(..., 1). This calculates how much a counter increases over a given period of time, which is the area under the curve of it's rate. increase(x[5m]) is equivilent to rate(x[5m]) * 300.	10 years ago
Julius Volz	d44a89c6e8	Implement changes() function. changes() takes a range vector and returns the number of times a value has changed in the given time window for each time series as an instant vector.	10 years ago
Julius Volz	6f33ed9e59	Add resets() function to count counter resets. resets() returns for every range vector element how many counter resets there have been in the specified range.	10 years ago
Fabian Reinartz	b7eb105ffa	Remove test flags in help output	10 years ago
beorn7	3b9c421a69	Weed out all the [Gg]et* method names. The only exception is getNumChunksToPersist to avoid naming the struct member numChunksToPersist in a weird way.	10 years ago
Fabian Reinartz	ac4d63b833	Merge pull request #689 from prometheus/fabxc/qltest Add basic testing language, migrate tests	10 years ago
Fabian Reinartz	0d3012a605	Migrate matrix tests, remove old test files.	10 years ago
Fabian Reinartz	71ef7ab405	Migrate remaining vector evaluation tests to new testing language.	10 years ago
Fabian Reinartz	3c22eded97	Migrate literal tests to testing language.	10 years ago
Fabian Reinartz	eba07a7d3d	Migrate histogram tests to test language.	10 years ago
Fabian Reinartz	03094eff04	Migrate parsing error tests. The promql_test checks failure of various bad syntaxed queries. Those are moved into the parser tests as the new testing language only deals with valid queries.	10 years ago
Fabian Reinartz	6321964738	Add parsing and execution of new test format. This commit adds a new test structure that parses and executes the new testing language.	10 years ago
Fabian Reinartz	a236c01457	Add time series description parsing. This commit adds parsing of time series description to the exisiting query language parser. Time series descriptions are defined by a metric followed by a sequence of values.	10 years ago
Fabian Reinartz	d122749b39	Enhance lexer testing output	10 years ago
Fabian Reinartz	ce487f763e	Simplify vector binary evaluation logic	10 years ago
Fabian Reinartz	8a109e061b	Extract OR operation into own eval method.	10 years ago
Fabian Reinartz	2c3e9e2e87	Extract AND operation into own eval method.	10 years ago
Fabian Reinartz	4234a45d2e	Merge pull request #693 from prometheus/fabxc/preload_timeout Check context before each preloading.	10 years ago
Fabian Reinartz	8ea94ebb58	Check context before each preloading.	10 years ago
Fabian Reinartz	969c231191	Make parser more strict about identifiers, extract number parsing	10 years ago
Fabian Reinartz	8707c54508	Fix single quote parsing, add tests	10 years ago
Fabian Reinartz	9ab1f6c690	Limit maximum number of concurrent queries. A high number of concurrent queries can slow each other down so that none of them is reasonbly responsive. This commit limits the number of queries being concurrently executed.	10 years ago
Fabian Reinartz	d59d1cb2c1	Fix Error() methods.	10 years ago
Fabian Reinartz	6649306e63	Merge pull request #660 from prometheus/fabxc/pql/parse-errs Fix and improve parsing error output.	10 years ago
Fabian Reinartz	279831cdf1	Fix and improve parsing error output.	10 years ago
Fabian Reinartz	fe935179cd	Stop routing rule statements through the engine.	10 years ago
Fabian Reinartz	25cdff3527	Remove `name` arg from `Parse*` functions, enhance parsing errors.	10 years ago
Fabian Reinartz	5602328c7c	Refactor query evaluation. This copies the evaluation logic from the current rules/ package. The new engine handles the execution process from query string to final result. It provides query timeout and cancellation and general flexibility for future changes. functions.go: Add evaluation implementation. Slight changes to in/out data but not to the processing logic. quantile.go: No changes. analyzer.go: No changes. engine.go: Actually new part. Mainly consists of evaluation methods which were not changed. setup_test.go: Copy of rules/helpers_test.go to setup test storage. promql_test.go: Copy of rules/rules_test.go.	10 years ago
Fabian Reinartz	32b7595c47	Create promql package with lexer/parser. This commit creates a (so far unused) package. It contains the a custom lexer/parser for the query language. ast.go: New AST that interacts well with the parser. lex.go: Custom lexer (new). lex_test.go: Lexer tests (new). parse.go: Custom parser (new). parse_test.go: Parser tests (new). functions.go: Changed function type, dummies for parser testing (barely changed/dummies). printer.go: Adapted from rules/ and adjusted to new AST (mostly unchanged, few additions).	10 years ago

... 3 4 5 6 7

326 Commits (9fe8bcf4be00a7ebd73932df0cf39cc837cbb993)