prometheus

Commit Graph

Author	SHA1	Message	Date
Tobias Guggenmos	8cb4a48e2e	PromQL: Parse label sets using the generated parser (#6432 ) * Add grammar for label_sets * Parse label Sets using the generated parser * Allow trailing commas for label sets and selectors * Add test to trigger all possible error messages for label matchers Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	35c1f31721	PromQL: Use more standart format for error positions (#6433 ) The most common format (used by go, gcc and clang) for compiler error positions seems to be `filename:line:char:` or `line:char:` if the filename is unknown. This PR adapts the PromQL parser to use this convention. Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	3bb715031f	PromQL: Use generated parser to parse label matchers (#6410 ) Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	bbd92b85da	promql: Use capitalized names for item types (#6371 ) For yacc generated parsers there is the convention to capitalize the names of item types provided by the lexer, which makes it easy to distinct lexer tokens (capitalized) from nonterminal symbols (not capitalized) in language grammars. This convention is also followed by the (non generated) go compiler (see https://golang.org/pkg/go/token/#Token). Part of the parser rewrite described in #6256. Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Tobias Guggenmos	ac3932ea35	Improve PromQL parser performance by making it non-concurrent (#6356 ) Before this commit, the PromQL parser ran in two goroutines: * The lexer goroutine that splits the input into tokens and sent them over a channel to * the parser goroutine which produces the abstract syntax tree The Problem with this approach is that the parser spends more time on goroutine creation and syncronisation than on actual parsing. This commit removes that concurrency and replaces the channel by a slice based buffer. Benchmarks show that this makes the up to 7 times faster than before. Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Alex Dzyoba	e3882629ba	promql: Move tests to testutil (#6103 ) * promql: Move tests to testutil Signed-off-by: Alex Dzyoba <alex@dzyoba.com> * promql: Match error type via errors.As in tests Signed-off-by: Alex Dzyoba <alex@dzyoba.com> * promql: Remove unused `expectedList` func from lex_test.go Signed-off-by: Alex Dzyoba <alex@dzyoba.com>	5 years ago
Chris Marchbanks	529ccff07b	Remove all usages of stretchr/testify Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	5 years ago
Simon Pasquier	45506841e6	*: enable all default linters (#5504 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Julius Volz	8155cc4992	Expose lexer item types (#5358 ) * Expose lexer item types We have generally agreed to expose AST types / values that are necessary to make sense of the AST outside of the promql package. Currently the `UnaryExpr`, `BinaryExpr`, and `AggregateExpr` AST nodes store the lexer item type to indicate the operator type, but since the individual item types aren't exposed, an external user of the package cannot determine the operator type. So this PR exposes them. Although not all item types are required to make sense of the AST (some are really only used in the lexer), I decided to expose them all here to be somewhat more consistent. Another option would be to not use lexer item types at all in AST nodes. The concrete motivation is my work on the PromQL->Flux transpiler, but this ought to be useful for other cases as well. Signed-off-by: Julius Volz <julius.volz@gmail.com> * Fix item type names in tests Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
JoeWrightss	e158c53fa9	Fix some typos in comment (#5175 ) Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>	6 years ago
Ganesh Vernekar	dbe55c1352	Subquery (#4831 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Brian Brazil	8edaa8ad4d	Fix goroutine leak in lexer/parser. (#4858 ) When there was an error in the parser, the lexer goroutine was left running. Also make runtime panic test actually test things. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Frederic Branczyk	bda9781ccd	Merge pull request #3839 from brancz/remove-old-alert-record promql: Remove old and unused alerting/reconding syntax	6 years ago
Ganesh Vernekar	73db8b8cea	[bugfix] Parse negative value in PromQL (#4564 ) * Parse negative value in PromQL * Enforce space between values Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Frederic Branczyk	b0b3e3dd74	promql: Remove old and unused alerting/reconding syntax Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	6 years ago
Karsten Weiss	d79d573f71	Fix spelling mistakes found by codespell (#4065 ) Signed-off-by: Karsten Weiss <knweiss@gmail.com>	7 years ago
Warren Fernandes	d49a3df55b	Parser test cleanup (#3977 ) * parser test cleanup - Test against the exported package functions instead of the private functions. * Improves readability of TestParseSeries - Moves package function closer to parser function	7 years ago
Brian Brazil	99905f82a6	Remove keep_common modifier. See #3060	7 years ago
Fabian Reinartz	ca2b68889b	Merge branch 'master' into dev-2.0	8 years ago
Fabian Reinartz	f46a8e9ea4	Merge pull request #2854 from prometheus/promql-rune Check for invalid utf-8 in lexer strings.	8 years ago
Brian Brazil	6f5d952132	Check for invalid utf-8 in lexer strings. This protects against invalid utf-8 sneaking in via label_replace.	8 years ago
Harsh Agarwal	16867c89a7	implement label_join issue 1147 (#2806 ) Replace OptionalArgs int with Variadic int.	8 years ago
Fabian Reinartz	71fe0c58a8	promql: misc fixes	8 years ago
Fabian Reinartz	fecf9532b9	*: fix misc compile errors	8 years ago
Fabian Reinartz	ac5d3bc05e	promql: scalar T/V and Point	8 years ago
Fabian Reinartz	a62df87022	promql: rename vector	8 years ago
Fabian Reinartz	15a931dbdb	promql: migrate model types, use tsdb interfaces	8 years ago
Tristan Colgate	68fc15fe4e	Report type names in the form used in documentation	8 years ago
Tobias Schmidt	29ced0090f	Fix common english misspellings	8 years ago
Tobias Schmidt	04ae6196f2	Fix parsing of label names which are also keywords The current separation between lexer and parser is a bit fuzzy when it comes to operators, aggregators and other keywords. The lexer already tries to determine the type of a token, even though that type might change depending on the context. This led to the problematic behavior that no tokens known to the lexer could be used as label names, including operators (and, by, ...), aggregators (count, quantile, ...) or other keywords (for, offset, ...). This change additionally checks whether an identifier is one of these types. We might want to check whether the specific item identification should be moved from the lexer to the parser.	8 years ago
beorn7	fc6737b7fb	storage: improve index lookups tl;dr: This is not a fundamental solution to the indexing problem (like tindex is) but it at least avoids utilizing the intersection problem to the greatest possible amount. In more detail: Imagine the following query: nicely:aggregating:rule{job="foo",env="prod"} While it uses a nicely aggregating recording rule (which might have a very low cardinality), Prometheus still intersects the low number of fingerprints for `{__name__="nicely:aggregating:rule"}` with the many thousands of fingerprints matching `{job="foo"}` and with the millions of fingerprints matching `{env="prod"}`. This totally innocuous query is dead slow if the Prometheus server has a lot of time series with the `{env="prod"}` label. Ironically, if you make the query more complicated, it becomes blazingly fast: nicely:aggregating:rule{job=~"foo",env=~"prod"} Why so? Because Prometheus only intersects with non-Equal matchers if there are no Equal matchers. That's good in this case because it retrieves the few fingerprints for `{__name__="nicely:aggregating:rule"}` and then starts right ahead to retrieve the metric for those FPs and checking individually if they match the other matchers. This change is generalizing the idea of when to stop intersecting FPs and go into "retrieve metrics and check them individually against remaining matchers" mode: - First, sort all matchers by "expected cardinality". Matchers matching the empty string are always worst (and never used for intersections). Equal matchers are in general consider best, but by using some crude heuristics, we declare some better than others (instance labels or anything that looks like a recording rule). - Then go through the matchers until we hit a threshold of remaining FPs in the intersection. This threshold is higher if we are already in the non-Equal matcher area as intersection is even more expensive here. - Once the threshold has been reached (or we have run out of matchers that do not match the empty string), start with "retrieve metrics and check them individually against remaining matchers". A beefy server at SoundCloud was spending 67% of its CPU time in index lookups (fingerprintsForLabelPairs), serving mostly a dashboard that is exclusively built with recording rules. With this change, it spends only 35% in fingerprintsForLabelPairs. The CPU usage dropped from 26 cores to 18 cores. The median latency for query_range dropped from 14s to 50ms(!). As expected, higher percentile latency didn't improve that much because the new approach is _occasionally_ running into the worst case while the old one was _systematically_ doing so. The 99th percentile latency is now about as high as the median before (14s) while it was almost twice as high before (26s).	8 years ago
Brian Brazil	16690736ab	Add count_values() aggregator. This is useful for counting how many instances of a job are running a particular version/build. Fixes #622	9 years ago
Brian Brazil	7f23a4a099	Add type check on topk/bottomk parameter.	9 years ago
Brian Brazil	3e5136e36d	Make topk/bottomk aggregators.	9 years ago
Brian Brazil	3b89616d82	Allow on, ignoring, by and without wit empty laberls. This offers new semantics in allowing on() for matching two single-element vectors with no known common labels. Previosuly this was often done using on(dummy). This also allows making it explict that you meant to do an aggregation without labels via by(). Fixes #1597.	9 years ago
Brian Brazil	246a817300	Flip vector matching to be ignoring by default. This is a noop semantically.	9 years ago
Ali Reza	e7eba75690	remove keeping_extra because it's replaced with keep_common change all keepExtra label into keepCommon, and move action into removed list change incorrect token list	9 years ago
Brian Brazil	68e70d992a	Clarify error message around on(x) group_left(x)	9 years ago
Brian Brazil	7201c010c4	Rename On to MatchingLabels	9 years ago
Brian Brazil	d991f0cf47	For many-to-one matches, always copy label from one side. This is a breaking change for everyone using the machine roles labeling approach.	9 years ago
Brian Brazil	d1edfb25b3	Add support for OneToMany with IGNORING. The labels listed in the group_ modifier will be copied from the one side to the many side. It will be valid to specify no labels. This is intended to replace the existing ON/GROUP_* support.,	9 years ago
Brian Brazil	1d08c4fef0	Add 'ignoring' as modifier for binops. Where 'on' uses the given labels to match, 'ignoring' uses all other labels to match. group_left/right is not supported yet.	9 years ago
Tobias Schmidt	8cc86f25c0	Implement relative complement set operator "unless" The `unless` set operator can be used to return all vector elements from the LHS which do not match the elements on the RHS. A use case is to return all metrics for nodes which do not have a specific role: node_load1 unless on(instance) chef_role{role="app"}	9 years ago
Fabian Reinartz	ab3d7a0ec0	Remove old alerting syntax	9 years ago
Patrick Bogen	2062fbae0f	rewrite operator balancing to be recursive	9 years ago
Brian Brazil	9d0112d7cf	Add without aggregator modifier. This has the advantage that the user doesn't need to list all labels they want to keep (as with "by") but without having to worry about inconsistent labels as when there's only one time series (as with "keeping_common"). Almost all aggregation should use this rather than the existing two options as it's much less error prone and easier to maintain due to not having to always add in "job" plus whatever other common job-level labels you have like "region".	9 years ago
Tobias Schmidt	1a91cd6e09	Rename matrix to range selector in external error messages The documentation speaks about range vectors and range vector selectors. This change does not fix all issues, we might still expose the term "Matrix" in error messages using %T.	9 years ago
Tobias Schmidt	411ca4dba1	Consolidate offset modifier parsing Remove duplicated offset modifier parsing and ensure offset can only appear at the end of a selector statement.	9 years ago
Fabian Reinartz	6b4a6962d2	Support old alerting rule syntax	9 years ago

1 2

79 Commits (c2e083c30f2cf94bfb5bc8bdfcb27dafb37e2349)