Commit Graph

1175 Commits (00146b69fc9486413332a230dbed407127952807)

Author SHA1 Message Date
Brian Brazil 74094947ea effect -> affect 2016-05-12 15:14:48 +01:00
Brian Brazil 68aaea618a Merge pull request #1624 from dmitris/golint
(trivial) fix several minor golint style issues
2016-05-11 14:20:19 +01:00
Fabian Reinartz bbc4f11bcc Merge pull request #945 from msiebuhr/fuzz
Fuzz parsers
2016-05-11 14:31:31 +02:00
Dmitry Savintsev 7fdb62c253 fix several minor golint style issues 2016-05-11 14:26:18 +02:00
Morten Siebuhr ffc8cab39a Updates fuzzers to discard less interesting data 2016-05-10 11:46:03 +02:00
Brian Brazil ef55fd6176 Add unittest for using a metric for thresholds with group_left. 2016-05-08 16:58:23 +01:00
Morten Siebuhr 981b636004 Bring fuzzer error handling in line. 2016-04-29 22:50:24 +02:00
Morten Siebuhr 9eb2e98509 Fix up documentation + go fmt. 2016-04-29 22:50:24 +02:00
Morten Siebuhr 7371dcc787 Fuzzing corpus for ParseMetric. 2016-04-29 22:50:24 +02:00
Morten Siebuhr 5fec020b27 Initial fuzzing corpus for ParseExpr. 2016-04-29 22:50:24 +02:00
Morten Siebuhr 0ebcca5eb7 Add basic fuzzer of the parser. 2016-04-29 22:50:24 +02:00
Brian Brazil 68e70d992a Clarify error message around on(x) group_left(x) 2016-04-26 14:31:00 +01:00
Brian Brazil 7201c010c4 Rename On to MatchingLabels 2016-04-26 14:28:36 +01:00
Brian Brazil d991f0cf47 For many-to-one matches, always copy label from one side.
This is a breaking change for everyone using the machine roles
labeling approach.
2016-04-21 19:35:41 +01:00
Brian Brazil 768d09fd2a Change on+group_* to take copy from the one side.
If the label doesn't exist on the one side, it's not copied.

All labels on the many inside are included, this is a breaking change
but likely low impact.
2016-04-21 19:35:40 +01:00
Brian Brazil d1edfb25b3 Add support for OneToMany with IGNORING.
The labels listed in the group_ modifier will be copied from the one
side to the many side. It will be valid to specify no labels.

This is intended to replace the existing ON/GROUP_* support.,
2016-04-21 19:35:35 +01:00
Brian Brazil 1d08c4fef0 Add 'ignoring' as modifier for binops.
Where 'on' uses the given labels to match,
'ignoring' uses all other labels to match.

group_left/right is not supported yet.
2016-04-21 19:34:29 +01:00
Brian Brazil f5084ab1c5 Add tests for group_left/group_right 2016-04-21 16:52:53 +01:00
Fabian Reinartz fceedfa807 Add error message if old alert rule tokens are read 2016-04-16 22:25:51 +02:00
Julius Volz 6ac39700ea Fix missing printed keep_common without grouping. 2016-04-15 19:48:17 +02:00
Jonathan Boulle 38098f8c95 Add missing license headers
Prometheus is Apache 2 licensed, and most source files have the
appropriate copyright license header, but some were missing it without
apparent reason. Correct that by adding it.
2016-04-13 16:08:22 +02:00
Fabian Reinartz 9ee91062c4 Merge pull request #1522 from prometheus/unless-operator
Implement relative complement set operator "unless"
2016-04-04 21:36:17 +02:00
Tobias Schmidt 8cc86f25c0 Implement relative complement set operator "unless"
The `unless` set operator can be used to return all vector elements from
the LHS which do not match the elements on the RHS. A use case is to
return all metrics for nodes which do not have a specific role:

    node_load1 unless on(instance) chef_role{role="app"}
2016-04-04 01:29:44 -04:00
Tobias Schmidt e82ef154ee Remove unused code leftovers 2016-04-02 20:20:55 -04:00
Tobias Schmidt 4c3dc25e35 Fix whitespace in promql test data 2016-04-02 18:25:26 -04:00
Fabian Reinartz 235e6c554b Use ContainsRune 2016-04-01 10:36:17 +02:00
Brian Brazil 24a3ad3d16 Merge pull request #1485 from eliothedeman/master
Adds holt-winters query function
2016-03-28 20:53:01 +01:00
eliothedeman 1543ef92b2 Adds holt-winters query function 2016-03-28 15:42:27 -04:00
beorn7 507f550cd4 Merge branch 'master' into beorn7/storage7 2016-03-24 14:21:28 +01:00
Brian Brazil 070d663948 Merge pull request #1501 from prometheus/and-dummy
Pull in fix for and with empty labelsets
2016-03-24 12:52:28 +00:00
Fabian Reinartz ab3d7a0ec0 Remove old alerting syntax 2016-03-23 10:19:00 +01:00
beorn7 4b574e8a61 Switch chunk encoding to type 2 where it was hardcoded type 1 before
The chunk encoding was hardcoded there because it mostly doesn't
matter what encoding is chosen in that test. Since type 1 is
battle-hardened enough, I'm switching to type 2 here so that we can
catch unexpected problems as a byproduct. My expectation is that the
chunk encoding doesn't matter anyway, as said, but then "unexpected
problems" contains the word "unexpected".
2016-03-20 23:32:20 +01:00
Brian Brazil 8788701ce7 Add test for incorrect behaviour 2016-03-18 12:07:40 +00:00
Brian Brazil 39d556f0d5 Move all the operator tests into one file 2016-03-18 12:02:44 +00:00
beorn7 99854a84d7 Merge branch 'beorn7/storage6' into beorn7/storage7 2016-03-09 17:23:25 +01:00
beorn7 d0a4477446 Merge branch 'beorn7/storage3' into beorn7/storage4
Conflicts:
	storage/local/preload.go
	storage/local/storage.go
	storage/local/storage_test.go
2016-03-09 17:13:16 +01:00
beorn7 dad302144d Make a naked return less naked 2016-03-09 15:06:00 +01:00
beorn7 836f1db04c Improve MetricsForLabelMatchers
WIP: This needs more tests.

It now gets a from and through value, which it may opportunistically
use to optimize the retrieval. With possible future range indices,
this could be used in a very efficient way. This change merely applies
some easy checks, which should nevertheless solve the use case of
heavy rule evaluations on servers with a lot of series churn.

Idea is the following:

- Only archive series that are at least as old as the headChunkTimeout
  (which was already extremely unlikely to happen).

- Then maintain a high watermark for the last archival, i.e. no
  archived series has a sample more recent than that watermark.

- Any query that doesn't reach to a time before that watermark doesn't
  have to touch the archive index at all. (A production server at
  Soundcloud with the aforementioned series churn and heavy rule
  evaluations spends 50% of its CPU time in archive index
  lookups. Since rule evaluations usually only touch very recent
  values, most of those lookup should disappear with this change.)

- Federation with a very broad label matcher will profit from this,
  too.

As a byproduct, the un-needed MetricForFingerprint method was removed
from the Storage interface.
2016-03-09 00:25:59 +01:00
beorn7 f7fc542db6 Merge branch 'master' into beorn7/storage4
Conflicts:
	storage/local/persistence.go
2016-03-08 00:14:00 +01:00
beorn7 3d86130d8c Merge branch 'master' into beorn7/storage3 2016-03-07 23:39:12 +01:00
Björn Rabenstein 2a2cc52828 Merge pull request #1405 from prometheus/beorn7/storage
Streamline series iterator creation
2016-03-07 13:30:56 +01:00
Patrick Bogen 250344b344 use short variable assignment 2016-03-03 09:46:50 -08:00
Patrick Bogen 2062fbae0f rewrite operator balancing to be recursive 2016-03-02 15:56:40 -08:00
beorn7 0ea5801e47 Handle errors caused by data corruption more gracefully
This requires all the panic calls upon unexpected data to be converted
into errors returned. This pollute the function signatures quite
lot. Well, this is Go...

The ideas behind this are the following:

- panic only if it's a programming error. Data corruptions happen, and
  they are not programming errors.

- If we detect a data corruption, we "quarantine" the series,
  essentially removing it from the database and putting its data into
  a separate directory for forensics.

- Failure during writing to a series file is not considered corruption
  automatically. It will call setDirty, though, so that a
  crashrecovery upon the next restart will commence and check for
  that.

- Series quarantining and setDirty calls are logged and counted in
  metrics, but are hidden from the user of the interfaces in
  interface.go, whith the notable exception of Append(). The reasoning
  is that we treat corruption by removing the corrupted series, i.e. a
  query for it will return no results on its next call anyway, so
  return no results right now. In the case of Append(), we want to
  tell the user that no data has been appended, though.

Minor side effects:

- Now consistently using filepath.* instead of path.*.

- Introduced structured logging where I touched it. This makes things
  less consistent, but a complete change to structured logging would
  be out of scope for this PR.
2016-03-02 23:02:34 +01:00
beorn7 8766f99085 Merge branch 'beorn7/storage2' into beorn7/storage3 2016-03-02 23:02:06 +01:00
beorn7 162f6fa6f6 Merge branch 'beorn7/storage' into beorn7/storage2 2016-03-02 23:01:26 +01:00
beorn7 79a2ae2d2e Add missing test file 2016-03-02 23:00:23 +01:00
beorn7 b6840997a7 Merge branch 'beorn7/storage2' into beorn7/storage3 2016-03-02 16:11:25 +01:00
beorn7 ce58fd357b Merge branch 'beorn7/storage' into beorn7/storage2
Conflicts:
	storage/local/chunk.go
	storage/local/interface.go
2016-03-02 16:09:32 +01:00
beorn7 2581648f70 Separate iterators by offset
Add test that exposes the problem.
2016-03-02 16:01:03 +01:00
Fabian Reinartz 95c9706d2d Fix missing comment period. 2016-03-02 09:16:56 +01:00
Julius Volz 9ea2465b99 Fix typo in lexer test. 2016-03-02 01:13:27 +01:00
Tobias Schmidt 907b1380a7 Add tests to specify the string escaping behavior 2016-03-01 17:23:18 -05:00
beorn7 c740789ce3 Improve predict_linear
Fixes https://github.com/prometheus/prometheus/issues/1401

This remove the last (and in fact bogus) use of BoundaryValues.

Thus, a whole lot of unused (and arguably sub-optimal / ugly) code can
be removed here, too.
2016-02-25 12:10:55 +01:00
beorn7 454ecf3f52 Rework the way ranges and instants are handled
In a way, our instants were also ranges, just with the staleness delta
as range length. They are no treated equally, just that in one case,
the range length is set as range, in the other the staleness
delta. However, there are "real" instants where start and and time of
a query is the same. In those cases, we only want to return a single
value (the one closest before or at the equal start and end time). If
that value is the last sample in the series, odds are we have it
already in the series object. In that case, there is no need to pin or
load any chunks. A special singleSampleSeriesIterator is created for
that. This should greatly speed up instant queries as they happen
frequently for rule evaluations.
2016-02-22 01:47:18 +01:00
beorn7 0e202dacb4 Streamline series iterator creation
This will fix issue #1035 and will also help to make issue #1264 less
bad.

The fundamental problem in the current code:

In the preload phase, we quite accurately determine which chunks will
be used for the query being executed. However, in the subsequent step
of creating series iterators, the created iterators are referencing
_all_ in-memory chunks in their series, even the un-pinned ones. In
iterator creation, we copy a pointer to each in-memory chunk of a
series into the iterator. While this creates a certain amount of
allocation churn, the worst thing about it is that copying the chunk
pointer out of the chunkDesc requires a mutex acquisition. (Remember
that the iterator will also reference un-pinned chunks, so we need to
acquire the mutex to protect against concurrent eviction.) The worst
case happens if a series doesn't even contain any relevant samples for
the query time range. We notice that during preloading but then we
will still create a series iterator for it. But even for series that
do contain relevant samples, the overhead is quite bad for instant
queries that retrieve a single sample from each series, but still go
through all the effort of series iterator creation. All of that is
particularly bad if a series has many in-memory chunks.

This commit addresses the problem from two sides:

First, it merges preloading and iterator creation into one step,
i.e. the preload call returns an iterator for exactly the preloaded
chunks.

Second, the required mutex acquisition in chunkDesc has been greatly
reduced. That was enabled by a side effect of the first step, which is
that the iterator is only referencing pinned chunks, so there is no
risk of concurrent eviction anymore, and chunks can be accessed
without mutex acquisition.

To simplify the code changes for the above, the long-planned change of
ValueAtTime to ValueAtOrBefore time was performed at the same
time. (It should have been done first, but it kind of accidentally
happened while I was in the middle of writing the series iterator
changes. Sorry for that.) So far, we actively filtered the up to two
values that were returned by ValueAtTime, i.e. we invested work to
retrieve up to two values, and then we invested more work to throw one
of them away.

The SeriesIterator.BoundaryValues method can be removed once #1401 is
fixed. But I really didn't want to load even more changes into this
PR.

Benchmarks:

The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times
faster) and allocate 95% fewer bytes. The reason for that is that the
benchmark reads one sample after another from the time series and
creates a new series iterator for each sample read.

To find out how much these improvements matter in practice, I have
mirrored a beefy Prometheus server at SoundCloud that suffers from
both issues #1035 and #1264. To reach steady state that would be
comparable, the server needs to run for 15d. So far, it has run for
1d. The test server currently has only half as many memory time series
and 60% of the memory chunks the main server has. The 90th percentile
rule evaluation cycle time is ~11s on the main server and only ~3s on
the test server. However, these numbers might get much closer over
time.

In addition to performance improvements, this commit removes about 150
LOC.
2016-02-19 16:24:38 +01:00
Julius Volz 9b6d69610a Fix various typos in comments.
Helpfully reported by
https://goreportcard.com/report/github.com/prometheus/prometheus :)
2016-02-10 03:47:00 +01:00
Brian Brazil 9d0112d7cf Add without aggregator modifier.
This has the advantage that the user doesn't need
to list all labels they want to keep (as with "by")
but without having to worry about inconsistent labels
as when there's only one time series (as with "keeping_common").

Almost all aggregation should use this rather than the existing
two options as it's much less error prone and easier to maintain
due to not having to always add in "job" plus whatever other common
job-level labels you have like "region".
2016-02-08 14:05:33 +00:00
Brian Brazil b7ef0b45e8 Break aggregation tests out. Add missing tests. 2016-02-07 18:02:51 +00:00
beorn7 a7408bfb47 Unify duration parsing
It's actually happening in several places (and for flags, we use the
standard Go time.Duration...). This at least reduces all our
home-grown parsing to one place (in model).
2016-01-29 15:41:50 +01:00
Fabian Reinartz a6935024e1 Remove old WITH clause in alert printing 2016-01-26 15:45:27 +01:00
Tobias Schmidt 1a91cd6e09 Rename matrix to range selector in external error messages
The documentation speaks about range vectors and range vector selectors.
This change does not fix all issues, we might still expose the term
"Matrix" in error messages using %T.
2016-01-25 13:25:56 -05:00
Tobias Schmidt 411ca4dba1 Consolidate offset modifier parsing
Remove duplicated offset modifier parsing and ensure offset can only
appear at the end of a selector statement.
2016-01-24 23:11:44 -05:00
Fabian Reinartz 6b4a6962d2 Support old alerting rule syntax 2016-01-11 12:14:06 +01:00
Brian Brazil c77c3a8c56 promql: Limit extrapolation of delta/rate/increase
The new implementation detects the start and end of a series by
looking at the average sample interval within the range. If the first
(last) sample in the range is more than 1.1*interval distant from the
beginning (end) of the range, it is considered the first (last) sample
of the series as a whole, and extrapolation is limited to half the
interval (rather than all the way to the beginning (end) of the
range). In addition, if the extrapolated starting point of a counter
(where it is zero) is within the range, it is used as the starting
point of the series.

Fixes #581
2016-01-08 15:32:43 +01:00
Brian Brazil 89760dd77d Handle NaN for min/max.
Similar to topk and sort, prefer not returning NaN
where possible.
2016-01-06 12:41:40 +00:00
Brian Brazil bac1f28cad Similar to topk/bottomk, have sort/sort_desc put NaN at end.
This makes topk and bottomk consistent with the sorting functions,
as per #1271.
2015-12-31 14:52:48 +00:00
Fabian Reinartz 4209ec6864 Change WITH keyword to LABELS 2015-12-23 14:54:02 +01:00
Brian Brazil 88ca82304c Make topk/bottomk prefer returning real numbers over NaN. 2015-12-22 13:53:43 +00:00
Brian Brazil edf3e123f5 Move topk/bottomk tests from legacy. 2015-12-22 12:38:32 +00:00
Fabian Reinartz af3a6661ed Implement new alerting rule syntax 2015-12-11 17:02:34 +01:00
James Sanford 5b53262b7a promql: Add clamp_max/clamp_min functions. 2015-11-26 13:38:06 -08:00
Brian Brazil a287264989 Print offsets in promql. 2015-11-15 16:24:29 +00:00
Fabian Reinartz 33aab4169c Anchor regexes in vector matching
This commit makes the regex behavior of vector matching consistent with
configuration and label_replace() by anchoring it.

Fixes #1200
2015-11-05 11:23:43 +01:00
Fabian Reinartz 51e8badc7f Merge pull request #1159 from prometheus/scalar-bool
promql: Remove scalar/scalar comparisons.
2015-10-16 12:28:56 +02:00
Brian Brazil c36961130b promql: Remove scalar/scalar comparisons.
This change is breaking, use the 'bool' modifier for such comprisons.

After this change all comparisons without 'bool' will filter, and all
comparisons with 'bool' will return 0/1. This makes the language more
consistent and orthogonal, and ultimately easier to learn and use.

If we ever figure out sane semantics for filtering scalar/scalar
comparisons we can add them in, which will most likely come out of how
the new vector() function is used.
2015-10-11 08:51:04 +01:00
Brian Brazil 5740a8fade promql: Remove deprecated 2nd argument to delta()
This change is breaking, use increase() instead.

I'm not cleaning up the function in this PR, as my solution to #581 will
rewrite and simplify increase/rate/delta.
2015-10-10 15:41:23 +01:00
Brian Brazil 965a71dc4d Merge pull request #1155 from prometheus/irate
promql: Add irate() function
2015-10-10 08:05:05 +01:00
Brian Brazil f08abdb48b promql: Add irate() function
irate is a rate function that only looks at the most
recent two data points, and calucaltes a per-second value
from that. This produces much more granular graphs for
fast moving data, and works sanely across many scrape intervals.

It doesn't do so well for slowly moving data.
2015-10-09 21:44:35 +01:00
Julius Volz 0088aa4d45 Merge pull request #1132 from prometheus/fix-quoting-and-escaping
Support escape sequences in strings and add raw strings
2015-10-08 20:51:18 +02:00
Julius Volz 46c5260761 Support escape sequences in strings and add raw strings.
This adapts some functionality from the Go standard library for string
literal lexing and unquoting/unescaping.

The following string types are now supported:

Double- or single-quoted strings:

  These support all escape sequences that Go supports in double-quoted
  string literals. The difference is that Prometheus also has
  single-quoted strings (instead of single-quoted runes in Go). Raw
  newlines are not allowed.

Backtick-quoted raw strings:

  Strings quoted in backticks are treated as raw strings just like in Go
  and may contain raw newlines and other special characters directly.

Fixes https://github.com/prometheus/prometheus/issues/1122
Fixes https://github.com/prometheus/prometheus/issues/1121
2015-10-08 19:17:21 +02:00
Fabian Reinartz e3b6ec9784 Switch to common/log 2015-10-03 10:21:43 +02:00
Brian Brazil 653ff71f1f promql: Reduce flakiness of concurrency test 2015-09-23 10:07:30 +01:00
Fabian Reinartz 171f50706a Fix unkeyed field errors. 2015-09-18 17:00:08 +02:00
Fabian Reinartz 36ec8ba460 Fix missing return on error 2015-09-18 16:50:13 +02:00
Fabian Reinartz e005f939fd Fix scalar construction in function 2015-09-18 16:49:32 +02:00
Fabian Reinartz eca41f5319 Run gofmt 2015-09-16 14:33:12 +02:00
Brian Brazil fa793d917e Merge pull request #1080 from prometheus/query-timeout-test
promql: Bump sleep in query timeout test
2015-09-14 13:00:47 +01:00
Brian Brazil ce7f31e03c promql: Bump sleep in query timeout test
This test is flaky, I'm presuming the time.AfterFunc
call is being delayed so the evaluation isn't getting
cancelled.
2015-09-14 11:49:18 +01:00
Julius Volz 347630431c Merge pull request #1077 from prometheus/cleanups
Fix some dead code, missing error checks, shadowings.
2015-09-14 12:37:26 +02:00
Julius Volz af513468eb Fix some dead code, missing error checks, shadowings.
I applied
https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567
and was greeted with a deluge of warnings, most of which were not
applicable or really fixable realistically. These are some of the first
ones I decided to fix.
2015-09-14 12:21:34 +02:00
Brian Brazil 29de4ee2b0 Merge pull request #1078 from prometheus/whats-our-vector-victor
Remove optional vector() 2nd argument
2015-09-13 14:14:20 +01:00
Brian Brazil 9b382647b5 Remove optional vector() 2nd argument 2015-09-13 09:13:22 +01:00
Fabian Reinartz a1617d90f4 Merge pull request #1073 from prometheus/whats-our-vector-victor
promql: Add vector function.
2015-09-12 08:36:13 +02:00
Brian Brazil 69f5fa0c1e promql: Add vector function.
Currently the only way to convert a scalar to a vector is to
use absent(), which isn't very clean. This adds a vector()
function that's the inverse of scalar() and lets your optionally
set labels.

Example usage would be
vector(time() % 86400) < 3600
to filter to only the first hour of the day.
2015-09-11 12:09:34 +01:00
Julius Volz 6d3e054692 Fix bool modifier in recording rules and printing.
Fixes https://github.com/prometheus/prometheus/issues/1065
2015-09-10 01:37:05 +02:00
Brian Brazil 9ec11b1847 Merge pull request #1049 from prometheus/bool-nofilter
promql: Add 'bool' modifier to comparison functions
2015-09-03 15:08:38 +01:00
Brian Brazil 29e8dc2c49 promql: Add 'bool' modifier to comparison functions
When doing comparison operations on vectors, filtering
sometimes gets in the way and you have to go to a fair bit of
effort to workaround it in order to always return a result.
The 'bool' modifier instead of filtering returns 0/1 depending
on the result of the compairson.

This is also a prerequisite to removing plain scalar/scalar comparisons,
as it maintains the current behaviour under a new syntax.
2015-09-02 14:51:44 +01:00
Julius Volz 61c42c8da0 Change relabel_replace() to do full-string matches.
THIS IS A BREAKING CHANGE.

Fixes part of https://github.com/prometheus/prometheus/issues/996
2015-09-01 15:49:28 +02:00
Julius Volz 744d5d5a7a Merge pull request #1029 from prometheus/vet-fixes
Fix "go vet" errors.
2015-08-26 12:50:18 +02:00
Julius Volz 995d3b831d Fix most golint warnings.
This is with `golint -min_confidence=0.5`.

I left several lint warnings untouched because they were either
incorrect or I felt it was better not to change them at the moment.
2015-08-26 12:44:46 +02:00
Julius Volz 963ad82dcb Fix "go vet" errors.
I ignored all errors of the type "composite literal uses unkeyed
fields". Most of them are wrong because of
https://github.com/golang/go/issues/9171.
2015-08-26 02:05:04 +02:00
Julius Volz 077a753e6b Merge pull request #1006 from prometheus/true-values
promql: Remove interpolation of vector values.
2015-08-25 16:11:07 +02:00
Fabian Reinartz d6b8da8d43 Switch promql types to common/model 2015-08-25 13:49:14 +02:00
Brian Brazil fb585e4591 promql: Remove interpolation of vector values.
The current behaviour produces values that are not
from rules or scrapes. So if for example I have
a boolean 0/1 it can be returned as 0.2344589. This
prevents a number of advanced use cases, introduces
race conditions and can produce misleading graphs.
2015-08-24 17:37:31 +01:00
Fabian Reinartz 1535ef1457 Replace metric.SamplePair with model.SamplePair 2015-08-22 14:52:35 +02:00
Fabian Reinartz 438e232c9b Fix grouping of import blocks 2015-08-22 09:42:45 +02:00
Fabian Reinartz 306e8468a0 Switch from client_golang/model to common/model 2015-08-21 13:33:38 +02:00
Brian Brazil 296f551418 Merge pull request #1014 from prometheus/scalar-rules
rules: Allow recorded rules expressions to be scalars.
2015-08-19 22:10:49 +01:00
Brian Brazil e6a67476c2 rules: Allow recorded rules expressions to be scalars.
This is useful if you want to build up a constant metric,
such as a set of alert thresholds that vary by label value.
2015-08-19 21:09:00 +01:00
Laurie Malau cdf38ab93a Log runtime errors during query evaluation instead of panicking. 2015-08-19 16:56:41 +02:00
Julius Volz 27ed874358 Implement label_replace()
Implements part of https://github.com/prometheus/prometheus/issues/959.
2015-08-18 14:20:07 +02:00
Fabian Reinartz 690b5f1575 Remove multi-statement queries
This commit removes the possibility to have multi-statement queries
which had no full support anyway. This makes the caller responsible
for multi-statement semantics.
Multiple tests are no longer timing-dependent.
2015-08-10 14:26:20 +02:00
Julius Volz e324910ff2 Merge pull request #936 from prometheus/predict
promql: Add support for predict(my_timeseries[1h], 2h)
2015-08-05 16:40:51 +02:00
Brian Brazil d6a80c2b76 promql: Add support for predict_linear(my_timeseries[1h], 7200)
This will give a prediction for the value of my_timeseries in 2 hours,
based on the last hour of data.
2015-08-05 15:16:49 +01:00
Fabian Reinartz 579fdf65e2 Implement unary expression for vector types.
Closes #956
2015-08-04 15:46:36 +02:00
Fabian Reinartz c322422412 Merge pull request #954 from prometheus/fabxc/fuzz-fix
Add missing check for nil expression
2015-08-03 16:48:20 +02:00
Fabian Reinartz adf109795c forbid unexpected (runtime) errors in parse tests 2015-08-03 12:53:31 +02:00
Fabian Reinartz c20e25f718 Add missing check for nil expression 2015-08-03 12:28:40 +02:00
Brian Brazil a0f0b82348 promql: Test errors aren't always ParseErr 2015-08-02 23:26:21 +01:00
Fabian Reinartz 5279d50d92 Handle parser runtime panics gracefully 2015-08-02 13:42:18 +02:00
Julius Volz 4e4b468fba Fix lexer bug treating non-Latin Unicode digits as digits.
Fixes https://github.com/prometheus/prometheus/issues/939
2015-07-29 02:11:13 +02:00
Fabian Reinartz 3d67d75935 promql: implement JSON array format for scalar and string 2015-07-06 13:09:26 +02:00
Fabian Reinartz 77e8983221 promql: add MarshalJSON method for SamplePair 2015-07-06 10:29:59 +02:00
Fabian Reinartz c1d37bc55b Merge pull request #843 from prometheus/fabxc/runbook
promql: add runbook to alert statement.
2015-06-25 14:07:45 +02:00
Fabian Reinartz 70d7a987a7 promql: add json tags, fix query constructor. 2015-06-25 13:44:05 +02:00
Fabian Reinartz 749ae450c5 promql: add runbook to alert statement.
This commit adds the RUNBOOK keyword to alert statements. The field
is optional and expected to be a link.
2015-06-25 13:00:52 +02:00
Fabian Reinartz 7f85b9b215 promql: add MarshalJSON method for ExprType. 2015-06-25 12:01:26 +02:00
Fabian Reinartz 1eff186555 Merge pull request #810 from prometheus/fabxc/lmatch
Match empty labels.
2015-06-22 15:45:50 +02:00
Fabian Reinartz 5b91ea9b36 storage: improve label matching and allow unset matching.
Matching of empty labels now also matches metrics where the label
was not explicitly set to the empty string.
2015-06-22 15:33:44 +02:00
Fabian Reinartz 94cd321be1 promql: error if all label matchers are empty. 2015-06-22 15:33:44 +02:00
Fabian Reinartz fe301d7946 promql: remove global flags 2015-06-15 19:01:06 +02:00
Julius Volz 5e2d1c1464 Deprecate `keeping_extra`, rename it to `keep_common`.
`keep_common` is more in line with the function name
`drop_common_labels()` terminology-wise, and also more in line with
`group_left`/`group_right` (no `...ing` verb suffix).

We could also go the full way and call it `keep_common_labels`. That
would have the benefit of being even more consistent with the function
`drop_common_labels()` and would be more explanatory, but it also seems
quite long.
2015-06-12 14:21:05 +02:00
Fabian Reinartz e7659f908c promql: remove DotGraph methods from nodes. 2015-06-12 09:48:14 +02:00
Fabian Reinartz c716d8a47b promql: fix aggregation expression String() method.
Fixes #794.
2015-06-12 09:48:01 +02:00
Fabian Reinartz c32ae22119 promql: fix missing metric in range results. 2015-06-11 23:50:53 +02:00
Fabian Reinartz 0acd44b0e3 promql: expose ParseMetric and ParseMetricSelector 2015-06-11 12:22:11 +02:00
Fabian Reinartz cb10ceac18 promql: allow scalar expressions in range queries, improve errors.
These changes allow to do range queries over scalar expressions.
Errors on bad types for range queries are now raised on query creation
rather than evaluation.
2015-06-10 18:36:02 +02:00
Fabian Reinartz ab9c98acac web/api: add initial API v1 implementation. 2015-06-06 21:47:36 +02:00
Fabian Reinartz 03b737322e promql: expose storage and query engine for tests 2015-06-04 19:00:22 +02:00
Fabian Reinartz 319068a7a6 promql: fix parsing ambiguity for serial repetitions 2015-06-04 19:00:22 +02:00
Fabian Reinartz c60e3387ef promql: fix lexing of \r as whitespaces 2015-06-02 18:33:49 +02:00
Fabian Reinartz 0de6edbdfc Move pkg/ to util/ 2015-06-01 21:12:32 +02:00
Fabian Reinartz dbc0d30e3e Move string functionality to pkg/strutil 2015-06-01 21:12:32 +02:00
Fabian Reinartz ccf51b132e Move stats package to pkg/stats 2015-06-01 21:12:31 +02:00
Fabian Reinartz 3c8fbf1e15 Move test package to pkg/testutil 2015-06-01 21:12:31 +02:00
Brian Brazil f34de493d5 Add increase() function, to replace delta(..., 1).
This calculates how much a counter increases over
a given period of time, which is the area under the curve
of it's rate.

increase(x[5m]) is equivilent to rate(x[5m]) * 300.
2015-05-26 22:49:21 +01:00
Julius Volz d44a89c6e8 Implement changes() function.
changes() takes a range vector and returns the number of times a value
has changed in the given time window for each time series as an instant
vector.
2015-05-26 19:06:22 +02:00
Julius Volz 6f33ed9e59 Add resets() function to count counter resets.
resets() returns for every range vector element how many counter
resets there have been in the specified range.
2015-05-26 17:56:52 +02:00
Fabian Reinartz b7eb105ffa Remove test flags in help output 2015-05-23 12:23:33 +02:00
beorn7 3b9c421a69 Weed out all the [Gg]et* method names.
The only exception is getNumChunksToPersist to avoid naming the struct
member numChunksToPersist in a weird way.
2015-05-20 19:13:06 +02:00
Fabian Reinartz ac4d63b833 Merge pull request #689 from prometheus/fabxc/qltest
Add basic testing language, migrate tests
2015-05-18 19:22:48 +02:00
Fabian Reinartz 0d3012a605 Migrate matrix tests, remove old test files. 2015-05-18 17:50:12 +02:00
Fabian Reinartz 71ef7ab405 Migrate remaining vector evaluation tests to new testing language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz 3c22eded97 Migrate literal tests to testing language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz eba07a7d3d Migrate histogram tests to test language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz 03094eff04 Migrate parsing error tests.
The promql_test checks failure of various bad syntaxed queries.
Those are moved into the parser tests as the new testing language
only deals with valid queries.
2015-05-18 17:47:47 +02:00
Fabian Reinartz 6321964738 Add parsing and execution of new test format.
This commit adds a new test structure that parses and executes
the new testing language.
2015-05-18 17:47:47 +02:00
Fabian Reinartz a236c01457 Add time series description parsing.
This commit adds parsing of time series description to the exisiting
query language parser. Time series descriptions are defined by a
metric followed by a sequence of values.
2015-05-18 17:29:32 +02:00
Fabian Reinartz d122749b39 Enhance lexer testing output 2015-05-18 17:19:17 +02:00
Fabian Reinartz ce487f763e Simplify vector binary evaluation logic 2015-05-17 00:02:34 +02:00
Fabian Reinartz 8a109e061b Extract OR operation into own eval method. 2015-05-16 14:00:11 +02:00
Fabian Reinartz 2c3e9e2e87 Extract AND operation into own eval method. 2015-05-16 13:33:03 +02:00
Fabian Reinartz 4234a45d2e Merge pull request #693 from prometheus/fabxc/preload_timeout
Check context before each preloading.
2015-05-12 12:25:22 +02:00
Fabian Reinartz 8ea94ebb58 Check context before each preloading. 2015-05-12 12:03:30 +02:00
Fabian Reinartz 969c231191 Make parser more strict about identifiers, extract number parsing 2015-05-11 11:45:23 +02:00
Fabian Reinartz 8707c54508 Fix single quote parsing, add tests 2015-05-08 16:43:02 +02:00
Fabian Reinartz 9ab1f6c690 Limit maximum number of concurrent queries.
A high number of concurrent queries can slow each other down
so that none of them is reasonbly responsive. This commit limits
the number of queries being concurrently executed.
2015-05-06 11:34:17 +02:00
Fabian Reinartz d59d1cb2c1 Fix Error() methods. 2015-05-01 17:58:58 +02:00
Fabian Reinartz 6649306e63 Merge pull request #660 from prometheus/fabxc/pql/parse-errs
Fix and improve parsing error output.
2015-04-30 13:31:13 +02:00
Fabian Reinartz 279831cdf1 Fix and improve parsing error output. 2015-04-30 12:19:39 +02:00
Fabian Reinartz fe935179cd Stop routing rule statements through the engine. 2015-04-29 18:01:43 +02:00
Fabian Reinartz 25cdff3527 Remove `name` arg from `Parse*` functions, enhance parsing errors. 2015-04-29 16:38:41 +02:00
Fabian Reinartz 5602328c7c Refactor query evaluation.
This copies the evaluation logic from the current rules/ package.
The new engine handles the execution process from query string to final result.
It provides query timeout and cancellation and general flexibility for
future changes.

functions.go: Add evaluation implementation. Slight changes to in/out data but
	not to the processing logic.
quantile.go: No changes.
analyzer.go: No changes.
engine.go: Actually new part. Mainly consists of evaluation methods
	which were not changed.
setup_test.go: Copy of rules/helpers_test.go to setup test storage.
promql_test.go: Copy of rules/rules_test.go.
2015-04-28 14:19:05 +02:00
Fabian Reinartz 32b7595c47 Create promql package with lexer/parser.
This commit creates a (so far unused) package. It contains the a custom
lexer/parser for the query language.

ast.go: New AST that interacts well with the parser.
lex.go: Custom lexer (new).
lex_test.go: Lexer tests (new).
parse.go: Custom parser (new).
parse_test.go: Parser tests (new).
functions.go: Changed function type, dummies for parser testing (barely changed/dummies).
printer.go: Adapted from rules/ and adjusted to new AST (mostly unchanged, few additions).
2015-04-23 16:04:50 +02:00