prometheus

Commit Graph

Author	SHA1	Message	Date
Brian Brazil	e041c0cd46	Add console and alert templates with access to all data. Move rulemanager to it's own package to break cicrular dependency. Make NewTestTieredStorage available to tests, remove duplication. Change-Id: I33b321245a44aa727bfc3614a7c9ae5005b34e03	11 years ago
Bjoern Rabenstein	ca6a4fccef	Weed out our homegrown test.Tester. The Go stdlib has testing.TB now, which fulfills the exact same purpose. Change-Id: I0db9c73400e208ca376b932a02b7e3402234b87c	11 years ago
Julius Volz	01f652cb4c	Separate storage implementation from interfaces. This was initially motivated by wanting to distribute the rule checker tool under `tools/rule_checker`. However, this was not possible without also distributing the LevelDB dynamic libraries because the tool transitively depended on Levigo: rule checker -> query layer -> tiered storage layer -> leveldb This change separates external storage interfaces from the implementation (tiered storage, leveldb storage, memory storage) by putting them into separate packages: - storage/metric: public, implementation-agnostic interfaces - storage/metric/tiered: tiered storage implementation, including memory and LevelDB storage. I initially also considered splitting up the implementation into separate packages for tiered storage, memory storage, and LevelDB storage, but these are currently so intertwined that it would be another major project in itself. The query layers and most other parts of Prometheus now have notion of the storage implementation anymore and just use whatever implementation they get passed in via interfaces. The rule_checker is now a static binary :) Change-Id: I793bbf631a8648ca31790e7e772ecf9c2b92f7a0	11 years ago
Julius Volz	d411a7d810	Allow reversing vector and scalar arguments in binops. This allows putting a scalar as the first argument of a binary operator in which the second argument is a vector: <scalar> <binop> <vector> For example, 1 / http_requests_total ...will output a vector in which every sample value is 1 divided by the respective input vector element. This even works for filter binary operators now: 1 == http_requests_total Returns a vector with all values set to 1 for every element in http_requests_total whose initial value was 1. Note: For filter binary operators, the resulting values are always taken from the left-hand-side of the operation, no matter whether the scalar or the vector argument is the left-hand-side. That is, 1 != http_requests_total ...will set all result vector sample values to 1, although these are exactly the sample elements that were != 1 in the input vector. If you want to just filter elements without changing their sample values, you still need to do: http_requests_total != 1 The new filter form is a bit exotic, and so probably won't be used often. But it was easier to implement it than disallow it completely or change its behavior. Change-Id: Idd083f2bd3a1219ba1560cf4ace42f5b82e797a5	11 years ago
Julius Volz	c7c0b33d0b	Add regex-matching support for labels. There are four label-matching ops for selecting timeseries now: - Equal: = - NotEqual: != - RegexMatch: =~ - RegexNoMatch: !~ Instead of looking up labels by a simple clientmodel.LabelSet (basically an equals op for every key/value pair in the set), timeseries fingerprint selection is now done via a list of metric.LabelMatchers. Change-Id: I510a83f761198e80946146770ebb64e4abc3bb96	11 years ago
Julius Volz	7e9ecaac3a	Add count_scalar() function. Change-Id: I63f09dd0479d0a6b016f5f857dd39dcbda56c7f9	11 years ago
Julius Volz	0378c2ca1f	Nonexistent labels in BY-clauses shouldn't propagate to result. This fixes bug 2. of https://github.com/prometheus/prometheus/issues/374 Change-Id: Ia4a13153616bafce5bf10597966b071434422d09	11 years ago
Julius Volz	6dc36d0c3e	Don't keep extra labels in aggregations by default. MIN/MAX/SUM/AVG/COUNT aggregations will now by default drop all labels that are not specifically part of a BY-clause, even if a label value is the same within all timeseries of an aggregation group. The old behavior of keeping extra labels may still be switched on by adding KEEPING_EXTRA to the end of an aggregation statement: sum(http_requests) by (job, method) keeping_extra I'm open to better syntax/naming suggestions. Change-Id: I21d3fe7af9e98552ce3dffa3ce7c0a4ba4c0b4a4	11 years ago
Julius Volz	740d448983	Use custom timestamp type for sample timestamps and related code. So far we've been using Go's native time.Time for anything related to sample timestamps. Since the range of time.Time is much bigger than what we need, this has created two problems: - there could be time.Time values which were out of the range/precision of the time type that we persist to disk, therefore causing incorrectly ordered keys. One bug caused by this was: https://github.com/prometheus/prometheus/issues/367 It would be good to use a timestamp type that's more closely aligned with what the underlying storage supports. - sizeof(time.Time) is 192, while Prometheus should be ok with a single 64-bit Unix timestamp (possibly even a 32-bit one). Since we store samples in large numbers, this seriously affects memory usage. Furthermore, copying/working with the data will be faster if it's smaller. MEMORY USAGE RESULTS Initial memory usage comparisons for a running Prometheus with 1 timeseries and 100,000 samples show roughly a 13% decrease in total (VIRT) memory usage. In my tests, this advantage for some reason decreased a bit the more samples the timeseries had (to 5-7% for millions of samples). This I can't fully explain, but perhaps garbage collection issues were involved. WHEN TO USE THE NEW TIMESTAMP TYPE The new clientmodel.Timestamp type should be used whenever time calculations are either directly or indirectly related to sample timestamps. For example: - the timestamp of a sample itself - all kinds of watermarks - anything that may become or is compared to a sample timestamp (like the timestamp passed into Target.Scrape()). When to still use time.Time: - for measuring durations/times not related to sample timestamps, like duration telemetry exporting, timers that indicate how frequently to execute some action, etc. NOTE ON OPERATOR OPTIMIZATION TESTS We don't use operator optimization code anymore, but it still lives in the code as dead code. It still has tests, but I couldn't get all of them to pass with the new timestamp format. I commented out the failing cases for now, but we should probably remove the dead code soon. I just didn't want to do that in the same change as this. Change-Id: I821787414b0debe85c9fffaeb57abd453727af0f	11 years ago
Julius Volz	35ee2cd3cb	Add alertmanager notification support to Prometheus. Alert definitions now also have mandatory SUMMARY and DESCRIPTION fields that get sent along a firing alert to the alert manager.	11 years ago
Julius Volz	64b0ade171	Swap rules lexer for much faster one. This swaps github.com/kivikakk/golex for github.com/cznic/golex. The old lexer would have taken 3.5 years to load a set of 5000 test rules (quadratic time complexity for input length), whereas this one takes only 32ms. Furthermore, since the new lexer is embedded differently, this gets rid of the global parser variables and makes the rule loader fully reentrant without a lock.	12 years ago
Matt T. Proud	30b1cf80b5	WIP - Snapshot of Moving to Client Model.	12 years ago
Julius Volz	fc97e688c6	Improve printing of rules and expressions.	12 years ago
Matt T. Proud	2c3df44af6	Ensure database access waits until it is started. This commit introduces a channel message to ensure serving state has been reached with the storage stack before anything attempts to use it.	12 years ago
Julius Volz	51689d965d	Add debug timers to instant and range queries. This adds timers around several query-relevant code blocks. For now, the query timer stats are only logged for queries initiated through the UI. In other cases (rule evaluations), the stats are simply thrown away. My hope is that this helps us understand where queries spend time, especially in cases where they sometimes hang for unusual amounts of time.	12 years ago
Julius Volz	138334fb31	Fix handling of negative deltas for non-counter values.	12 years ago
Julius Volz	750f862d9a	Use GetBoundaryValues() for non-counter deltas.	12 years ago
Bernerd Schaefer	63d9988b9c	Drop unused writeMemoryInterval	12 years ago
Bernerd Schaefer	aa96c7d141	Fix rules_test.go This is smelly, but for now we copy a helper method from the metric tests into rules.	12 years ago
Julius Volz	0877680761	Implement a COUNT ... BY aggregation operator. This also removes the now obsolete scalar count() function and corrects the expressions test naming (broken in `2202cd71c9 (L6R59)`) so that the expression tests will actually run.	12 years ago
Julius Volz	56324d8ce2	Make AST query storage non-global.	12 years ago
Matt T. Proud	ce45787dbf	Storage interface to TieredStorage. This commit drops the Storage interface and just replaces it with a publicized TieredStorage type. Storage had been anticipated to be used as a wrapper for testability but just was not used due to practicality. Merely overengineered. My bad. Anyway, we will eventually instantiate the TieredStorage dependencies in main.go and pass them in for more intelligent lifecycle management. These changes will pave the way for managing the curators without Law of Demeter violations.	12 years ago
Julius Volz	2202cd71c9	Track alerts over time and write out alert timeseries.	12 years ago
Julius Volz	c0601abf46	Implement initial no-op alert parsing and rule parsing tests.	12 years ago
Matt T. Proud	f9e99bd08a	Refresh SampleValue to 64-bit floating point. We always knew that this needed to be fixed.	12 years ago
Julius Volz	5f5ea03105	Run "make format".	12 years ago
Julius Volz	1cff4f3d91	Fix rate() per-second adjustment. This got broken during the depointerization of the Vector type.	12 years ago
juliusv	62f33f1fc2	Merge pull request #138 from prometheus/julius-fix-aliasing Correct delta()/rate() intervals and temporal aliasing.	12 years ago
Julius Volz	d53b8cf956	Correct delta()/rate() intervals and temporal aliasing.	12 years ago
Julius Volz	000f6a2e23	Allow lower-case BY operator.	12 years ago
Julius Volz	1bc83e1b65	Also allow lower-cased aggregation ops.	12 years ago
Julius Volz	6cb3c51d24	Add sort() and sort_desc() expression language functions.	12 years ago
Julius Volz	e31591e6fe	Allow single-letter identifiers (metric and label names).	12 years ago
Julius Volz	ec413459fa	Depointerize Matrix/Vector types as well as time.Time arguments.	12 years ago
Julius Volz	55ca65aa6e	More userfriendly output when we fail to create the tiered storage.	12 years ago
Matt T. Proud	b86b0ea41a	Create temporary directory handler.	12 years ago
Julius Volz	8e4c5b0cea	Use AST query analyzer and views with tiered storage.	12 years ago
Matt T. Proud	41068c2e84	Checkpoint.	12 years ago
Julius Volz	16d9dcd6a8	Add copyright notices to all remaining files.	12 years ago
Matt T. Proud	ea54751431	Update import paths to new location. This repository moved from matttproud/prometheus to prometheus/prometheus, and all import paths need to be updated.	12 years ago
Julius Volz	c049ae39af	Cleanups to rules/persistence adapter code.	12 years ago
Julius Volz	a85204a0a4	Add support for matrix duration strings without quotes.	12 years ago
Matt T. Proud	efe61c18fa	Refactor target scheduling to separate facility. ``Target`` will be refactored down the road to support various nuanced endpoint types. Thusly incorporating the scheduling behavior within it will be problematic. To that end, the scheduling behavior has been moved into a separate assistance type to improve conciseness and testability. ``make format`` was also run.	12 years ago
Julius Volz	483bd81a44	Allow grammar to parse both rules and single expressions.	12 years ago
Julius Volz	56384bf42a	Add initial config and rule language implementation.	12 years ago

45 Commits (aed1d384a911e0931a587990ee76ec8b6b369d73)