prometheus

Commit Graph

Author	SHA1	Message	Date
Julius Volz	d4374a9265	More efficient JSON query result format. This depends on https://github.com/prometheus/client_golang/pull/51. For vectors, the result format looks like this: ```json { "version": 1, "type" : "vector", "value" : [ { "timestamp" : 1421765411.045, "value" : "65.475000", "metric" : { "quantile" : "0.5", "instance" : "http://localhost:9090/metrics", "job" : "prometheus", "__name__" : "http_request_duration_microseconds", "handler" : "/static/", "method" : "get", "code" : "304" } }, { "timestamp" : 1421765411.045, "value" : "5826.339000", "metric" : { "quantile" : "0.9", "instance" : "http://localhost:9090/metrics", "job" : "prometheus", "__name__" : "http_request_duration_microseconds", "handler" : "prometheus", "method" : "get", "code" : "200" } }, /* ... / ] } ``` For matrices, it looks like this: ```json { "version": 1, "type" : "matrix", "value" : [ { "metric" : { "quantile" : "0.99", "instance" : "http://localhost:9090/metrics", "job" : "prometheus", "__name__" : "http_request_duration_microseconds", "handler" : "/static/", "method" : "get", "code" : "200" }, "values" : [ [ 1421765547.659, "29162.953000" ], [ 1421765548.659, "29162.953000" ], [ 1421765549.659, "29162.953000" ], / ... */ ] } ] } ```	10 years ago
Bjoern Rabenstein	5859b74f1b	Clean up license issues. - Move CONTRIBUTORS.md to the more common AUTHORS. - Added the required NOTICE file. - Changed "Prometheus Team" to "The Prometheus Authors". - Reverted the erroneous changes to the Apache License.	10 years ago
Julius Volz	cc27fb8aab	Rename remaining all-caps constants in AST layer. Change-Id: Ibe97e30981969056ffcdb89e63c1468ea1ffa140	10 years ago
Julius Volz	00a2a93a05	Add regression tests for metrics mutations in AST. It turned out in the end, that only drop_common_metrics() produced any erroneous output in the old system. The second expression in the test ("sum(testmetric) keeping_extra") already worked in the old code, but why not keep it in... The way to test ranged evaluations is a bit clumsy so far, so I want to build a nicer test framework in the end, where all the test cases can be specified as text files which specify desired inputs, outputs, query step widths, etc. Change-Id: I821859789e69b8232bededf670a1b76e9e8c8ca4	10 years ago
Julius Volz	c9618d11e8	Introduce copy-on-write for metrics in AST. This depends on changes in: https://github.com/prometheus/client_golang/tree/cow-metrics. Change-Id: I80b94833a60ddf954c7cd92fd2cfbebd8dd46142	10 years ago
Bjoern Rabenstein	b1e4956142	Apply a giant code cleanup. Essentially: - Remove unused code. - Make it 'go vet' clean. The only remaining warnings are in generated code. - Make it 'golint' clean. The only remaining warnings are in gerenated code. - Smoothed out same minor things. Change-Id: I3fe5c1fbead27b0e7a9c247fee2f5a45bc2d42c6	10 years ago
Bjoern Rabenstein	f5f9f3514a	Major code cleanup. - Make it go-vet and golint clean. - Add comments, TODOs, etc. Change-Id: If1392d96f3d5b4cdde597b10c8dff1769fcfabe2	10 years ago
Julius Volz	e7ed39c9a6	Initial experimental snapshot of next-gen storage. Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d	10 years ago
Brian Brazil	e041c0cd46	Add console and alert templates with access to all data. Move rulemanager to it's own package to break cicrular dependency. Make NewTestTieredStorage available to tests, remove duplication. Change-Id: I33b321245a44aa727bfc3614a7c9ae5005b34e03	11 years ago
Julius Volz	6297a405f2	Do not indent API JSON responses. In one example response, this reduced the uncompressed size by 25% and the gzipped size by 11%. Change-Id: Ie80d44253124b9f8601b8ef9fc978e92dacff523	11 years ago
Julius Volz	01f652cb4c	Separate storage implementation from interfaces. This was initially motivated by wanting to distribute the rule checker tool under `tools/rule_checker`. However, this was not possible without also distributing the LevelDB dynamic libraries because the tool transitively depended on Levigo: rule checker -> query layer -> tiered storage layer -> leveldb This change separates external storage interfaces from the implementation (tiered storage, leveldb storage, memory storage) by putting them into separate packages: - storage/metric: public, implementation-agnostic interfaces - storage/metric/tiered: tiered storage implementation, including memory and LevelDB storage. I initially also considered splitting up the implementation into separate packages for tiered storage, memory storage, and LevelDB storage, but these are currently so intertwined that it would be another major project in itself. The query layers and most other parts of Prometheus now have notion of the storage implementation anymore and just use whatever implementation they get passed in via interfaces. The rule_checker is now a static binary :) Change-Id: I793bbf631a8648ca31790e7e772ecf9c2b92f7a0	11 years ago
Julius Volz	c7c0b33d0b	Add regex-matching support for labels. There are four label-matching ops for selecting timeseries now: - Equal: = - NotEqual: != - RegexMatch: =~ - RegexNoMatch: !~ Instead of looking up labels by a simple clientmodel.LabelSet (basically an equals op for every key/value pair in the set), timeseries fingerprint selection is now done via a list of metric.LabelMatchers. Change-Id: I510a83f761198e80946146770ebb64e4abc3bb96	11 years ago
Julius Volz	3f226c9724	Rename {Scalar,Vector}Literal to {Scalar,Vector}Selector. Change-Id: Ie92301f47f5f49f30b3a62c365e377108982b080	11 years ago
Bjoern Rabenstein	fd63500ed3	Make rules/ast golint clean. Mostly, that means adding compliant doc strings to exported items. Also, remove 'go vet' warnings where possible. (Some are unfortunately not to avoid, arguably bugs in 'go vet'.) Change-Id: I2827b6dd317492864c1383c3de1ea9eac5a219bb	11 years ago
Julius Volz	7e9ecaac3a	Add count_scalar() function. Change-Id: I63f09dd0479d0a6b016f5f857dd39dcbda56c7f9	11 years ago
Julius Volz	740d448983	Use custom timestamp type for sample timestamps and related code. So far we've been using Go's native time.Time for anything related to sample timestamps. Since the range of time.Time is much bigger than what we need, this has created two problems: - there could be time.Time values which were out of the range/precision of the time type that we persist to disk, therefore causing incorrectly ordered keys. One bug caused by this was: https://github.com/prometheus/prometheus/issues/367 It would be good to use a timestamp type that's more closely aligned with what the underlying storage supports. - sizeof(time.Time) is 192, while Prometheus should be ok with a single 64-bit Unix timestamp (possibly even a 32-bit one). Since we store samples in large numbers, this seriously affects memory usage. Furthermore, copying/working with the data will be faster if it's smaller. MEMORY USAGE RESULTS Initial memory usage comparisons for a running Prometheus with 1 timeseries and 100,000 samples show roughly a 13% decrease in total (VIRT) memory usage. In my tests, this advantage for some reason decreased a bit the more samples the timeseries had (to 5-7% for millions of samples). This I can't fully explain, but perhaps garbage collection issues were involved. WHEN TO USE THE NEW TIMESTAMP TYPE The new clientmodel.Timestamp type should be used whenever time calculations are either directly or indirectly related to sample timestamps. For example: - the timestamp of a sample itself - all kinds of watermarks - anything that may become or is compared to a sample timestamp (like the timestamp passed into Target.Scrape()). When to still use time.Time: - for measuring durations/times not related to sample timestamps, like duration telemetry exporting, timers that indicate how frequently to execute some action, etc. NOTE ON OPERATOR OPTIMIZATION TESTS We don't use operator optimization code anymore, but it still lives in the code as dead code. It still has tests, but I couldn't get all of them to pass with the new timestamp format. I commented out the failing cases for now, but we should probably remove the dead code soon. I just didn't want to do that in the same change as this. Change-Id: I821787414b0debe85c9fffaeb57abd453727af0f	11 years ago
Matt T. Proud	30b1cf80b5	WIP - Snapshot of Moving to Client Model.	12 years ago
Julius Volz	0226d1ac7a	Implement alerts dashboard and expression console links.	12 years ago
Julius Volz	fc97e688c6	Improve printing of rules and expressions.	12 years ago
Julius Volz	74cb676537	Implement Stringer interface for rules and all their children.	12 years ago
Julius Volz	51689d965d	Add debug timers to instant and range queries. This adds timers around several query-relevant code blocks. For now, the query timer stats are only logged for queries initiated through the UI. In other cases (rule evaluations), the stats are simply thrown away. My hope is that this helps us understand where queries spend time, especially in cases where they sometimes hang for unusual amounts of time.	12 years ago
Julius Volz	0877680761	Implement a COUNT ... BY aggregation operator. This also removes the now obsolete scalar count() function and corrects the expressions test naming (broken in `2202cd71c9 (L6R59)`) so that the expression tests will actually run.	12 years ago
Julius Volz	56324d8ce2	Make AST query storage non-global.	12 years ago
Julius Volz	dcf2e82752	Cleanup and idiomaticize rule/expression dot graph output.	12 years ago
Julius Volz	6cb3c51d24	Add sort() and sort_desc() expression language functions.	12 years ago
Julius Volz	ec413459fa	Depointerize Matrix/Vector types as well as time.Time arguments.	12 years ago
Julius Volz	2b8f0b2cc7	Constantize metric name label name.	12 years ago
Julius Volz	8e4c5b0cea	Use AST query analyzer and views with tiered storage.	12 years ago
Matt T. Proud	5959cd9e53	Include Julius' feedback.	12 years ago
Matt T. Proud	a70ee43ad3	Niladic ``ToString()`` to idiomatic ``String()``.	12 years ago
Julius Volz	c3d31febd6	Move durationToString to common place and cleanup error handling.	12 years ago
Julius Volz	16d9dcd6a8	Add copyright notices to all remaining files.	12 years ago
Julius Volz	1760d927c8	Add error propagation to web UI via special JSON error type.	12 years ago
Julius Volz	93670aa129	Return API errors in JSON format.	12 years ago
Julius Volz	a20bf35997	Fix whitespace with "make format".	12 years ago
Julius Volz	2c8595f96e	First graphing support.	12 years ago
Matt T. Proud	efe61c18fa	Refactor target scheduling to separate facility. ``Target`` will be refactored down the road to support various nuanced endpoint types. Thusly incorporating the scheduling behavior within it will be problematic. To that end, the scheduling behavior has been moved into a separate assistance type to improve conciseness and testability. ``make format`` was also run.	12 years ago
Julius Volz	17a4a442b3	Add REST API, expression browser, and text/JSON output formats.	12 years ago
Julius Volz	56384bf42a	Add initial config and rule language implementation.	12 years ago

39 Commits (63a79821fce78285a6a91363859da1c2646dbfe0)