prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	a7f19b5775	labels: add a test for JSON and YAML marshalling Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	10699c37a3	labels: test BytesWithoutLabels does not remove __name__ by default Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	d46ef0aa8e	labels: tweak BenchmarkLabels_Get() So the benchmark works without requiring `Labels` internals to be a slice. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	24ebff9c4a	labels: don't test that Hash() works on unordered labels Labels are required to be ordered. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	fb2d883f1e	labels: test Compare without knowing the exact result Only the sign of the result is important Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Paschalis Tsilias	d1122e0743	Introduce TSDB changes for appending metadata to the WAL (#10972 ) * Append metadata to the WAL Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Remove extra whitespace; Reword some docstrings and comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use RLock() for hasNewMetadata check Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use single byte for metric type in RefMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Update proposed WAL format for single-byte type metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Implementa MetadataAppender interface for the Agent Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Address first round of review comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Amend description of metadata in wal.md Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Correct key used to retrieve metadata from cache When we're setting metadata entries in the scrapeCace, we're using the p.Help(), p.Unit(), p.Type() helpers, which retrieve the series name and use it as the cache key. When checking for cache entries though, we used p.Series() as the key, which included the metric name _with_ its labels. That meant that we were never actually hitting the cache. We're fixing this by utiling the __name__ internal label for correctly getting the cache entries after they've been set by setHelp(), setType() or setUnit(). Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Put feature behind a feature flag Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix AppendMetadata docstring Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reorder WAL format document Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Change error message of AppendMetadata; Fix access of s.meta in AppendMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reuse temporary buffer in Metadata encoder Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Only keep latest metadata for each refID during checkpointing Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix test that's referencing decoding metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Avoid creating metadata block if no new metadata are present Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add tests for corrupt metadata block and relevant record type Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix CR comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Extract logic about changing metadata in an anonymous function Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Implement new proposed WAL format and amend relevant tests Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use 'const' for metadata field names Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Apply metadata to head memSeries in Commit, not in AppendMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add docstring and rename extracted helper in scrape.go Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add tests for tsdb-related cases Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linter issues vol1 Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linter issues vol2 Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix Windows test by closing WAL reader files Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use switch instead of two if statements in metadata decoding Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix review comments around TestMetadata* tests Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add code for replaying WAL; test correctness of in-memory data after a replay Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Remove scrape-loop related code from PR Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Address first round of comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Simplify tests by sorting slices before comparison Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix test to use separate transactions Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Empty out buffer and record slices after encoding latest metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linting issue Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Update calculation for DroppedMetadata metric Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Rename MetadataAppender interface and AppendMetadata method to MetadataUpdater/UpdateMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reuse buffer when encoding latest metadata for each series Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix review comments; Check all returned error values using two helpers Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Simplify use of helpers Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Satisfy linter Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>	2 years ago
Julien Pivotto	c637705403	Merge pull request #10893 from prymitive/unwrap_errors Implement Unwrap() on errors returned from rulefmt	2 years ago
Matthieu MOREL	c2b4de3611	refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10747 ) Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com> Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	2 years ago
Łukasz Mierzwa	648b12d8c5	Implement Unwrap() on errors returned from rulefmt I'd like to unwrap errors returned from rulefmt but both Error and WrappedError types are missing Unwrap() method. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	2 years ago
Paschalis Tsilias	4f3791024a	Drop extra string held in relabel.Regexp struct (#10846 ) * Drop extra string held in relabel.Regexp struct Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use slice operations instead of TrimPrefix/TrimSuffix; Override String() method Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>	2 years ago
Bryan Boreham	2e2c014d52	Labels: optimise creation of signature with/without labels (#10667 ) * Labels: create signature with/without labels Instead of creating a new Labels slice then converting to signature, go directly to the signature and save time. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Labels: refactor Builder tests Have one test with a range of cases, and have them check the final output rather than checking the internal structure of the Builder. Also add a couple of cases where the value is "", which should be interpreted as 'delete'. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Labels: add 'Keep' function to Builder This lets us replace `Labels.WithLabels` with the more general `Builder`. In `engine.resultMetric()` we can call `Keep()` instead of checking and calling `Del()`. Avoid calling `Sort()` in `Builder.Labels()` if we didn't add anything, so that `Keep()` has the same performance as `WithLabels()`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Łukasz Mierzwa	b11062bfcc	Don't count added labels when initializing slice (#10756 ) This was added in #10749 but based on feedback it might be over-allocating so worth removing. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Łukasz Mierzwa	08262454a3	Preallocate Labels in labels.Builder (#10749 ) This tries to avoid re-allocations of labels slice since we know possible max size Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Łukasz Mierzwa	89de30a0b7	Avoid reallocating map in MatchLabels (#10715 ) We know the max size of our map so we can create it with that information and avoid extra allocations Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Julien Pivotto	71dbb4d091	Add lowercase and uppercase relabel action (#10641 ) Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	3 years ago
Matthieu MOREL	e2ede285a2	refactor: move from io/ioutil to io and os packages (#10528 ) * refactor: move from io/ioutil to io and os packages * use fs.DirEntry instead of os.FileInfo after os.ReadDir Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>	3 years ago
Peter Štibraný	cd18da3605	Fix OpenMetrics parser to sort uppercase labels correctly. (#10510 ) Signed-off-by: Peter Štibraný <pstibrany@gmail.com>	3 years ago
Nick Pillitteri	53ac9d6d66	Compare equality of label.Label structs directly (#10427 ) * labels.Equal benchmark for equal, not equal, and differing lengths Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Compare equality of label.Label structs directly Compare the structs using `==` instead of the name and value of each label. This is functionally equivalent and about ~10% faster in my testing. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use longer more realistic names and values in benchmark Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	3 years ago
Levi Harrison	a9e2ffb717	Specify type in metadata parser errors (#10269 )	3 years ago
Łukasz Mierzwa	da23c4649a	Enable misspell check in golangci-lint (#10393 ) Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Matthias Loibl	afdd1357e0	Merge pull request #10267 from roidelapluie/contenttype Improve content-type error handling	3 years ago
Julien Pivotto	9a2e93228e	Switch to grafana/regexp everywhere (#10268 ) Let's have a consistent library for regexp. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Bryan Boreham	579331446a	Allow downstream projects to use faster regexp (#10251 ) * Add benchmark for FastRegexMatcher Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Use modified regexp package with optimisations See https://github.com/grafana/regexp/tree/speedup#readme Includes the following changes proposed upstream: * [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711) * [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909) * [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756) * [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789) * [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294) Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Use regexp code identical to upstream Go Change `grafana/regexp` import to use `main` branch. This means Prometheus is not using the proposed optimisations, but downstream users of Prometheus code are able to `replace` the library with the `speedup` branch which does. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Julien Pivotto	f695df843f	Improve content-type error handling - Call err everywhere - Change log message to underscore-separated field Followup on #10186 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Matheus Pimenta	8d8ce641a4	error for invalid media type should not be completely swallowed (#10186 ) * error for invalid media type should not be completely swallowed Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>	3 years ago
ozairasim	52159840eb	Expose Error function for WrappedError (#9662 )	3 years ago
beorn7	c954cd9d1d	Move packages out of deprecated pkg directory This creates a new `model` directory and moves all data-model related packages over there: exemplar labels relabel rulefmt textparse timestamp value All the others are more or less utilities and have been moved to `util`: gate logging modetimevfs pool runtime Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Bjoern Rabenstein	006b5517e2	Simplify makefiles. This removes the dependancy on C leveldb and snappy. It also takes care of fewer dependencies as they would anyway not work on any non-Debian, non-Brew system. Change-Id: Ia70dce1ba8a816a003587927e0b3a3f8ad2fd28c	10 years ago
Julius Volz	ae30453214	Add label names -> label values index. Change-Id: Ie39b4044558afc4d1aa937de7dcf8df61f821fb4	11 years ago
Julius Volz	1eee448bc1	Store samples in custom binary encoding. This has been shown to provide immense decoding speed benefits. See also: https://groups.google.com/forum/#!topic/prometheus-developers/FeGl_qzGrYs Change-Id: I7d45b4650e44ddecaa91dad9d7fdb3cd0b9f15fe	11 years ago
Julius Volz	a310afd28f	Update "build" -> ".build" in remaining Makefiles.	12 years ago
Matt T. Proud	30b1cf80b5	WIP - Snapshot of Moving to Client Model.	12 years ago
Julius Volz	0226d1ac7a	Implement alerts dashboard and expression console links.	12 years ago
Matt T. Proud	f895acb9ef	Trailing build system cleanups. The .gitignore files had problems, and the build steps could be simplified.	12 years ago
Julius Volz	fc97e688c6	Improve printing of rules and expressions.	12 years ago
Julius Volz	74cb676537	Implement Stringer interface for rules and all their children.	12 years ago
Julius Volz	dcfd09c801	Prepend "exporter_" to labels that already exist in exported metrics. If the metrics exported by a process already contain any of a target's base labels (such as "job" or "instance", but also any manually assigned target-group label), don't overwrite that label, but instead add a new label consisting of the original label name prepended with "exporter_". This is to accomodate intermediate exporter jobs, which might indicate e.g. the jobs and instances for which they are exporting data.	12 years ago
Julius Volz	5b105c77fc	Repointerize fingerprints.	12 years ago
Julius Volz	259a0ea467	Optimize fingerprint comparison time.	12 years ago
Matt T. Proud	8f4c7ece92	Destroy naked returns in half of corpus. The use of naked return values is frowned upon. This is the first of two bulk updates to remove them.	12 years ago
Matt T. Proud	244a4a9cdb	Update to go1.1. This commit updates the documentation, Makefiles, formatting, and code semantics to support the 1.1. runtime, which includes ... 1. ``make advice``, 2. ``make format``, and 3. ``go fix`` on various targets.	12 years ago
Matt T. Proud	161c8fbf9b	Include deletion processor for long-tail values. This commit extracts the model.Values truncation behavior into the actual tiered storage, which uses it and behaves in a peculiar way—notably the retention of previous elements if the chunk were to ever go empty. This is done to enable interpolation between sparse sample values in the evaluation cycle. Nothing necessarily new here—just an extraction. Now, the model.Values TruncateBefore functionality would do what a user would expect without any surprises, which is required for the DeletionProcessor, which may decide to split a large chunk in two if it determines that the chunk contains the cut-off time.	12 years ago
Matt T. Proud	38d839d810	Include generated Protocol Buffers artifacts. This commit reduces the general compile time dependencies to omit the Protocol Buffer compiler and the Go Protocol Buffer generator tool. The build steps to furnish them still remain, but they can optionally be called if data.proto or config.proto are under work.	12 years ago
Matt T. Proud	3b9b1c6ab4	Define dependencies for web. stack concretely. This commit destroys the use of AppState, which makes passing concrete state along to various serving components onerous.	12 years ago
Matt T. Proud	6fac20c8af	Harden the tests against OOMs. This commit employs explicit memory freeing for the in-memory storage arenas. Secondarily, we take advantage of smaller channel buffer sizes in the test.	12 years ago
Bernerd Schaefer	b04cd28862	Merge pull request #192 from prometheus/feature/negotiate-telemetry-schema-through-mime-type Use Content-Type data for telemetry versioning	12 years ago
Matt T. Proud	b3e34c6658	Implement batch database sample curator. This commit introduces to Prometheus a batch database sample curator, which corroborates the high watermarks for sample series against the curation watermark table to see whether a curator of a given type needs to be run. The curator is an abstract executor, which runs various curation strategies across the database. It remarks the progress for each type of curation processor that runs for a given sample series. A curation procesor is responsible for effectuating the underlying batch changes that are request. In this commit, we introduce the CompactionProcessor, which takes several bits of runtime metadata and combine sparse sample entries in the database together to form larger groups. For instance, for a given series it would be possible to have the curator effectuate the following grouping: - Samples Older than Two Weeks: Grouped into Bunches of 10000 - Samples Older than One Week: Grouped into Bunches of 1000 - Samples Older than One Day: Grouped into Bunches of 100 - Samples Older than One Hour: Grouped into Bunches of 10 The benefits hereof of such a compaction are 1. a smaller search space in the database keyspace, 2. better employment of compression for repetious values, and 3. reduced seek times.	12 years ago
juliusv	348b527f89	Merge pull request #194 from prometheus/julius-alert-tracking Track alerts over time and write out alert timeseries.	12 years ago
Julius Volz	2202cd71c9	Track alerts over time and write out alert timeseries.	12 years ago
Bernerd Schaefer	dfd5c9ce28	Refactor processor for 0.0.2 schema Primary changes: * Strictly typed unmarshalling of metric values * Schema types are contained by the processor (no "type entity002") Minor changes: * Added ProcessorFunc type for expressing processors as simple functions. * Added non-destructive `Merge` method to `model.LabelSet`	12 years ago

1 2

96 Commits (2d9b3f6e967173ad6e52bb235c145718c577b449)