prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	8065bef172	Move metric type definitions to common/model They are used in multiple repos, so common is a better place for them. Several packages now don't depend on `model/textparse`, e.g. `storage/remote`. Also remove `metadata` struct from `api.go`, since it was identical to a struct in the `metadata` package. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	11 months ago
Arthur Silva Sens	5082655392	Append Created Timestamps (#12733 ) * Append created timestamps. Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Log when created timestamps are ignored Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Proposed changes to Append CT PR. Changes: * Changed textparse Parser interface for consistency and robustness. * Changed CT interface to be more explicit and handle validation. * Simplified test, change scrapeManager to allow testability. * Added TODOs. Signed-off-by: bwplotka <bwplotka@gmail.com> * Updates. Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * Refactor head_appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Fix linter issues Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> * Use model.Sample in head appender test Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> --------- Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: bwplotka <bwplotka@gmail.com>	12 months ago
Matthieu MOREL	9c4782f1cc	golangci-lint: enable testifylint linter (#13254 ) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	12 months ago
zenador	32ee1b15de	Fix error on ingesting out-of-order exemplars (#13021 ) Fix and improve ingesting exemplars for native histograms. See code comment for a detailed explanation of the algorithm. Note that this changes the current behavior for all kind of samples slightly: We now allow exemplars with the same timestamp as during the last scrape if the value or the labels have changed. Also note that we now do not ingest exemplars without timestamps for native histograms anymore. Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in> --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: zenador <zenador@users.noreply.github.com> Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Co-authored-by: Björn Rabenstein <github@rabenste.in>	1 year ago
Matthieu MOREL	fe057fc60d	use Go standard errors package Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	1 year ago
beorn7	b428416f06	textparse: Update comment about timestamp_ms protobuf parsing By now, we know better what the plan is. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
Arthur Silva Sens	ef8e6ae780	Parse created timestamps from Prometheus Protobuf (#12973 ) Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	1 year ago
Matthieu MOREL	0a513f827d	ci(lint): enable errorlint linter on model Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	1 year ago
beorn7	65ccf4460a	textparse: Fix endless loop #12731 PR #12557 introduced the possibility of parsing multiple exemplars per native histograms. It did so by requiring the `Exemplar` method of the parser to be called repeatedly until it returns false. However, the protobuf parser code wasn't correctly updated for the old case of a single exemplar for a classic bucket (if actually parsed as a classic bucket) and a single exemplar on a counter. In those cases, the method would return `true` forever, yielding the same exemplar again and again, leading to an endless loop. With this fix, the state is now tracked and the single exemplar is only returned once. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	3d9a830f2f	textparse: Expose #12731 in protobufparse_test.go Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
György Krajcsovits	e846736134	Fix typo Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	1 year ago
beorn7	c58e20ad0e	histogram: Identify native histograms even without observations Native histograms without observations and with a zero threshold of zero look the same as classic histograms in the protobuf exposition format. According to https://github.com/prometheus/client_golang/issues/1127 , the idea is to add a no-op span to those histograms to mark them as native histograms. This commit enables Prometheus to detect that no-op span and adds a doc comment to the proto spec describing the behavior. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
Julien Pivotto	7905594b52	Merge pull request #12557 from prometheus/beorn7/histogram scrape: Enable ingestion of multiple exemplars per sample	1 year ago
beorn7	071f4bbea4	histograms: Fix parsing float histograms without zero bucket If a float histogram has a zero bucket with a threshold of zero _and_ an empty zero bucket, it wasn't identified as a native histogram because the `isNativeHistogram` helper function only looked at integer buckets. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	2ea8df4734	histogram: Expose #12305 Native histograms without a zero threshold aren't federated properly. This adds a test to prove the specific failure mode, which is that histograms with a zero threshold of zero are federated as classic histograms. The underlying reason is that the protobuf parser identifies a native histogram by detecting a zero bucket or by detecting integer buckets. Therefore, a float histogram with a zero threshold of zero and an unpopulated zero bucket falls through the cracks (no integer buckets, no zero bucket). This commit also addse a test case for the latter. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	0e3f35324b	scrape: Enable ingestion of multiple exemplars per sample This has become a requirement for native histograms, as a single histogram sample commonly has many buckets, so that providing many exemplars makes sense. Since OM text doesn't support native histograms yet, the test had to be expanded to also support protobuf test cases. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	da047c6857	histograms: Fix bug #12552 The problem was the following: When trying to parse native histograms and classic histograms in parallel, the parser would first parse the histogram proto messages as a native histogram and then parse the same message again, but now as a classic histogram. Afterwards, it would forget that it was dealing with a metric family that contains native histograms and would parse the rest of the metric family as classic histograms only. The fix is to check again after being done with a classic histogram. Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	1c3bd04bea	histograms: Modify test to expose bug #12552 Signed-off-by: beorn7 <beorn@grafana.com>	1 year ago
beorn7	9e500345f3	textparse/scrape: Add option to scrape both classic and native histograms So far, if a target exposes a histogram with both classic and native buckets, a native-histogram enabled Prometheus would ignore the classic buckets. With the new scrape config option `scrape_classic_histograms` set, both buckets will be ingested, creating all the series of a classic histogram in parallel to the native histogram series. For example, a histogram `foo` would create a native histogram series `foo` and classic series called `foo_sum`, `foo_count`, and `foo_bucket`. This feature can be used in a migration strategy from classic to native histograms, where it is desired to have a transition period during which both native and classic histograms are present. Note that two bugs in classic histogram parsing were found and fixed as a byproduct of testing the new feature: 1. Series created from classic _gauge_ histograms didn't get the _sum/_count/_bucket prefix set. 2. Values of classic _float_ histograms weren't parsed properly. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Matthieu MOREL	bae9a21200	Merge branch 'main' into linter/nilerr Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Matthieu MOREL	fb3eb21230	enable gocritic, unconvert and unused linters Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Jonatan Ivanov	9225d118dc	Add Exemplars support for all time series This change removes restrictions to allow adding exemplars to all time series. It also contains some improvements in test values so that it is easier to track what is tested. The advantage of doing this is having a little less error-prone tests: "yy" is not really descriptive but "counter-test" can give people a better idea about what is tested so it is harder to make mistakes. Closes gh-11982 Signed-off-by: Jonatan Ivanov <jonatan.ivanov@gmail.com>	2 years ago
Damien Grisonnet	0fee615719	model/textparse: improve openmetrics error outputs Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	2 years ago
Damien Grisonnet	e3b5416d05	model/textparse: improve error outputs Parsing errors in the Prometheus HTTP format parser are very hard to investigate since they only approximately indicate what is going wrong in the parser and don't provide any information about the incorrect input. As such it is very hard to tell what is wrong in the format exposed by the application. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	2 years ago
beorn7	3e5ad99c33	textparse: Minor doc comment improvement Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	3f977e89ab	textparse: Add tests for float and gauge histograms Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	c26891043f	textparse: Set CounterResetHint If a (float or integer) histogram is a gauge histogram, set the CounterResetHint accordingly. (The default value is fine for the normal counter histograms.) Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Björn Rabenstein	f07adbd45f	Merge pull request #11814 from prometheus/beorn7/protobuf textparse: Add gauge histogram support	2 years ago
Ganesh Vernekar	c94a41c4b2	Merge pull request #11785 from Fish-pro/erroris Use errors.Is to check for a specific error	2 years ago
beorn7	b5d4a94e9d	textparse: Add gauge histogram support to protobuf parsing With this commit, the parser stops to see a gauge histogram (whether native or conventional) as an unexpected metric type. It ingests it normally, it even sets the `GaugeHistogram` type in the metadata (as it has already done for a conventional gauge histogram scraped using OpenMetrics), but it otherwise treats it as a normal counter-like histogram. Once #11783 is merged, though, it should be very easy to utilize the type information. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	e9d9bb1b08	textparse: Handle unknown metric types in protobuf gracefully So far, the parser hasn't validated that the type is valid in the `Next()` call. Later, in the `Series()` call, however, it assumes that we will only see valid types and therefore panics with `encountered unexpected metric type, this is a bug`. This commit fixes said bug by adding validation to the `Next()` call. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Damien Grisonnet	96b9d8cebb	model/textparse: associate correct token to errors In some cases, the Prometheus HTTP format parser was not returning the right token in the error output which made debugging impossible. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>	2 years ago
Fish-pro	6ed71a229e	Use errors.Is to check for a specific error Signed-off-by: Fish-pro <zechun.chen@daocloud.io>	2 years ago
Bryan Boreham	8d350d9e0c	Update package model/textparse tests for new labels.Labels type We don't want to touch the result labels now we create them differently. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	1f04899ae3	Update package model/textparse for new labels.Labels type Parse metrics using labels.ScratchBuilder, so we reduce assumptions about internals of Labels. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Jesus Vazquez	e934d0f011	Merge 'main' into sparsehistogram Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>	2 years ago
Bryan Boreham	8120af22e2	benchmarks: SetBytes takes bytes per operation Where the code was multiplying bytes by number of operations, this resulted in absurdly high throughput numbers. Also, in `BenchmarkParse()`, don't run the `expfmt` case twice. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
beorn7	a7c519930e	histograms: Add Compact method to the normal integer Histogram And use the new method to call to compact Histograms during parsing. This happens for both `Histogram` and `FloatHistogram`. In this way, if targets decide to optimize the exposition size by merging spans with empty buckets in between, we still get a normalized results. It will also normalize away any valid but weird representations like empty spans, spans with offset zero, and empty buckets at the start or end of a span. The implementation seemed easy at first as it just turns the `compactBuckets` helper into a generic function (which now got its own file). However, the integer Histograms have delta buckets instead of absolute buckets, which had to be treated specially in the generic `compactBuckets` function. To make sure it works, I have added plenty of explicit tests for `Histogram` in addition to the `FloatHistogram` tests. I have also updated the doc comment for the `Compact` method. Based on the insights now expressed in the doc comment, compacting with a maxEmptyBuckets > 0 is rarely useful. Therefore, this commit also sets the value to 0 in the two cases we were using 3 so far. We might still want to reconsider, so I don't want to remove the maxEmptyBuckets parameter right now. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Julien Pivotto	96d5a32659	Update go to 1.19, set min version to 1.18 (#11279 ) * Update go to 1.19, set min version to 1.18 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> * Update golangci-lint Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu> Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Marc Tudurí	f7df3b86ba	histograms: parse float histograms from proto definition (#11149 ) * histograms: parse float histograms from proto definition Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Improve comment Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Ignore float buckets Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Refactor Histogram() function Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Fix test_float_histogram Signed-off-by: Marc Tuduri <marctc@protonmail.com> * Update model/textparse/protobufparse.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update protobufparse.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape.go Signed-off-by: Marc Tudurí <marctc@protonmail.com> * Update scrape/scrape.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Marc Tuduri <marctc@protonmail.com> Signed-off-by: Marc Tudurí <marctc@protonmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	2 years ago
beorn7	89fb7117c3	textparse: Remove TODO that is actually done Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	c40b105efd	histograms: Move to new exposition protobuf format This is an incompatible protobuf change. Instrumented targets must include https://github.com/prometheus/client_golang/pull/1092 to make this work. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	28f028e938	Merge branch 'main' into sparsehistogram	2 years ago
Matthieu MOREL	c2b4de3611	refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10747 ) Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com> Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	2 years ago
beorn7	3bc711e333	Merge branch 'main' into sparsehistogram	3 years ago
Matthieu MOREL	e2ede285a2	refactor: move from io/ioutil to io and os packages (#10528 ) * refactor: move from io/ioutil to io and os packages * use fs.DirEntry instead of os.FileInfo after os.ReadDir Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>	3 years ago
beorn7	7ee1836ef5	Merge branch 'main' into sparsehistogram	3 years ago
Peter Štibraný	cd18da3605	Fix OpenMetrics parser to sort uppercase labels correctly. (#10510 ) Signed-off-by: Peter Štibraný <pstibrany@gmail.com>	3 years ago
beorn7	4210aac74a	Merge branch 'main' into sparsehistogram	3 years ago

1 2

55 Commits (8065bef172e8d88e22399504b175a8c9115e9da3)