prometheus

Commit Graph

Author	SHA1	Message	Date
Kemal Akkoyun	66dfb951c4	: Consistent Error/Warning handling for SeriesSet iterator: Allowing Async Select (#7251 ) Add errors and Warnings to SeriesSet Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Change Querier interface and refactor accordingly Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor promql/engine to propagate warnings at eval stage Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Make sure all the series from all Selects are pre-advanced Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Separate merge series sets Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Clean Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor merge querier failure handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactored and simplified fanout with improvements from incoming chunk iterator PRs. * Secondary logic is hidden, instead of weird failed series set logic we had. * Fanout is well commented * Fanout closing record all errors * MergeQuerier improved API (clearer) * deferredGenericMergeSeriesSet is not needed as we return no samples anyway for failed series sets (next = false). Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix formatting Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix CI issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Added final tests for error handling. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. * Moved hints in populate to be allocated only when needed. * Used sync.Once in secondary Querier to achieve all-or-nothing partial response logic. * Select after first Next is done will panic. NOTE: in lazySeriesSet in theory we could just panic, I think however we can totally just return error, it will panic in expand anyway. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Utilize errWithWarnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix recently introduced expansion issue Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add tests for secondary querier error handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Implement lazy merge Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add name to test cases Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Reorganize Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Remove redundant warnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix rebase mistake Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Brian Brazil	f9d21f10ec	Only relabelling should apply for scrape_samples_scraped_post_relabelling. (#7342 ) More consistent variable names. Fixes #7298 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	5 years ago
Brian Brazil	c9565f08aa	Pass reference to checkAddError so appendErrors is updated. (#7294 ) This was preventing the warnings from being logged. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	5 years ago
ZouYu	2b7437d60e	Fix some warnings: 'redundant type from array, slice, or map composite literal' (#7109 ) Signed-off-by: ZouYu <zouy.fnst@cn.fujitsu.com>	5 years ago
Callum Styan	c453def8c5	Separate scrape add error checking out into it's own function. (#6930 ) * Separate scrape add error checking out into it's own function. Signed-off-by: Callum Styan <callumstyan@gmail.com> * pass sampleLimitError to checkAddError instead of returning an error Signed-off-by: Callum Styan <callumstyan@gmail.com> * Return bool, error from checkAddError so we can properly handle ErrNotFound for AddFast. This should in theory never happen, but the previous code path handled this case. Adds a test for this, which master passes and the previous commit fails. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address comment changes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Move sampleAdded inside the loop iteration within append, since that's the only block the variable is used in. Signed-off-by: Callum Styan <callumstyan@gmail.com>	5 years ago
Bartlomiej Plotka	c4eefd1b3a	storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response. This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet which rely on sorting. I found during work on https://github.com/prometheus/prometheus/pull/5882 that we do so many repetitions because of this, for not good reason. I think I found a good balance between convenience and readability with just one method. Smaller the interface = better. Also I don't know what TestSelectSorted was testing, but now it's testing sorting. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Julien Pivotto	d6ad5551c9	Scrape: do not put staleness marker when cache is reused (#7011 ) * Scrape: do not put staleness marker when cache is reused Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	8907ba6235	Make TSDB use storage errors This fixes #6992, which was introduced by #6777. There was an intermediate component which translated TSDB errors into storage errors, but that component was deleted and this bug went unnoticed, until we were watching at the Prombench results. Without this, scrape will fail instead of dropping samples or using "Add" when the series have been garbage collected. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	ed623f69e2	tsdb: don't allow ingesting empty labelsets (#6891 ) * tsdb: don't allow ingesting empty labelsets When we ingest an empty labelset in the head, further blocks can not be compacted, with the error: ``` level=error ts=2020-02-27T21:26:58.379Z caller=db.go:659 component=tsdb msg="compaction failed" err="persist head block: write compaction: add series: out-of-order series added with label set \"{}\" / prev: \"{}\"" ``` We should therefore reject those invalid empty labelsets upfront. This can be reproduced with the following: ``` cat << END > prometheus.yml scrape_configs: - job_name: 'prometheus' scrape_interval: 1s basic_auth: username: test password: test metric_relabel_configs: - regex: ".*" action: labeldrop static_configs: - targets: - 127.0.1.1:9090 END ./prometheus --storage.tsdb.min-block-duration=1m ``` And wait a few minutes. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Boqin Qin	0e51cf65e7	scrape_test: fix send-to-closed-channel bugs (#6849 ) Signed-off-by: BurtonQin <bobbqqin@gmail.com>	5 years ago
Bartlomiej Plotka	34426766d8	Unify Iterator interfaces. All point to storage now. This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Boqin Qin	cdbd42393e	scrape: fix goroutine leak in test (#6812 ) * scrape: fix goroutine leak in test Signed-off-by: BurtonQin <bobbqqin@gmail.com>	5 years ago
Julien Pivotto	9c67fce6e0	Scrape: test samples_post_metric_relabeling when metrics are dropped (#6720 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	fafb7940b1	Pass over scrape cache to the next scrape (#6670 ) * Pass over scrape cache to the next scrape Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	46d18112a3	tsdb: error on series with duplicate labels (#6664 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
gotjosh	05842176a6	Make the scrape.metricMetadataStore interface public To test the implementation of our metric metadata API, we need to represent various states of metadata in the scrape metadata store. That is currently not possible as the interface and method to set the store are private. This changes the interface, list and get methods, and the SetMetadaStore function to be public. Incidentally, the scrapeCache implementation needs to be renamed to match the new signature. Signed-off-by: gotjosh <josue@grafana.com>	5 years ago
Geoffrey Beausire	5cb7987314	Fix relabaling collision when using exported label When using both a label and the suffix+label in the relabel config. It's possible that Prometheus remove the suffx+label for no obvious reason. It's due to a collision when merging labels from target and from the sample. Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>	5 years ago
Dustin Hooten	ca60bf298c	React UI: Implement /targets page (#6276 ) * Add LastScrapeDuration to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Add Scrape job name to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Implement the /targets page in react Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Add state query param to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Use state filter in api call Signed-off-by: Dustin Hooten <dhooten@splunk.com> * api feedback Signed-off-by: Dustin Hooten <dhooten@splunk.com> * pr feedback frontend Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Implement and use localstorage hook Signed-off-by: Dustin Hooten <dhooten@splunk.com> * PR feedback Signed-off-by: Dustin Hooten <dhooten@splunk.com>	5 years ago
Alex Dzyoba	1a38075f83	scrape: Move tests to testutil (#6187 ) Part of the fix for #3242. Signed-off-by: Alex Dzyoba <alex@dzyoba.com>	5 years ago
yuxiaobo	47e51c8b2b	Correct spelling mistakes Signed-off-by: yuxiaobo <yuxiaobogo@163.com>	5 years ago
Brian Brazil	e62f30d497	Correctly handle empty labels from alert templates. (#5845 ) Fixes https://github.com/prometheus/common/issues/36 Move logic handling this into the labels package, so all the cases are handled in one place and we're less likely to have this come up again. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	5 years ago
Chris Marchbanks	529ccff07b	Remove all usages of stretchr/testify Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	5 years ago
Chris Marchbanks	0685eb5395	Refactor testutil.NewStorage into a new package This avoids a circular dependency between the testutil and storage packages. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	5 years ago
Brian Brazil	b98e818876	Add scrape_series_added per-scrape metric. (#5546 ) This is an estimate of churn, with series being added to the cache being considered churn. This will have both false positives (e.g. series appearing and disappearing) and false negatives (e.g. series hit sample_limit, but still created in head block), but should be generally useful as-is. Relevant docs live in another repo. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Simon Pasquier	c1682adb2f	Bump prometheus/common to v0.3.0 (#5344 ) * Reload certificates from disk automatically This change bumps github.com/prometheus/common to include https://github.com/prometheus/common/pull/173 Signed-off-by: Simon Pasquier <spasquie@redhat.com> * scrape: close idle connections on reload/stop Signed-off-by: Simon Pasquier <spasquie@redhat.com> * use v0.3.0 tag Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Brian Brazil	f7184978f4	Protect against memory exhaustion when scraping. Now that we're not losing the scrape cache across failed scrape, a scrape that continually failed but had varying series or metadata (e.g. timestamps in metric names, plus hitting smaple_limit) would grow the cache indefinitely. Add some code to catch that, and flush the cache anyway. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	dd3073616c	Don't lose the scrape cache on a failed scrape. This avoids CPU usage increasing when the target comes back. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Julien Pivotto	4397916cb2	Add honor_timestamps (#5304 ) Fixes #5302 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
xjewer	0d1a69353e	scrape: Add global jitter for HA server (#5181 ) * scrape: Add global jitter for HA server Covers issue in https://github.com/prometheus/prometheus/pull/4926#issuecomment-449039848 where the HA setup become a problem for targets unable to be scraped simultaneously. The new jitter per server relies on the hostname and external labels which necessarily to be uniq. As before, scrape offset will be calculated with regard the absolute time, so even restart/reload doesn't change scrape time per scrape target + prometheus instance. Use fqdn if possible, otherwise fall back to the hostname. It adds extra random seed to calculate server hash to be distinguish on machines with the same hostname, but different DC. Signed-off-by: Aleksei Semiglazov <xjewer@gmail.com>	6 years ago
Julien Pivotto	04ce817c49	scrape: Rewrite scrape loop options as a struct (#5314 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
Simon Pasquier	12708acd15	scrape: catch errors when creating HTTP clients (#5182 ) * scrape: catch errors when creating HTTP clients This change makes sure that no scrape pool is created with a nil HTTP client. Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Address Tariq's comment Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Address Brian's comment Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
JoeWrightss	4cb6c202ff	Fix fmt.Errorf error message (#5199 ) Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>	6 years ago
Matt Layher	302148fd69	*: apply gofmt -s Signed-off-by: Matt Layher <mdlayher@gmail.com>	6 years ago
Bartek Płotka	62c8337e77	Moved configuration into `relabel` package. (#4955 ) Adapted top dir relabel to use pkg relabel structs. Removal of this in a separate tracked here: https://github.com/prometheus/prometheus/issues/3647 Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	6 years ago
Brian Brazil	d2f0f54d68	Pass through content-type for non-compressed output. (#4912 ) Fixes #4911 Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Simon Pasquier	ed19373a78	: remove use of golang.org/x/net/context (#4869 ) : remove use of golang.org/x/net/context Signed-off-by: Simon Pasquier <spasquie@redhat.com> scrape: fix TestTargetScrapeScrapeCancel Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Brian Brazil	9c03e11c2c	Hook OpenMetrics parser into scraping. Extend metadata api to support units. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	ffe7efb411	Prepare for multiple text formats Pass content type down to text parser. Add layer of indirection in front of text parser, and rename to avoid future clashes. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Krasi Georgiev	47a673c3a0	process scrape loops reloading in parallel (#4526 ) The scrape manage receiver's channel now just saves the target sets and another backgorund runner updates the scrape loops every 5 seconds. This is so that the scrape manager doesn't block the receiving channel when it does the long background reloading of the scrape loops. Active and dropped targets are now saved in each scrape pool instead of the scrape manager. This is mainly to avoid races when getting the targets via the web api. When reloading the scrape loops now happens in parallel to speed up the final disared state and this also speeds up the prometheus's shutting down. Also updated some funcs signatures in the web package for consistency. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Fabian Reinartz	ad4c33c1ff	scrape,api: provide per-target metric metadata This adds a per-target cache of scraped metadata. The metadata is only available for the lifecycle of the attached target. An API endpoint allows to select metadata by metric name and a label selection of targets. Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Karsten Weiss	d79d573f71	Fix spelling mistakes found by codespell (#4065 ) Signed-off-by: Karsten Weiss <knweiss@gmail.com>	7 years ago
Björn Rabenstein	91e470d733	Merge pull request #4096 from simonpasquier/fix-scrape-races-2.2 Fix scrape races (release-2.2 branch)	7 years ago
Simon Pasquier	2cbba4e948	scrape: fix data races This commit avoids passing the full scrape configuration down to the scrape loop to fix data races when the scrape configuration is being reloaded. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	7 years ago
Simon Pasquier	8b89ab0173	scrape: add test detecting data races Signed-off-by: Simon Pasquier <spasquie@redhat.com>	7 years ago
Mario Trangoni	464e747f1e	fix some comments typos (#4059 )	7 years ago
Krasi Georgiev	675ce533c9	refactored TestScrapeLoopAppend and added a test for empty labels	7 years ago
Krasi Georgiev	404b306fb9	Meta labels sd 3693 (#3805 ) Always keep the discovered labels up to date. add test that DiscoveredLabels are always updated	7 years ago
Krasi Georgiev	b75428ec19	rename package retrieve to scrape no fucnctinal changes just renaming retrieval to scrape	7 years ago

49 Commits (268b4c29e101706cbff0f90f0882bf05608fc56c)