prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	b96b89ef8b	Merge pull request #12048 from bboreham/faster-targets Scraping targets are synced by creating the full set, then adding/removing any which have changed. This PR speeds up the process of creating the full set. I added a benchmark for `TargetsFromGroup`; it uses configuration from a typical Kubernetes SD. The crux of the change is to do relabeling inside labels.Builder instead of converting to labels.Labels and back again for every rule. The change is broken into several commits for easier review. This is a breaking change to `scrape.PopulateLabels()`, but `relabel.Process` is left as-is, with a new `relabel.ProcessBuilder` option.	2 years ago
Julien Pivotto	1fd59791e1	Update tests Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Bryan Boreham	5cfe759348	scrape: make TargetsFromGroup work with Builder not []Label Save work converting to `Labels` then to `Builder`. `PopulateLabels()` now takes as Builder as input. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	95fc032a61	scrape: add benchmark for TargetsFromGroup `loadConfiguration` is made more general. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Łukasz Mierzwa	e1b7082008	Show individual scrape pools on /targets page (#11142 ) * Add API endpoints for getting scrape pool names This adds api/v1/scrape_pools endpoint that returns the list of names of all the scrape pools configured. Having it allows to find out what scrape pools are defined without having to list and parse all targets. The second change is adding scrapePool query parameter support in api/v1/targets endpoint, that allows to filter returned targets by only finding ones for passed scrape pool name. Both changes allow to query for a specific scrape pool data, rather than getting all the targets for all possible scrape pools. The problem with api/v1/targets endpoint is that it returns huge amount of data if you configure a lot of scrape pools. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com> * Add a scrape pool selector on /targets page Current targets page lists all possible targets. This works great if you only have a few scrape pools configured, but for systems with a lot of scrape pools and targets this slow things down a lot. Not only does the /targets page load very slowly in such case (waiting for huge API response) but it also take a long time to render, due to huge number of elements. This change adds a dropdown selector so it's possible to select only intersting scrape pool to view. There's also scrapePool query param that will open selected pool automatically. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com> Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	2 years ago
Bryan Boreham	9bc6d7a7db	Update package scrape tests for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	4927e13537	scrape tests: undo EmptyLabels change Needs other code changes otherwise tests fail Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	14780c3b4e	scrape: in tests use labels.FromStrings And a few cases of `EmptyLabels()`. Replacing code which assumes the internal structure of `Labels`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Cosrider	bef6556ca5	delete redundant alias (#11180 ) Signed-off-by: Cosrider <cosrider7@gmail.com> Signed-off-by: Cosrider <cosrider7@gmail.com>	2 years ago
Levi Harrison	d61459d826	`no-default-scrape-port` feature flag (#9523 ) * Add `no-default-scrape-port` flag Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Xiaonan Shen	0c3abdc26d	Keep relabeled scrape interval and timeout on reloads (#10916 ) * Preserve relabeled scrape interval and timeout on reloads Signed-off-by: Xiaonan Shen <s@sxn.dev>	2 years ago
beorn7	c954cd9d1d	Move packages out of deprecated pkg directory This creates a new `model` directory and moves all data-model related packages over there: exemplar labels relabel rulefmt textparse timestamp value All the others are more or less utilities and have been moved to `util`: gate logging modetimevfs pool runtime Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Darshan Chaudhary	a7e554b158	add check service-discovery command (#8970 ) Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
SuperQ	31f4108758	Add scrape_timeout_seconds metric Add a new built-in metric `scrape_timeout_seconds` to allow monitoring of the ratio of scrape duration to the scrape timeout. Hide behind a feature flag to avoid additional cardinality by default. Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
Levi Harrison	70f597b033	Configure Scrape Interval and Timeout Via Relabeling (#8911 ) * Configure scrape interval and timeout with labels Signed-off-by: Levi Harrison <git@leviharrison.dev>	3 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	31700a05df	Improve testutil.ErrorEqual (#6471 ) Also improves TestPopulateLabels: testutil.ErrorEqual just returned a bool without failing the test. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
johncming	4757c69157	scrape: close manager gracefully at end. (#6044 ) Signed-off-by: johncming <johncming@yahoo.com>	5 years ago
Simon Pasquier	c1682adb2f	Bump prometheus/common to v0.3.0 (#5344 ) * Reload certificates from disk automatically This change bumps github.com/prometheus/common to include https://github.com/prometheus/common/pull/173 Signed-off-by: Simon Pasquier <spasquie@redhat.com> * scrape: close idle connections on reload/stop Signed-off-by: Simon Pasquier <spasquie@redhat.com> * use v0.3.0 tag Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388 ) i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
xjewer	0d1a69353e	scrape: Add global jitter for HA server (#5181 ) * scrape: Add global jitter for HA server Covers issue in https://github.com/prometheus/prometheus/pull/4926#issuecomment-449039848 where the HA setup become a problem for targets unable to be scraped simultaneously. The new jitter per server relies on the hostname and external labels which necessarily to be uniq. As before, scrape offset will be calculated with regard the absolute time, so even restart/reload doesn't change scrape time per scrape target + prometheus instance. Use fqdn if possible, otherwise fall back to the hostname. It adds extra random seed to calculate server hash to be distinguish on machines with the same hostname, but different DC. Signed-off-by: Aleksei Semiglazov <xjewer@gmail.com>	6 years ago
Julien Pivotto	04ce817c49	scrape: Rewrite scrape loop options as a struct (#5314 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
Simon Pasquier	12708acd15	scrape: catch errors when creating HTTP clients (#5182 ) * scrape: catch errors when creating HTTP clients This change makes sure that no scrape pool is created with a nil HTTP client. Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Address Tariq's comment Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Address Brian's comment Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Matt Layher	302148fd69	*: apply gofmt -s Signed-off-by: Matt Layher <mdlayher@gmail.com>	6 years ago
Bartek Płotka	62c8337e77	Moved configuration into `relabel` package. (#4955 ) Adapted top dir relabel to use pkg relabel structs. Removal of this in a separate tracked here: https://github.com/prometheus/prometheus/issues/3647 Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	6 years ago
Krasi Georgiev	47a673c3a0	process scrape loops reloading in parallel (#4526 ) The scrape manage receiver's channel now just saves the target sets and another backgorund runner updates the scrape loops every 5 seconds. This is so that the scrape manager doesn't block the receiving channel when it does the long background reloading of the scrape loops. Active and dropped targets are now saved in each scrape pool instead of the scrape manager. This is mainly to avoid races when getting the targets via the web api. When reloading the scrape loops now happens in parallel to speed up the final disared state and this also speeds up the prometheus's shutting down. Also updated some funcs signatures in the web package for consistency. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	9f2f6accba	fix the TestManagerReloadNoChange test (#4267 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Elif T. Kuş	57dcdfb15f	Rewrote tests with testutil for several test files (#4086 ) * promql: Rewrote tests with testutil for functions_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * pkg/relabel: Rewrote tests with testutil for relabel_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * discovery/consul: Rewrote tests with testutil for consul_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * scrape: Rewrote tests with testutil for manager_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com>	7 years ago
Simon Pasquier	2cbba4e948	scrape: fix data races This commit avoids passing the full scrape configuration down to the scrape loop to fix data races when the scrape configuration is being reloaded. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	7 years ago
Krasi Georgiev	b75428ec19	rename package retrieve to scrape no fucnctinal changes just renaming retrieval to scrape	7 years ago

31 Commits (842ab019edd51d09e008ccae5f74169d2061e1b5)