prometheus

Commit Graph

Author	SHA1	Message	Date
Christian Simon	9781e51f59	Correct spelling of "iterable" (#8713 ) Signed-off-by: Christian Simon <simon@swine.de>	4 years ago
Bryan Boreham	c7a62b95ce	GetRef() now returns the label set (#8641 ) The purpose of GetRef() is to allow Append() to be called without the caller needing to copy the labels. To avoid a race where a series is removed from TSDB between the calls to GetRef() and Append(), we return TSDB's copy of the labels. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	4 years ago
Bryan Boreham	d614ae9ecf	[RFC] Add method to get reference number for TSDB Appender (#8600 ) * Add method to get reference number for TSDB Appender In situations where we need to copy labels before calling Add(), GetRef() allows to check first, then call AddFast() in the case that the series is already known. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add explicit interface for GetRef() method Suggested in code review by @bwplotka Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Rename OptionalGetRef to GetRef Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Simplify return value of GetRef() 0 can be relied on to mean 'no reference' Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	4 years ago
Marco Pracucci	e246670193	Further increase max log line in remote write client (#8616 ) Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Callum Styan	289ba11b79	Add circular in-memory exemplars storage (#6635 ) * Add circular in-memory exemplars storage Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Signed-off-by: Martin Disibio <mdisibio@gmail.com> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com> Co-authored-by: Martin Disibio <mdisibio@gmail.com> * Fix some comments, clean up exemplar metrics struct and exemplar tests. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix exemplar query api null vs empty array issue. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com> Co-authored-by: Martin Disibio <mdisibio@gmail.com>	4 years ago
Julien Pivotto	76750d2a96	Merge pull request #8585 from pracucci/optimize-buffered-iterator Optimized vectorSelectorSingle()	4 years ago
Marco Pracucci	6f050f66c7	Update storage/memoized_iterator.go Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	4 years ago
Julius Volz	cf4250cff3	Fix sample deduplication in chainSampleIterator Fixes https://github.com/prometheus/prometheus/issues/8558 Signed-off-by: Julius Volz <julius.volz@gmail.com>	4 years ago
Marco Pracucci	b92c03023d	Optimized vector selector Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Robert Fratto	5b78aa0649	Contribute grafana/agent sigv4 code (#8509 ) * Contribute grafana/agent sigv4 code * address review feedback - move validation logic for RemoteWrite into unmarshal - copy configuration fields from ec2 SD config - remove enabled field, use pointer for enabling sigv4 * Update config/config.go * Don't provide credentials if secret key / access key left blank * Add SigV4 headers to the list of unchangeable headers. * sigv4: don't include all headers in signature * only test for equality in the authorization header, not the signed date * address review feedback 1. s/httpClientConfigEnabled/httpClientConfigAuthEnabled 2. bearer_token tuples to "authorization" 3. Un-export NewSigV4RoundTripper * add x-amz-content-sha256 to list of unchangeable headers * Document sigv4 configuration * add suggestion for using default AWS SDK credentials Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	4 years ago
Julien Pivotto	9c4bc38c94	Merge pull request #8516 from Harkishen-Singh/headers-remote-read-on-round-tripper Custom headers on remote-read and refactor implementation to roundtripper	4 years ago
Tom Wilkie	ce97cdd477	Move remote read handler to remote package. (#8536 ) * Move remote read handler to remote package. This follows the pattern I started with the remote write handler. The api/v1 package is getting pretty cluttered. Moving code to other packages helps reduce this size and also makes it reusable - eg Cortex doesn't do streaming remote writes yet, and will very soon. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Deal with a nil remoteReadHandler for tests. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Remove the global metrics. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Fix test. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Harkishen-Singh	79ba53a6c4	Custom headers on remote-read and refactor implementation to roundtripper. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Tom Wilkie	7369561305	Combine Appender.Add and AddFast into a single Append method. (#8489 ) This moves the label lookup into TSDB, whilst still keeping the cached-ref optimisation for repeated Appends. This makes the API easier to consume and implement. In particular this change is motivated by the scrape-time-aggregation work, which I don't think is possible to implement without it as it needs access to label values. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Harkishen-Singh	77c20fd2f8	Adds support to configure retry on Rate-Limiting from remote-write config. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Harkishen Singh	cd412470d7	Consider status code 429 as recoverable errors to avoid resharding (#8237 ) * Consider status code 429 as recoverable errors to avoid resharding. * Adds support for Retry-After in backoff logic in remote storage. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
fuling	47d7e3781f	[fix] remote_storage : change "write_hander.go" to "write_handler.go" Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	4 years ago
Mauro Stettler	7715fe3219	Add matchers to LabelValues() call (#8400 ) * Accept matchers in querier LabelValues() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * create matcher to only select metrics which have searched label Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * test case for merge querier with matchers Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * test LabelValues with matchers on head Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * add test for LabelValues on block Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * formatting fix Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Add comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * add missing lock release Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unused parameter Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Benchmarks for LabelValues() methods on block/head Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Better comment Co-authored-by: Julien Pivotto <roidelapluie@gmail.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * update comment Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * minor refactor make code cleaner Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * better comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * fix expected errors in test Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Deleting parameter which can only be empty Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * fix comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unnecessary lock Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * only lookup label value if label name was looked up Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Return error when there is one Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Call .Get() on decoder before checking errors Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * only lock head.symMtx when necessary Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unnecessary delete() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * re-use code instead of duplicating it Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Consistently return error from LabelValueFor() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * move helper func from util.go to querier.go Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Fix test expectation Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> * ensure result de-duplication and sorting works Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * return named error from LabelValueFor() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	4 years ago
Tom Wilkie	d479151f1f	Various enhancements and refactorings for remote write receiver: - Remove unrelated changes - Refactor code out of the API module - that is already getting pretty crowded. - Don't track reference for AddFast in remote write. This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series. For now, its easier and safer to always use the 'slow' path. - Return 400 on out of order samples. - Use remote.DecodeWriteRequest in the remote write adapters. - Put this behing the 'remote-write-server' feature flag - Add some (very) basic docs. - Used named return & add test for commit error propagation Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Julien Pivotto	6368227f0c	Merge pull request #8443 from liguozhong/remote_storage [remote storage] remove sendWriteRequestWithBackoff() "s" and "req" param	4 years ago
Nándor István Krácser	509000269a	remote_write: allow passing along custom HTTP headers (#8416 ) * remote_write: allow passing along custom HTTP headers Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * add warning Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * remote_write: add header valadtion Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * extend tests for bad remote write headers Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * remote_write: add note about the authorization header Signed-off-by: Nandor Kracser <bonifaido@gmail.com>	4 years ago
fuling	4a407210f5	[remote storage] remove sendWriteRequestWithBackoff() "s" and "req" param Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	4 years ago
Chris Marchbanks	275f7e7766	Log recoverable remote write errors as warnings (#8412 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	4 years ago
kevinForMyself	db445844d3	Fix garbage collection about t.droppedSeries in QueueManager.SeriesReset. (#8387 ) * Fix memory leak about t.droppedSeries in QueueManager.SeriesReset. Signed-off-by: kevinForMyself <zise_2001@163.com> * Fix garbage collection about t.droppedSeries in QueueManager.SeriesReset Signed-off-by: kevinForMyself <zise_2001@163.com>	4 years ago
Ben Ye	caa173d2aa	Support matchers for Labels API (#8301 ) Signed-off-by: Ben Ye <yb532204897@gmail.com> Co-authored-by: Erik Klockare <eklockare@gmail.com>	4 years ago
gotjosh	4eca4dffb8	Allow metric metadata to be propagated via Remote Write. (#6815 ) * Introduce a metadata watcher Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage. Signed-off-by: gotjosh <josue@grafana.com> * Additional fixes after rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Rework samples/metadata metrics. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use more descriptive variable names in MetadataWatcher collect. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix issues caused during rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix missing metric add and unneeded config code. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix metrics and docs Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Replace assert with require Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Bring back max_samples_per_send metric Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	3d8826a3d4	MultiError: Refactored MultiError for more concise and safe usage. (#8066 ) * MultiError: Refactored MultiError for more concise and safe usage. * Less lines * Goland IDE was marking every usage of old MultiError "potential nil" error * It was easy to forgot using Err() when error was returned, now it's safely assured on compile time. NOTE: Potentially I would rename package to merrors. (: In different PR. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed review comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix after rebase. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Jorge Luis Betancourt	4dc755cd27	Add a metric for tracking max_samples_per_send (#8102 ) Currently there is no way of tracking the value of the `max_samples_per_send` configuration option, which is commonly tweaked when integrating with a remote write backend. Signed-off-by: Jorge Luis Betancourt Gonzalez <jorge-luis.betancourt@trivago.com>	4 years ago
Julien Pivotto	1282d1b39c	Refactor test assertions (#8110 ) * Refactor test assertions This pull request gets rid of assert.True where possible to use fine-grained assertions. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	8c9850c2bb	Remote: Do not collect non-initialized timestamp metrics (#8060 ) * Remote: Do not collect non-initialized timestamp metrics Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Frederic Branczyk	21a2e8c320	storage/remote/intern_test.go: Fix linting errors Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Siddhant Sinha	d9dcf2c90f	Fixes #7982 . Increase maxErrMsgLen value remote_write api (#8017 ) Signed-off-by: Siddhant Sinha <sid.sinha94@gmail.com>	4 years ago
Harkishen Singh	072b9649a3	Refactor vars to avoid test failures in storage/remote with -count > 1 (#7934 ) * Refactor global vars to avoid failure with run test more than once. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Register highestRecvTimestamp metric. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Use local interner vars. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Declare interner in write storage. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Chris Marchbanks	f0f8e50567	Fix missing remote read spans (#7914 ) The remote read client needs to use the nethttp.Transport wrapper in order for spans to be instrumented properly. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	4 years ago
Julien Pivotto	a6ee1f8517	Merge pull request #7913 from prometheus/release-2.21 Merge release 2.21 into master	4 years ago
Bartlomiej Plotka	088fcc9e48	Fixed iterator regression: Avoid using heap for each sample when iterating. (#7900 ) * Fixed iterator regression: Avoid using heap for each sample when iterating. Fixes: https://github.com/prometheus/prometheus/issues/7873 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Check for .At() called after .Next() returned false Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * More comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Bartlomiej Plotka	a399227a9f	Revert "Fixed iterator regression: Avoid using heap for each sample when iterating." This reverts commit `2c8b2c5915`.	4 years ago
Julien Pivotto	2c8b2c5915	Fixed iterator regression: Avoid using heap for each sample when iterating. Fixes: https://github.com/prometheus/prometheus/issues/7873 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Anand Sanmukhani	f4a1b3cef6	Rename `remote.newReadClient()` to `remote.NewReadClient()` (#7881 ) This should make the `NewReadClient()` exported outside the `remote` package Signed-off-by: Anand Sanmukhani <asanmukh@redhat.com>	4 years ago
showuon	dfdc358a5b	Fix the duplicated results issue from /api/v1/series (#7862 ) * Fix the duplicated results issue from /api/v1/series Signed-off-by: Luke Chen <showuon@gmail.com>	4 years ago
Robert Fratto	2bd077ed97	expose UserAgent to make it changeable by importers (#7832 ) Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 years ago
Joe Elliott	624075eafe	Exposes remote storage http client to allow for customization (#7831 ) * Exposes remote write client to allow for customizations to the http client Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed changes to vendored file Signed-off-by: Joe Elliott <number101010@gmail.com>	4 years ago
Andy Bursavich	4e6a94a27d	Invert service discovery dependencies (#7701 ) This also fixes a bug in query_log_file, which now is relative to the config file like all other paths. Signed-off-by: Andy Bursavich <abursavich@gmail.com>	4 years ago
Julien Pivotto	a55c69c4c3	Apply gofmt -s on storage/remote/write_test.go (#7791 ) Noticed in https://goreportcard.com/report/github.com/prometheus/prometheus This is the only file. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Mucahit Kurt	869f1bc587	use remote.Storage in TestSampleDelivery instead of direct creation of QueueManager (#4758 ) Signed-off-by: Mucahit Kurt <mucahitkurt@gmail.com>	4 years ago
johncming	2b75c1b199	storage/remote: rename httpclient name. (#7747 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Guangwen Feng	2c4a4548a8	Fix golint issue caused by incorrect func name (#7756 ) Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	4 years ago
Bartlomiej Plotka	28c5cfaf0d	tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714 ) I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (: ## Changes * Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code. * Moved `fanout test` code from weird separate package to storage. * Unskiped one test: TestFanout_SelectSorted/chunk_querier * Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago

1 2 3 4 5 ...

1124 Commits (a06dcd77719b49ce98abbc111ce2e30af0cb6690)