prometheus

Commit Graph

Author	SHA1	Message	Date
Marco Pracucci	b92c03023d	Optimized vector selector Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Robert Fratto	5b78aa0649	Contribute grafana/agent sigv4 code (#8509 ) * Contribute grafana/agent sigv4 code * address review feedback - move validation logic for RemoteWrite into unmarshal - copy configuration fields from ec2 SD config - remove enabled field, use pointer for enabling sigv4 * Update config/config.go * Don't provide credentials if secret key / access key left blank * Add SigV4 headers to the list of unchangeable headers. * sigv4: don't include all headers in signature * only test for equality in the authorization header, not the signed date * address review feedback 1. s/httpClientConfigEnabled/httpClientConfigAuthEnabled 2. bearer_token tuples to "authorization" 3. Un-export NewSigV4RoundTripper * add x-amz-content-sha256 to list of unchangeable headers * Document sigv4 configuration * add suggestion for using default AWS SDK credentials Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	4 years ago
Julien Pivotto	9c4bc38c94	Merge pull request #8516 from Harkishen-Singh/headers-remote-read-on-round-tripper Custom headers on remote-read and refactor implementation to roundtripper	4 years ago
Tom Wilkie	ce97cdd477	Move remote read handler to remote package. (#8536 ) * Move remote read handler to remote package. This follows the pattern I started with the remote write handler. The api/v1 package is getting pretty cluttered. Moving code to other packages helps reduce this size and also makes it reusable - eg Cortex doesn't do streaming remote writes yet, and will very soon. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Deal with a nil remoteReadHandler for tests. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Remove the global metrics. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Fix test. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Harkishen-Singh	79ba53a6c4	Custom headers on remote-read and refactor implementation to roundtripper. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Tom Wilkie	7369561305	Combine Appender.Add and AddFast into a single Append method. (#8489 ) This moves the label lookup into TSDB, whilst still keeping the cached-ref optimisation for repeated Appends. This makes the API easier to consume and implement. In particular this change is motivated by the scrape-time-aggregation work, which I don't think is possible to implement without it as it needs access to label values. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Harkishen-Singh	77c20fd2f8	Adds support to configure retry on Rate-Limiting from remote-write config. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Harkishen Singh	cd412470d7	Consider status code 429 as recoverable errors to avoid resharding (#8237 ) * Consider status code 429 as recoverable errors to avoid resharding. * Adds support for Retry-After in backoff logic in remote storage. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
fuling	47d7e3781f	[fix] remote_storage : change "write_hander.go" to "write_handler.go" Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	4 years ago
Mauro Stettler	7715fe3219	Add matchers to LabelValues() call (#8400 ) * Accept matchers in querier LabelValues() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * create matcher to only select metrics which have searched label Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * test case for merge querier with matchers Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * test LabelValues with matchers on head Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * add test for LabelValues on block Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * formatting fix Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Add comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * add missing lock release Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unused parameter Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Benchmarks for LabelValues() methods on block/head Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Better comment Co-authored-by: Julien Pivotto <roidelapluie@gmail.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * update comment Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * minor refactor make code cleaner Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * better comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * fix expected errors in test Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Deleting parameter which can only be empty Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * fix comments Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unnecessary lock Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * only lookup label value if label name was looked up Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Return error when there is one Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Call .Get() on decoder before checking errors Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * only lock head.symMtx when necessary Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * remove unnecessary delete() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * re-use code instead of duplicating it Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Consistently return error from LabelValueFor() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * move helper func from util.go to querier.go Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Fix test expectation Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> * ensure result de-duplication and sorting works Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * return named error from LabelValueFor() Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	4 years ago
Tom Wilkie	d479151f1f	Various enhancements and refactorings for remote write receiver: - Remove unrelated changes - Refactor code out of the API module - that is already getting pretty crowded. - Don't track reference for AddFast in remote write. This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series. For now, its easier and safer to always use the 'slow' path. - Return 400 on out of order samples. - Use remote.DecodeWriteRequest in the remote write adapters. - Put this behing the 'remote-write-server' feature flag - Add some (very) basic docs. - Used named return & add test for commit error propagation Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	4 years ago
Julien Pivotto	6368227f0c	Merge pull request #8443 from liguozhong/remote_storage [remote storage] remove sendWriteRequestWithBackoff() "s" and "req" param	4 years ago
Nándor István Krácser	509000269a	remote_write: allow passing along custom HTTP headers (#8416 ) * remote_write: allow passing along custom HTTP headers Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * add warning Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * remote_write: add header valadtion Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * extend tests for bad remote write headers Signed-off-by: Nandor Kracser <bonifaido@gmail.com> * remote_write: add note about the authorization header Signed-off-by: Nandor Kracser <bonifaido@gmail.com>	4 years ago
fuling	4a407210f5	[remote storage] remove sendWriteRequestWithBackoff() "s" and "req" param Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	4 years ago
Chris Marchbanks	275f7e7766	Log recoverable remote write errors as warnings (#8412 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	4 years ago
kevinForMyself	db445844d3	Fix garbage collection about t.droppedSeries in QueueManager.SeriesReset. (#8387 ) * Fix memory leak about t.droppedSeries in QueueManager.SeriesReset. Signed-off-by: kevinForMyself <zise_2001@163.com> * Fix garbage collection about t.droppedSeries in QueueManager.SeriesReset Signed-off-by: kevinForMyself <zise_2001@163.com>	4 years ago
Ben Ye	caa173d2aa	Support matchers for Labels API (#8301 ) Signed-off-by: Ben Ye <yb532204897@gmail.com> Co-authored-by: Erik Klockare <eklockare@gmail.com>	4 years ago
gotjosh	4eca4dffb8	Allow metric metadata to be propagated via Remote Write. (#6815 ) * Introduce a metadata watcher Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage. Signed-off-by: gotjosh <josue@grafana.com> * Additional fixes after rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Rework samples/metadata metrics. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use more descriptive variable names in MetadataWatcher collect. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix issues caused during rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix missing metric add and unneeded config code. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix metrics and docs Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Replace assert with require Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Bring back max_samples_per_send metric Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	3d8826a3d4	MultiError: Refactored MultiError for more concise and safe usage. (#8066 ) * MultiError: Refactored MultiError for more concise and safe usage. * Less lines * Goland IDE was marking every usage of old MultiError "potential nil" error * It was easy to forgot using Err() when error was returned, now it's safely assured on compile time. NOTE: Potentially I would rename package to merrors. (: In different PR. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed review comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix after rebase. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Jorge Luis Betancourt	4dc755cd27	Add a metric for tracking max_samples_per_send (#8102 ) Currently there is no way of tracking the value of the `max_samples_per_send` configuration option, which is commonly tweaked when integrating with a remote write backend. Signed-off-by: Jorge Luis Betancourt Gonzalez <jorge-luis.betancourt@trivago.com>	4 years ago
Julien Pivotto	1282d1b39c	Refactor test assertions (#8110 ) * Refactor test assertions This pull request gets rid of assert.True where possible to use fine-grained assertions. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	8c9850c2bb	Remote: Do not collect non-initialized timestamp metrics (#8060 ) * Remote: Do not collect non-initialized timestamp metrics Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Frederic Branczyk	21a2e8c320	storage/remote/intern_test.go: Fix linting errors Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Siddhant Sinha	d9dcf2c90f	Fixes #7982 . Increase maxErrMsgLen value remote_write api (#8017 ) Signed-off-by: Siddhant Sinha <sid.sinha94@gmail.com>	4 years ago
Harkishen Singh	072b9649a3	Refactor vars to avoid test failures in storage/remote with -count > 1 (#7934 ) * Refactor global vars to avoid failure with run test more than once. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Register highestRecvTimestamp metric. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Use local interner vars. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * Declare interner in write storage. Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Chris Marchbanks	f0f8e50567	Fix missing remote read spans (#7914 ) The remote read client needs to use the nethttp.Transport wrapper in order for spans to be instrumented properly. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	4 years ago
Julien Pivotto	a6ee1f8517	Merge pull request #7913 from prometheus/release-2.21 Merge release 2.21 into master	4 years ago
Bartlomiej Plotka	088fcc9e48	Fixed iterator regression: Avoid using heap for each sample when iterating. (#7900 ) * Fixed iterator regression: Avoid using heap for each sample when iterating. Fixes: https://github.com/prometheus/prometheus/issues/7873 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Check for .At() called after .Next() returned false Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * More comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Bartlomiej Plotka	a399227a9f	Revert "Fixed iterator regression: Avoid using heap for each sample when iterating." This reverts commit `2c8b2c5915`.	4 years ago
Julien Pivotto	2c8b2c5915	Fixed iterator regression: Avoid using heap for each sample when iterating. Fixes: https://github.com/prometheus/prometheus/issues/7873 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Anand Sanmukhani	f4a1b3cef6	Rename `remote.newReadClient()` to `remote.NewReadClient()` (#7881 ) This should make the `NewReadClient()` exported outside the `remote` package Signed-off-by: Anand Sanmukhani <asanmukh@redhat.com>	4 years ago
showuon	dfdc358a5b	Fix the duplicated results issue from /api/v1/series (#7862 ) * Fix the duplicated results issue from /api/v1/series Signed-off-by: Luke Chen <showuon@gmail.com>	4 years ago
Robert Fratto	2bd077ed97	expose UserAgent to make it changeable by importers (#7832 ) Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 years ago
Joe Elliott	624075eafe	Exposes remote storage http client to allow for customization (#7831 ) * Exposes remote write client to allow for customizations to the http client Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed changes to vendored file Signed-off-by: Joe Elliott <number101010@gmail.com>	4 years ago
Andy Bursavich	4e6a94a27d	Invert service discovery dependencies (#7701 ) This also fixes a bug in query_log_file, which now is relative to the config file like all other paths. Signed-off-by: Andy Bursavich <abursavich@gmail.com>	4 years ago
Julien Pivotto	a55c69c4c3	Apply gofmt -s on storage/remote/write_test.go (#7791 ) Noticed in https://goreportcard.com/report/github.com/prometheus/prometheus This is the only file. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Mucahit Kurt	869f1bc587	use remote.Storage in TestSampleDelivery instead of direct creation of QueueManager (#4758 ) Signed-off-by: Mucahit Kurt <mucahitkurt@gmail.com>	4 years ago
johncming	2b75c1b199	storage/remote: rename httpclient name. (#7747 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Guangwen Feng	2c4a4548a8	Fix golint issue caused by incorrect func name (#7756 ) Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	4 years ago
Bartlomiej Plotka	28c5cfaf0d	tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714 ) I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (: ## Changes * Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code. * Moved `fanout test` code from weird separate package to storage. * Unskiped one test: TestFanout_SelectSorted/chunk_querier * Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Julien Pivotto	9848e77678	storage/remote: remove unused code now that tsdb implements ChunkQuerier (#7715 ) * storage/remote: remove unused code now that tsdb implements ChunkQuerier Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * encodeChunks is unused too Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	e6d7cc5fa4	tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069 ) * tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating. Chained to https://github.com/prometheus/prometheus/pull/7059 * NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it. * Added single SeriesEntry / ChunkEntry for all series implementations. * Unified all vertical, and non vertical for compact and querying to single merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before) * Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples. * Refactored endpoint tests and querier tests to include subtests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments from Brian and Beorn. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed snapshot test and added chunk iterator support for DBReadOnly. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed race when iterating over Ats first. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed populate block tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed endpoints test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added test & fixed case of head open chunk. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed DBReadOnly tests and bug producing 1 sample chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added cases for partial block overlap for multiple full chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added extra tests for chunk meta after compaction. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed small vertical merge bug and added more tests for that. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Annanay	ec562f152b	Merge branch 'master' into appender-context Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	9bba8a6eae	Merge branch 'master' into appender-context Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	89129cd39a	Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Javier Palomo Almena	b58a613443	Replace sync/atomic with uber-go/atomic (#7683 ) * storage: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * cmd: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * scripts: Verify that we are not using restricted packages It checks that we are not directly importing 'sync/atomic'. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Reorganise imports in blocks Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier/test: Apply PR suggestions Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * storage/remote: avoid storing references on newEntry Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Revert "scripts: Verify that we are not using restricted packages" This reverts commit `278d32748e`. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Group imports accordingly Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
Joe Elliott	04b028f1e6	Exports recoverable error (#7689 ) Signed-off-by: Joe Elliott <number101010@gmail.com>	4 years ago
Annanay	f40e4579b7	gofmt Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago

1 2 3 4 5 ...

1116 Commits (b92c03023d67c93f1862f875ddf0a39603cf2016)