prometheus

Commit Graph

Author	SHA1	Message	Date
Tom Wilkie	c7b3535997	Use pkg/relabelling in remote write. - Unmarshall external_labels config as labels.Labels, add tests. - Convert some more uses of model.LabelSet to labels.Labels. - Remove old relabel pkg (fixes #3647). - Validate external label names. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Palash Nigam	09208b1a58	queryRange: Add more descriptive error messages (#5229 ) Fixes: https://github.com/prometheus/prometheus/issues/4811 Signed-off-by: Palash Nigam <npalash25@gmail.com>	6 years ago
Callum Styan	6f69e31398	Tail the TSDB WAL for remote_write This change switches the remote_write API to use the TSDB WAL. This should reduce memory usage and prevent sample loss when the remote end point is down. We use the new LiveReader from TSDB to tail WAL segments. Logic for finding the tracking segment is included in this PR. The WAL is tailed once for each remote_write endpoint specified. Reading from the segment is based on a ticker rather than relying on fsnotify write events, which were found to be complicated and unreliable in early prototypes. Enqueuing a sample for sending via remote_write can now block, to provide back pressure. Queues are still required to acheive parallelism and batching. We have updated the queue config based on new defaults for queue capacity and pending samples values - much smaller values are now possible. The remote_write resharding code has been updated to prevent deadlocks, and extra tests have been added for these cases. As part of this change, we attempt to guarantee that samples are not lost; however this initial version doesn't guarantee this across Prometheus restarts or non-retryable errors from the remote end (eg 400s). This changes also includes the following optimisations: - only marshal the proto request once, not once per retry - maintain a single copy of the labels for given series to reduce GC pressure Other minor tweaks: - only reshard if we've also successfully sent recently - add pending samples, latest sent timestamp, WAL events processed metrics Co-authored-by: Chris Marchbanks <csmarchbanks.com> (initial prototype) Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com> (sharding changes) Signed-off-by: Callum Styan <callumstyan@gmail.com>	6 years ago
Hrishikesh Barman	a1f34bec2e	Added CORS Origin flag (#5011 ) Signed-off-by: Hrishikesh Barman <hrishikeshbman@gmail.com>	6 years ago
Matt Layher	302148fd69	*: apply gofmt -s Signed-off-by: Matt Layher <mdlayher@gmail.com>	6 years ago
Callum Styan	5358f76c5c	update remote write path proto so that Labels/Timeseries can't be nil (#4957 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	6 years ago
Simon Pasquier	f678e27eb6	: use latest release of staticcheck (#5057 ) : use latest release of staticcheck It also fixes a couple of things in the code flagged by the additional checks. Signed-off-by: Simon Pasquier <spasquie@redhat.com> Use official release of staticcheck Also run 'go list' before staticcheck to avoid failures when downloading packages. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Tom Wilkie	6e08029b56	Move err to be the last return value from storage.Select. (#5054 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Julius Volz	11a52be1d8	Better rounding for incoming query timestamps (#4941 ) Fixes https://github.com/prometheus/prometheus/issues/4939 Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
mknapphrt	f0e9196dca	Return warnings on a remote read fail (#4832 ) Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>	6 years ago
Ben Kochie	c6399296dc	Fix spelling/typos (#4921 ) * Fix spelling/typos Fix spelling/typos reported by codespell/misspell. * UK -> US spelling changes. Signed-off-by: Ben Kochie <superq@gmail.com>	6 years ago
Fabian Reinartz	a9803e9ecb	Correctly skip mismatching targets Signed-off-by: Fabian Reinartz <freinartz@google.com>	6 years ago
Alex Yu	5dcce32ef8	update promlog to latest version (#4876 ) * update promlog to latest version Signed-off-by: Alex Yu <yu.alex96@gmail.com> * Update api tests, fix main setup Signed-off-by: Alex Yu <yu.alex96@gmail.com> * tidy go.sum Signed-off-by: Alex Yu <yu.alex96@gmail.com> * revendor prometheus/common Signed-off-by: Alex Yu <yu.alex96@gmail.com> * only initialize config; use kingpin for remote_storage_adapter Signed-off-by: Alex Yu <yu.alex96@gmail.com> * actually parse the flags Signed-off-by: Alex Yu <yu.alex96@gmail.com> * clean up imports Signed-off-by: Alex Yu <yu.alex96@gmail.com>	6 years ago
Ganesh Vernekar	ca93fd544b	/api/v1/labels endpoint for getting all label names (#4835 ) * vendor: update tsdb Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * /api/v1/labels endpoint Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * regex matchers for API Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add docs Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Matchers behaving as OR Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Removed the matchers Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * vendor: update tsdb using go mod Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * vendor update: tsdb Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Added LabelNames() to storage.Querier Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Test for api.labelNames Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Simon Pasquier	6fa8de132b	web/v1/api: add tests for admin actions (#4767 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Goutham Veeramachaneni	7acedbce64	web(api): Make query and range api errors match Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Simon Pasquier	a308a186e4	web/api/v1: fix targets endpoint Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Brian Brazil	9c03e11c2c	Hook OpenMetrics parser into scraping. Extend metadata api to support units. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Simon Pasquier	c4a6acfb1e	: move to go 1.11 (#4626 ) : move to go 1.11 Signed-off-by: Simon Pasquier <spasquie@redhat.com> Reduce number of places where we specify the Go version Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Goutham Veeramachaneni	ffb7f829ec	Merge pull request #4730 from prometheus/release-2.4 Release 2.4	6 years ago
Tariq Ibrahim	d371697841	Adding new metric type to track in-flight queries via the remote read API endpoint. (#4699 ) * Adding new metric type to track in-flight queries via the remote read API endpoint. Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com> * fix review comments Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com> * fix comments Signed-off-by: tariqibrahim <tariq.ibrahim@microsoft.com>	6 years ago
Callum Styan	9bca041285	WIP: keep track of samples per query, set a max # of samples (#4513 ) * keep track of samples per query, set a max # of samples that can be in memory at once Signed-off-by: Callum Styan <callumstyan@gmail.com>	6 years ago
Simon Pasquier	3c00eeaf16	web/api/v1: fix optional skip_head for snapshot (#4674 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Krasi Georgiev	47a673c3a0	process scrape loops reloading in parallel (#4526 ) The scrape manage receiver's channel now just saves the target sets and another backgorund runner updates the scrape loops every 5 seconds. This is so that the scrape manager doesn't block the receiving channel when it does the long background reloading of the scrape loops. Active and dropped targets are now saved in each scrape pool instead of the scrape manager. This is mainly to avoid races when getting the targets via the web api. When reloading the scrape loops now happens in parallel to speed up the final disared state and this also speeds up the prometheus's shutting down. Also updated some funcs signatures in the web package for consistency. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Tom Wilkie	4c52400708	Limit concurrent remote reads. (#4656 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	d3a1ff1abf	Reduce memory usage of remote read by reducing pointer usage. (#4655 ) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Goutham Veeramachaneni	3e87c04b83	Logger is nil for API. Fixes #4577 (#4583 ) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Tom Wilkie	457e4bb58e	Limit the number of samples remote read can return. (#4532 ) * Limit the number of samples remote read can return. - Return 413 entity too large. - Limit can be set be a flag. Allow 0 to mean no limit. - Include limit in error message. - Set default limit to 50M (* 16 bytes = 800MB). Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Max Inden	ecf676cf97	web/api: Expose rule health and last error (#4501 ) Expose rule health and last evaluation error on `/api/v1/rules`. Signed-off-by: Max Leonard Inden <IndenML@gmail.com>	6 years ago
Julius Volz	8fbe1b5133	Handle a bunch of unchecked errors (#4461 ) There are many more (mostly finalizers like Close/Stop/etc.), but most of the others seemed like one couldn't do much about them anyway. Signed-off-by: Julius Volz <julius.volz@gmail.com>	6 years ago
Ganesh Vernekar	f1db699dff	Persist alert 'for' state across restarts (#4061 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Max Leonard Inden	71fafad099	api/v1: Coninue work exposing rules and alerts Signed-off-by: Max Leonard Inden <IndenML@gmail.com>	6 years ago
mg03	31f8ca0dfb	api v1 alerts/rules json endpoint Signed-off-by: mg03 <mgeng03@gmail.com>	6 years ago
Tom Wilkie	b1f600343f	Merge pull request #4359 from prometheus/report-errors Log errors encountered when marshalling and writing responses.	6 years ago
Tom Wilkie	02534510ca	Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	901e6d1f82	Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Thomas Jackson	92c6f0c92e	Add offset to selectParams (#4226 ) * Add Start/End to SelectParams * Make remote read use the new selectParams for start/end This commit will continue sending the start/end time of the remote read query as the overarching promql time and the specific range of data that the query is intersted in receiving a response to is now part of the ReadHints (upstream discussion in #4226). * Remove unused vendored code The genproto.sh script was updated, but the code wasn't regenerated. This simply removes the vendored deps that are no longer part of the codegen output. Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>	6 years ago
Tom Wilkie	f83155b11e	Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	ccb2ee607b	Log errors encountered when marshalling and writing responses. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	fcc3f43acd	spelling. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	ae29512444	Extend API tests to cover remote read API. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Fabian Reinartz	057a5ae2b1	Address comments Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Fabian Reinartz	ad4c33c1ff	scrape,api: provide per-target metric metadata This adds a per-target cache of scraped metadata. The metadata is only available for the lifecycle of the attached target. An API endpoint allows to select metadata by metric name and a label selection of targets. Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Brian Brazil	dd6781add2	Optimise PromQL (#3966 ) * Move range logic to 'eval' Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make aggregegate range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * PromQL is statically typed, so don't eval to find the type. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Extend rangewrapper to multiple exprs Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Start making function evaluation ranged Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make instant queries a special case of range queries Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Eliminate evalString Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Evaluate range vector functions one series at a time Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make unary operators range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make binops range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Pass time to range-aware functions. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple _over_time functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce allocs when working with matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add basic benchmark for range evaluation Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse objects for function arguments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Do dropmetricname and allocating output vector only once. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add range-aware support for range vector functions with params Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise holt_winters, cut cpu and allocs by ~25% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make rate&friends range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware. Document calling convention. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make date functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple math functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Convert more functions to be range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Specialcase timestamp() with vector selector arg for range awareness Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove transition code for functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the rest of the engine transition code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove more obselete code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the last uses of the eval* functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove engine finalizers to prevent corruption The finalizers set by matrixSelector were being called just before the value they were retruning to the pool was then being provided to the caller. Thus a concurrent query could corrupt the data that the user has just been returned. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add new benchmark suite for range functinos Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Migrate existing benchmarks to new system Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand promql benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simply test by removing unused range code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * When testing instant queries, check range queries too. To protect against subsequent steps in a range query being affected by the previous steps, add a test that evaluates an instant query that we know works again as a range query with the tiimestamp we care about not being the first step. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse ring for matrix iters. Put query results back in pool. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse buffer when iterating over matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Unary minus should remove metric name Cut down benchmarks for faster runs. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce repetition in benchmark test cases Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Work series by series when doing normal vectorSelectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise benchmark setup, cuts time by 60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Have rangeWrapper use an evalNodeHelper to cache across steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use evalNodeHelper with functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cache dropMetricName within a node evaluation. This saves both the calculations and allocs done by dropMetricName across steps. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse input vectors in rangewrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse the point slices in the matrixes input/output by rangeWrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make benchmark setup faster using AddFast Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simplify benchmark code. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add caching in VectorBinop Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use xor to have one-level resultMetric hash key Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add more benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Call Query.Close in apiv1 This allows point slices allocated for the response data to be reused by later queries, saving allocations. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise histogram_quantile It's now 5-10% faster with 97% less garbage generated for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make the input collection in rangeVector linear rather than quadratic Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_join, 1.8x faster and 11x less memory for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand benchmarks, cleanup comments, simplify numSteps logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Fabian's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Comments from Alin. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address jrv's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove dead code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Simon's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Rename populateIterators, pre-init some sizes Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Handle case where function has non-matrix args first Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Split rangeWrapper out to rangeEval function, improve comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cleanup and make things more consistent Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make EvalNodeHelper public Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Fabian's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Henri DF	2952387ed1	Pass query hints down into remote read query proto (#4122 ) Signed-off-by: Henri DF <henridf@gmail.com>	7 years ago
beorn7	94ff07b81d	Merge branch 'release-2.2' Signed-off-by: beorn7 <beorn@soundcloud.com>	7 years ago
Krasi Georgiev	ddd46de6f4	Races/3994 (#4005 ) Fix race by properly locking access to scrape pools. Use separate mutex for information needed by UI so that UI isn't blocked when targets are being updated.	7 years ago
Brian Brazil	cc39021b2b	Provide custom marshalling for Point Point has a non-standard marshalling, and is also where the vast majority of CPU time is spent so it is worth optimising.	7 years ago
Brian Brazil	299b78a887	Switch to json-iterator for v1 api. This makes queries ~15% faster and cuts cpu time spent on json encoding by ~40%.	7 years ago
Brian Brazil	8ede14b24c	Add unittests for Point json output	7 years ago
Brian Brazil	ecd0a9c6ba	web: Add benchmark for respond()	7 years ago
Simon Pasquier	83325c8d82	web: replace deprecated InstrumentHandler() (#3862 ) * web: replace deprecated InstrumentHandler() This change replaces the deprecated InstrumentHandler function by the equivalent functions from the promhttp package. The following metrics are removed: * http_request_duration_microseconds (Summary). * http_request_size_bytes (Summary). * http_requests_total (Counter). And the following metrics are added instead: * prometheus_http_request_duration_seconds (Histogram). * prometheus_http_response_size_bytes (Histogram). * promhttp_metric_handler_requests_in_flight (Gauge). * promhttp_metric_handler_requests_total (Counter). * Update github.com/prometheus/common/route package * web: refactor using the new prometheus/common/route package	7 years ago
Fabian Reinartz	3e6c890aea	api: add flag to skip head on snapshots	7 years ago
Conor Broderick	99006d3baf	Added dropped targets API to targets endpoint (#3870 )	7 years ago
Conor Broderick	1fd20fc954	Add dropped alertmanagers to alertmanagers API (#3865 )	7 years ago
Bartek Plotka	93a63ac5fd	api: Added v1/status/flags endpoint. (#3864 ) Endpoint URL: /api/v1/status/flags Example Output: ```json { "status": "success", "data": { "alertmanager.notification-queue-capacity": "10000", "alertmanager.timeout": "10s", "completion-bash": "false", "completion-script-bash": "false", "completion-script-zsh": "false", "config.file": "my_cool_prometheus.yaml", "help": "false", "help-long": "false", "help-man": "false", "log.level": "info", "query.lookback-delta": "5m", "query.max-concurrency": "20", "query.timeout": "2m", "storage.tsdb.max-block-duration": "36h", "storage.tsdb.min-block-duration": "2h", "storage.tsdb.no-lockfile": "false", "storage.tsdb.path": "data/", "storage.tsdb.retention": "15d", "version": "false", "web.console.libraries": "console_libraries", "web.console.templates": "consoles", "web.enable-admin-api": "false", "web.enable-lifecycle": "false", "web.external-url": "", "web.listen-address": "0.0.0.0:9090", "web.max-connections": "512", "web.read-timeout": "5m", "web.route-prefix": "/", "web.user-assets": "" } } ``` Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	7 years ago
Fabian Reinartz	7ccd4b39b8	*: implement query params This adds a parameter to the storage selection interface which allows query engine(s) to pass information about the operations surrounding a data selection. This can for example be used by remote storage backends to infer the correct downsampling aggregates that need to be provided.	7 years ago
Krasi Georgiev	b75428ec19	rename package retrieve to scrape no fucnctinal changes just renaming retrieval to scrape	7 years ago
Goutham Veeramachaneni	2d73d2b892	Merge pull request #3570 from Gouthamve/colon-snapshot Make the date returned by snapshot script friendly	7 years ago
Goutham Veeramachaneni	bee6864c14	Make the date returned by snapshot script friendly Fixes #3568 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Ed Schouten	bb724f1bef	Deprecate DeduplicateSeriesSet() in favor of NewMergeSeriesSet(). Federation makes use of dedupedSeriesSet to merge SeriesSets for every query into one output stream. If many match[] arguments are provided, many dedupedSeriesSet objects will get chained. This has the downside of causing a potential O(nk) running time, where n is the number of series and k the number of match[] arguments. In the mean time, the storage package provides a mergeSeriesSet that accomplishes the same with an O(nlog(k)) running time by making use of a binary heap. Let's just get rid of dedupedSeriesSet and change all existing callers to use mergeSeriesSet.	7 years ago
Goutham Veeramachaneni	e0d917e2f5	Merge pull request #3523 from Gouthamve/clean-tomb Add endpoint to cleanup tombstones	7 years ago
Goutham Veeramachaneni	f0599d4dbf	Incorporate review-feedback Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Julius Volz	ab11a457e8	Remove obsolete TODO in API code In https://github.com/prometheus/prometheus/pull/3230/files, contexts were added to the Querier() method instead, and Cortex is fine with that.	7 years ago
Goutham Veeramachaneni	311edc5a38	Merge branch 'master' into clean-tomb Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	d8515b2580	Move Admin APIs to v1 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Brian Brazil	d7b3df5ae1	Fix staticcheck errors	7 years ago
Fabian Reinartz	83cd270ea4	*: adapt to storage interface changes	7 years ago
David Kaltschmidt	af75ce02c1	Review feedback * renamed MakeQueryStats * added stats to query() as well * gofmt	7 years ago
David Kaltschmidt	c93e54d240	Adds execution timer stats to the range query API consumers should be able to get insight into the query run times. The UI currently measures total roundtrip times. This PR allows for more fine grained metrics to be exposed. * adds new timer for total execution time (queue + eval) * expose new timer, queue timer, and eval timer in stats field of the range query response: ```json { "status": "success", "data": { "resultType": "matrix", "result": [], "stats": { "execQueueTimeNs": 4683, "execTotalTimeNs": 2086587, "totalEvalTimeNs": 2077851 } } } ``` * stats field is optional, only set when query parameter `stats` is not empty Try it via ```sh curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true' ``` Review feedback * moved query stats json generation to query_stats.go * use seconds for all query timers * expose all timers available * Changed ExecTotalTime string representation from Exec queue total time to Exec total time	7 years ago
Alexey Miroshkin	8c681f4a6c	Provide POST endpoint for query+query_range (#3322 ) This PR fixes #3072 by providing POST endpoints for `query` and `query_range`. POST request must be made with `Content-Type: application/x-www-form-urlencoded` header.	7 years ago
Tom Wilkie	746752b946	Merge external labels in order.	7 years ago
Tom Wilkie	b22485bef0	Remove spurious test import.	7 years ago
Tom Wilkie	ee011d906d	Port remote read server to 2.0.	7 years ago
Julius Volz	099df0c5f0	Migrate "golang.org/x/net/context" -> "context" (#3333 ) In some places, where ctxhttp or gRPC are concerned, we still need to use the old contexts.	7 years ago
Fabian Reinartz	abf7c975c9	Merge branch 'master' into dev-2.0	7 years ago
beorn7	f20e6a0ae4	Only respond to API requests once the server is ready	7 years ago
Julius Volz	f7e8348a88	Re-add contexts to storage.Storage.Querier() (#3230 ) * Re-add contexts to storage.Storage.Querier() These are needed when replacing the storage by a multi-tenant implementation where the tenant is stored in the context. The 1.x query interfaces already had contexts, but they got lost in 2.x. * Convert promql.Engine to use native contexts	7 years ago
Tom Wilkie	758d64ffd9	s/EncodReadResponse/EncodeReadResponse/	7 years ago
Tom Wilkie	febed48703	Implement remote read server in Prometheus.	7 years ago
Fabian Reinartz	87918f3097	Merge branch 'master' into dev-2.0	7 years ago
Max Leonard Inden	1c96fbb992	Expose current Prometheus config via /status/config This PR adds the `/status/config` endpoint which exposes the currently loaded Prometheus config. This is the same config that is displayed on `/config` in the UI in YAML format. The response payload looks like such: ``` { "status": "success", "data": { "yaml": <CONFIG> } } ```	7 years ago
Fabian Reinartz	ccf9e62972	*: add admin grpc API	8 years ago
Fabian Reinartz	6e804b3497	Merge branch 'master' into dev-2.0	8 years ago
Tom Wilkie	4d9b917d11	Instrument Prometheus with OpenTracing (#2554 ) * Use request.Context() instead of a global map of contexts. * Add some basic opentracing instrumentation on the query path. * Remove tracehandler endpoint.	8 years ago
Fabian Reinartz	73b8ff0ddc	Merge branch 'master' into dev-2.0	8 years ago
David Symonds	04ad889751	Preserve Alertmanager URLs as *url.URL. Render a nicer link in the web UI.	8 years ago
Julius Volz	beeb0b55c0	Merge pull request #2572 from weaveworks/2571-propagate-api-error Add promql.ErrStorage, which the API propagates as a 500.	8 years ago
Tom Wilkie	f0e8a5f37c	Add promql.ErrStorage, which is interpreted by the API as a 500.	8 years ago
Alexey Palazhchenko	17f15d024a	Small fixes. (#2578 ) Fix typos. Simplify with gofmt -s	8 years ago
Fabian Reinartz	bbcf20ba01	web: deduplicate series in federation	8 years ago
Fabian Reinartz	f56644e3ae	api/v1: deduplicate selected series	8 years ago
Fabian Reinartz	c389193b37	Merge branch 'master' into dev-2.0	8 years ago
Bplotka	1823ae8bc4	Fixed int64 overflow for timestamp in v1/api parseDuration and parseTime (#2501 ) * Fixed int64 overflow for timestamp in v1/api parseDuration and parseTime This led to unexpected results on wrong query with "(...)&start=148966367200.372&end=1489667272.372" That query is wrong because of `start > end` but actually internal int64 overflow caused start to be something around MinInt64 (huge negative value) and was passing validation. BTW: Not sure if negative timestamp makes sense even.. But model.Earliest is actually MinInt64, can someone explain me why? Signed-off-by: Bartek Plotka <bwplotka@gmail.com> * Added missing trailing periods on comments. Signed-off-by: Bartek Plotka <bwplotka@gmail.com> * MOved to only `<` and `>`. Removed equal. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Goutham Veeramachaneni	4b0270290b	Fix comments to match convention	8 years ago
Goutham Veeramachaneni	c6b329c55b	Support Custom Timeouts for Queries	8 years ago
Goutham Veeramachaneni	6634984a38	Comments and Typo Fixes	8 years ago
Fabian Reinartz	1d3cdd0d67	Merge branch 'master' into dev-2.0-rebase	8 years ago
Fabian Reinartz	d80a3de235	pkg/textparse: add documentation	8 years ago
Fabian Reinartz	157e698958	web/api: fix min/max timestamps to valid range	8 years ago
Frederic Branczyk	bd92571bdd	web/api: make target and alertmanager api responses consistent	8 years ago
Frederic Branczyk	389c6d0043	web/api: add alertmanager api	8 years ago
beorn7	767c0709b1	Retrieval: Avoid copying Target retreival.Target contains a mutex. It was copied in the Targets() call. This potentially can wreak a lot of havoc. It might even have caused the issues reported as #2266 and #2262 .	8 years ago
Fabian Reinartz	bc20d93f0a	storage: rename iterator value getters to At()	8 years ago
Fabian Reinartz	28f547bcc7	api/v1: fix tests, restore series queries	8 years ago
Fabian Reinartz	f8fc1f5bb2	*: migrate ingestion to new batch Appender	8 years ago
Fabian Reinartz	5817cb5bde	: migrate from model. to promql.* types	8 years ago
Frederic Branczyk	33b583d50e	web/api: add targets endpoint	8 years ago
Fabian Reinartz	30af47535d	Merge remote-tracking branch 'origin/release-1.2'	8 years ago
Anders Daljord Morken	5c025a2f41	query_range: Validate that start comes before end	8 years ago
Fabian Reinartz	8fa18d564a	storage: enhance Querier interface usage This extracts Querier as an instantiateable and closeable object rather than just defining extending methods of the storage interface. This improves composability and allows abstracting query transactions, which can be useful for transaction-level caches, consistent data views, and encapsulating teardown.	8 years ago
Julius Volz	f92532f254	api: Consolidate web API contexts This is based on the common/route changes in https://github.com/prometheus/common/pull/61.	8 years ago
Julius Volz	c187308366	storage: Contextify storage interfaces. This is based on https://github.com/prometheus/prometheus/pull/1997. This adds contexts to the relevant Storage methods and already passes PromQL's new per-query context into the storage's query methods. The immediate motivation supporting multi-tenancy in Frankenstein, but this could also be used by Prometheus's normal local storage to support cancellations and timeouts at some point.	8 years ago
Julius Volz	ed5a0f0abe	promql: Allow per-query contexts. For Weaveworks' Frankenstein, we need to support multitenancy. In Frankenstein, we initially solved this without modifying the promql package at all: we constructed a new promql.Engine for every query and injected a storage implementation into that engine which would be primed to only collect data for a given user. This is problematic to upstream, however. Prometheus assumes that there is only one engine: the query concurrency gate is part of the engine, and the engine contains one central cancellable context to shut down all queries. Also, creating a new engine for every query seems like overkill. Thus, we want to be able to pass per-query contexts into a single engine. This change gets rid of the promql.Engine's built-in base context and allows passing in a per-query context instead. Central cancellation of all queries is still possible by deriving all passed-in contexts from one central one, but this is now the responsibility of the caller. The central query context is now created in main() and passed into the relevant components (web handler / API, rule manager). In a next step, the per-query context would have to be passed to the storage implementation, so that the storage can implement multi-tenancy or other features based on the contextual information.	8 years ago
Anders Daljord Morken	e9885ecb94	Bugfix: Avoid divide-by-zero panic on query_range?step=0	8 years ago
Julius Volz	3bfec97d46	Make the storage interface higher-level. See discussion in https://groups.google.com/forum/#!topic/prometheus-developers/bkuGbVlvQ9g The main idea is that the user of a storage shouldn't have to deal with fingerprints anymore, and should not need to do an individual preload call for each metric. The storage interface needs to be made more high-level to not expose these details. This also makes it easier to reuse the same storage interface for remote storages later, as fewer roundtrips are required and the fingerprint concept doesn't work well across the network. NOTE: this deliberately gets rid of a small optimization in the old query Analyzer, where we dedupe instants and ranges for the same series. This should have a minor impact, as most queries do not have multiple selectors loading the same series (and at the same offset).	8 years ago
Julius Volz	0c1e7a5b00	Support time range in /api/v1/series endpoint. This adds optional "start" and "end" form values that may be used to restrict the time range of returned series. Fixes https://github.com/prometheus/prometheus/issues/1542	9 years ago
Jonathan Boulle	38098f8c95	Add missing license headers Prometheus is Apache 2 licensed, and most source files have the appropriate copyright license header, but some were missing it without apparent reason. Correct that by adding it.	9 years ago
beorn7	836f1db04c	Improve MetricsForLabelMatchers WIP: This needs more tests. It now gets a from and through value, which it may opportunistically use to optimize the retrieval. With possible future range indices, this could be used in a very efficient way. This change merely applies some easy checks, which should nevertheless solve the use case of heavy rule evaluations on servers with a lot of series churn. Idea is the following: - Only archive series that are at least as old as the headChunkTimeout (which was already extremely unlikely to happen). - Then maintain a high watermark for the last archival, i.e. no archived series has a sample more recent than that watermark. - Any query that doesn't reach to a time before that watermark doesn't have to touch the archive index at all. (A production server at Soundcloud with the aforementioned series churn and heavy rule evaluations spends 50% of its CPU time in archive index lookups. Since rule evaluations usually only touch very recent values, most of those lookup should disappear with this change.) - Federation with a very broad label matcher will profit from this, too. As a byproduct, the un-needed MetricForFingerprint method was removed from the Storage interface.	9 years ago
beorn7	a7408bfb47	Unify duration parsing It's actually happening in several places (and for flags, we use the standard Go time.Duration...). This at least reduces all our home-grown parsing to one place (in model).	9 years ago
Julius Volz	1ae23bf5e9	Handle OPTIONS HTTP requests correctly. Fixes https://github.com/prometheus/prometheus/issues/1346	9 years ago
Tobias Schmidt	7a6a0630d1	Merge pull request #1213 from prometheus/fix-wrong-http-status-codes Return HTTP server error codes for execution errors	9 years ago
Tobias Schmidt	bf84faa010	Return HTTP server error codes for execution errors	9 years ago
Tobias Schmidt	50079a85a1	Make time parameter optional in v1 query API If no time paramter is provided, the current server timestamp is used.	9 years ago
Fabian Reinartz	33aab4169c	Anchor regexes in vector matching This commit makes the regex behavior of vector matching consistent with configuration and label_replace() by anchoring it. Fixes #1200	9 years ago
Fabian Reinartz	398bbf906b	Switch to common/route package	9 years ago
Fabian Reinartz	171f50706a	Fix unkeyed field errors.	9 years ago
Fabian Reinartz	f8a25f6af7	Apply HTTP handler compression everywhere	9 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	9 years ago
Fabian Reinartz	1535ef1457	Replace metric.SamplePair with model.SamplePair	9 years ago
Fabian Reinartz	438e232c9b	Fix grouping of import blocks	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
Fabian Reinartz	8f904d6a54	api/v1: fix response format tests	10 years ago
Fabian Reinartz	b36fa7ad61	api/v1: fix Content-Type in response	10 years ago
Julius Volz	bc1c789bab	Disallow cross-origin DELETE and POST requests.	10 years ago
Fabian Reinartz	5b91ea9b36	storage: improve label matching and allow unset matching. Matching of empty labels now also matches metrics where the label was not explicitly set to the empty string.	10 years ago
Fabian Reinartz	94cd321be1	promql: error if all label matchers are empty.	10 years ago
Fabian Reinartz	7bb7e565a4	web/api: add GET and DELETE /series endpoints	10 years ago
Fabian Reinartz	7be94ce962	web/api: improve errors, add tests	10 years ago
Fabian Reinartz	75b0b7420e	web/api: replace /metrics/names with /label/:name/values endpoint.	10 years ago
Fabian Reinartz	ab9c98acac	web/api: add initial API v1 implementation.	10 years ago

... 2 3 4 5 6

291 Commits (ae597cac62425f0d3017860af6be5f65666c9e93)