766 Commits (8d61a3e325d424efaec2a051553f2ff8a9b1430d)

Author	SHA1	Message	Date
Julius Volz	8fda83ea12	Make rules only read local data	8 years ago
Julius Volz	406b65d0dc	Rename remote.Storage to remote.Writer	8 years ago
Julius Volz	02395a224d	[WIP] Remote Read	8 years ago
Fabian Reinartz	b586781283	*: update tsdb vendoring and add retention flag	8 years ago
Goutham Veeramachaneni	f35816613e	Refactored Notifier to use Registerer ... * Brought metrics back into Notifier Notifier still implements a Collector. Check if that is needed.	8 years ago
Fabian Reinartz	9304179ef7	Merge branch 'master' into dev-2.0	8 years ago
Fabian Reinartz	4397b4d508	*: pass Prometheus registry into storage	8 years ago
Julius Volz	beb3c4b389	Remove legacy remote storage implementations ... This removes legacy support for specific remote storage systems in favor of only offering the generic remote write protocol. An example bridge application that translates from the generic protocol to each of those legacy backends is still provided at: documentation/examples/remote_storage/remote_storage_bridge See also https://github.com/prometheus/prometheus/issues/10 The next step in the plan is to re-add support for multiple remote storages.	8 years ago
Fabian Reinartz	ea3ba338dd	main: add flags for new storage	8 years ago
Fabian Reinartz	5772f1a7ba	retrieval/storage: adapt to new interface ... This simplifies the interface to two add methods for appends with labels or faster reference numbers.	8 years ago
Fabian Reinartz	1d3cdd0d67	Merge branch 'master' into dev-2.0-rebase	8 years ago
Fabian Reinartz	035976b275	retrieval: handle not found error correctly	8 years ago
Bartek Plotka	579e33f19a	Fixed style issues.	8 years ago
Bartek Plotka	d7febe97fa	Fixed regression in -alertmanager.url flag. Basic auth was ignored. ... - Included basic auth parsing while parsing to AlertmanagerConfig - Added test case Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Fabian Reinartz	ad9bc62e4c	storage: extend appender and adapt it	8 years ago
Fabian Reinartz	e631a1260d	retrieval: use separate appender per target	8 years ago
Fabian Reinartz	68dc358496	cmd/prometheus: remove tests for old flags	8 years ago
Fabian Reinartz	f8fc1f5bb2	*: migrate ingestion to new batch Appender	8 years ago
Fabian Reinartz	1becee3f6c	main: remove Alertmanager legacy flag configuration	8 years ago
Fabian Reinartz	15a931dbdb	promql: migrate model types, use tsdb interfaces	8 years ago
Fabian Reinartz	8b84ee5ee6	storage: remove old storage ... This removes all old storage files and only keeps interfaces to still allow the code to compile.	8 years ago
Fabian Reinartz	11a731ba82	remote: remove hard-coded remote storages ... This commit removes the flag-configured remote storage integrations in favor of the generic remote write path.	8 years ago
Erdem Agaoglu	054f8ebbfb	Increase default max-connections	8 years ago
Erdem Agaoglu	e487477a17	LimitListener to limit max number of connections ... This also drops tcp keep-alive in ListenAndServe but it's no longer necessary since we now close idle connections long before that.	8 years ago
Erdem Agaoglu	9986b28380	Set read-timeout for http.Server ... This also specifies a timeout for idle client connections, which may cause "too many open files" errors. See #2238	8 years ago
Fabian Reinartz	3fb4d1191b	config: rename AlertingConfig, resolve file paths	8 years ago
Fabian Reinartz	d4deb8bbf2	web: show discovered Alertmanagers in UI	8 years ago
Fabian Reinartz	f210d96497	notifier: use dynamic service discovery	8 years ago
Fabian Reinartz	200bbe1bad	config: extract SD and HTTPClient configurations	8 years ago
beorn7	5c41ca84e5	Catch negative staleness delta set on the command line	8 years ago
Brian Brazil	6bc29ba857	Fix regression from #1957, specify non-zero default timeout. (#2121) ... Fixes #2075	8 years ago
Julius Volz	ab80ced756	storage: separate chunk package, publish more names ... This is a followup to https://github.com/prometheus/prometheus/pull/2011. This publishes more of the methods and other names of the chunk code and moves the chunk code to its own package. There's some unavoidable ugliness: the chunk and chunkDesc metrics are used by both packages, so I had to move them to the chunk package. That isn't great, but I don't see how to do it better without a larger redesign of everything. Same for the evict requests and some other types.	8 years ago
Fabian Reinartz	57b358b82a	vendor: update govalidator (#2023) ... Fixes #2022	8 years ago
Matt Bostock	dd98766b32	cmd/prometheus/main.go: Fix typo in comment	8 years ago
Tom Wilkie	4520e12440	Add HTTP Basic Auth & TLS support to the generic write path. (#1957) ... * Add config, HTTP Basic Auth and TLS support to the generic write path. - Move generic write path configuration to the config file - Factor out config.TLSConfig -> tlf.Config translation - Support TLSConfig for generic remote storage - Rename Run to Start, and make it non-blocking. - Dedupe code in httputil for TLS config. - Make remote queue metrics global.	8 years ago
Julius Volz	c187308366	storage: Contextify storage interfaces. ... This is based on https://github.com/prometheus/prometheus/pull/1997. This adds contexts to the relevant Storage methods and already passes PromQL's new per-query context into the storage's query methods. The immediate motivation supporting multi-tenancy in Frankenstein, but this could also be used by Prometheus's normal local storage to support cancellations and timeouts at some point.	8 years ago
Julius Volz	ed5a0f0abe	promql: Allow per-query contexts. ... For Weaveworks' Frankenstein, we need to support multitenancy. In Frankenstein, we initially solved this without modifying the promql package at all: we constructed a new promql.Engine for every query and injected a storage implementation into that engine which would be primed to only collect data for a given user. This is problematic to upstream, however. Prometheus assumes that there is only one engine: the query concurrency gate is part of the engine, and the engine contains one central cancellable context to shut down all queries. Also, creating a new engine for every query seems like overkill. Thus, we want to be able to pass per-query contexts into a single engine. This change gets rid of the promql.Engine's built-in base context and allows passing in a per-query context instead. Central cancellation of all queries is still possible by deriving all passed-in contexts from one central one, but this is now the responsibility of the caller. The central query context is now created in main() and passed into the relevant components (web handler / API, rule manager). In a next step, the per-query context would have to be passed to the storage implementation, so that the storage can implement multi-tenancy or other features based on the contextual information.	8 years ago
Julius Volz	5f5a78e807	Merge pull request #1974 from prometheus/disable-local-storage ... Allow disabling local storage.	8 years ago
Tom Wilkie	d83879210c	Switch back to protos over HTTP, instead of GRPC. ... My aim is to support the new grpc generic write path in Frankenstein. On the surface this seems easy - however I've hit a number of problems that make me think it might be better to not use grpc just yet. The explanation of the problems requires a little background. At weave, traffic to frankenstein need to go through a couple of services first, for SSL and to be authenticated. So traffic goes: internet -> frontend -> authfe -> frankenstein - The frontend is Nginx, and adds/removes SSL. Its done this way for legacy reasons, so the certs can be managed in one place, although eventually we imagine we'll merge it with authfe. All traffic from frontend is sent to authfe. - Authfe checks the auth tokens / cookie etc and then picks the service to forward the RPC to. - Frankenstein accepts the reads and does the right thing with them. First problem I hit was Nginx won't proxy http2 requests - it can accept them, but all calls downstream are http1 (see https://trac.nginx.org/nginx/ticket/923). This wasn't such a big deal, so it now looks like: internet --(grpc/http2)--> frontend --(grpc/http1)--> authfe --(grpc/http1)--> frankenstein Next problem was golang grpc server won't accept http1 requests (see https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms). It is possible to link a grpc server in with a normal go http mux, as long as the mux server is serving over SSL, as the golang http client & server won't do http2 over anything other than an SSL connection. This would require making all our service to service comms SSL. So I had a go a writing a grpc http1 server, and got pretty far. But is was a bit of a mess. So finally I thought I'd make a separate grpc frontend for this, running in parallel with the frontend/authfe combo on a different port - and first up I'd need a grpc reverse proxy. Ideally we'd have some nice, generic reverse proxy that only knew about a map from service names -> downstream service, and didn't need to decode & re-encode every request as it went through. It seems like this can't be done with golang's grpc library - see https://github.com/mwitkow/grpc-proxy/issues/1. And then I was surprised to find you can't do grpc from browsers! See http://www.grpc.io/faq/ - not important to us, but I'm starting to question why we decided to use grpc in the first place? It would seem we could have most of the benefits of grpc with protos over HTTP, and this wouldn't preclude moving to grpc when its a bit more mature? In fact, the grcp FAQ even admits as much: > Why is gRPC better than any binary blob over HTTP/2? > This is largely what gRPC is on the wire.	8 years ago
Tobias Schmidt	29ced0090f	Fix common english misspellings	8 years ago
Julius Volz	b24e5d63bc	Add noop local storage engine. ... This adds a flag -storage.local.engine which allows turning off local storage in Prometheus. Instead of adding if-conditions and nil checks to all parts of Prometheus that deal with Prometheus's local storage (including the web interface), disabling local storage simply means replacing the normal local storage with a noop version that throws samples away and returns empty query results. We also don't add the noop storage to the fanout appender to decrease internal overhead. Instead of returning empty results, an alternate behavior could be to return errors on any query that point out that the local storage is disabled. Not sure which one is more preferable, so I went with the empty result option for now.	8 years ago
Julius Volz	a88e950d1f	Mark remote write address flag as experimental.	8 years ago
Julius Volz	aa3f2b7216	Generic write cleanups and changes. ... - fold metric name into labels - return initialization errors back to main - add snappy compression - better context handling - pre-allocation of labels - remove generic naming - other cleanups	8 years ago
Brian Brazil	36d2c4bd0b	Add generic write path using grpc. ... This uses a new proto format, with scope for multiple samples per timeseries in future. This will allow users to pump samples out to whatever they like without having to change the core Prometheus code. There's also an example receiver to save users figuring out the boilerplate themselves.	8 years ago
Julius Volz	4a866c13be	Fix ApplyConfig() error handling ... Currently, Prometheus starts up without any error when there is an invalid rule file :-/	8 years ago
Julius Volz	08891beb5f	Merge pull request #1828 from drawks/iss-1821 ... Error on non-flag commandline arguments	8 years ago
Björn Rabenstein	12709af249	Merge pull request #1838 from prometheus/release-1.0 ... Explicitly add logging flags to our custom flag set	8 years ago
Dave Rawks	00ea36cdbe	Error on non-flag commandline arguments ... - Added minor cmdline parsing logic change to bail on unconsumed arguments. Fixes #1821	8 years ago
beorn7	bf6201483c	Improve wording on log flag comment	8 years ago
beorn7	25385aafcb	Explicitly add logging flags to our custom flag set ... In https://github.com/prometheus/prometheus/pull/1782 , we moved to a custom flag set to avoid getting test flags into the main prometheus binary. However, that removed the logging flags, too. This commit updates the vendoring to a version of the log package that allows adding the log flags to our flag set explicitly.	8 years ago
Dmitry Vorobev	273e457da4	web: return status code and error message for config resource	8 years ago
Fabian Reinartz	59d26e8536	web: add -web.route-prefix flag ... Fixes #1191	9 years ago
Fabian Reinartz	8c24dfdb86	cmd/prometheus: use own flag set ... Fixes #1743	9 years ago
Fabian Reinartz	dd57e7ef5c	Merge pull request #1699 from prometheus/fabxc-multiam ... notifier: dispatch to multiple Alertmanagers	9 years ago
Fabian Reinartz	9baf120cd5	notifier: dispatch to multiple Alertmanagers ... This commit extends the notifier to dispatch alert batches to multiple Alertmanagers concurrently. It changes the `-alertmanager.url` flag to accept a comma separated list of URLs and/or to be set multiple times.	9 years ago
beorn7	99881ded63	Make the number of fingerprint mutexes configurable ... With a lot of series accessed in a short timeframe (by a query, a large scrape, checkpointing, ...), there is actually quite a significant amount of lock contention if something similar is running at the same time. In those cases, the number of locks needs to be increased. On the same front, as our fingerprints don't have a lot of entropy, I introduced some additional shuffling. With the current state, anly changes in the least singificant bits of a FP would matter.	9 years ago
beorn7	da8cb10b43	Partition the status tab into items in a dropdown ... I got feedback from different sources about rules and targets being too heavy in the status tab if their are lots of them. This change also allows for more fine-granular locking.	9 years ago
Steve Durrheimer	399d5c6375	Make version informations consistent between prometheus components	9 years ago
beorn7	865d16f870	Rename Gorilla into varbit	9 years ago
beorn7	8cdced3850	Implement Gorilla-inspired chunk encoding ... This is not a verbatim implementation of the Gorilla encoding. First of all, it could not, even if we wanted, because Prometheus has a different chunking model (constant size, not constant time). Second, this adds a number of changes that improve the encoding in general or at least for the specific use case of Prometheus (and are partially only possible in the context of Prometheus). See comments in the code for details.	9 years ago
Tobias Schmidt	2f151d02eb	Merge pull request #1456 from prometheus/validate-alertmanager-url ... Validate alertmanager URL	9 years ago
Tobias Schmidt	7763bbd993	Validate alertmanager URL	9 years ago
beorn7	b6fdb355d7	Move dump-heads into its own tool	9 years ago
beorn7	f193f2b8ef	Add a command to promtool that dumps metadata of heads.db ... I needed this today for debugging. It can certainly be improved, but it's already quite helpful. I refactored the reading of heads.db files out of persistence, which is an improvement, too. I made minor changes to the cli package to allow outputting via the io.Writer interface.	9 years ago
Fabian Reinartz	bfa8aaa017	Rename notification to notifier	9 years ago
Fabian Reinartz	fce17b41c5	Merge pull request #1408 from prometheus/hostname ... Log argument parse errors	9 years ago
Fabian Reinartz	e62677d7ba	Log argument parse errors ... Fixes #1407	9 years ago
Ignacio Carbajo	6a323b1e6d	Fix minor typo	9 years ago
beorn7	ec08c9a391	Rework the way to communicate backpressure (AKA suspended ingestion) ... This gives up on the idea to communicate throuh the Append() call (by either not returning as it is now or returning an error as suggested/explored elsewhere). Here I have added a Throttled() call, which has the advantage that it can be called before a whole _batch_ of Append()'s. Scrapes will happen completely or not at all. Same for rule group evaluations. That's a highly desired behavior (as discussed elsewhere). The code is even simpler now as the whole ingestion buffer could be removed. Logging of throttled mode has been streamlined and will create at most one message per minute.	9 years ago
Fabian Reinartz	d9f836e5b8	Merge pull request #1340 from prometheus/validate-externa-url ... Validate URL parameters	9 years ago
beorn7	a2cd479058	Fix calculation of chunks to persist after restart ... Since we are not overestimating the number of chunks to persist anymore, this commit also adjusts the default value for -storage.local.memory-chunks. Update of documentation will follow.	9 years ago
Tobias Schmidt	122d73858d	Validate URL parameters	9 years ago
Julius Volz	b150c5768c	Add missing word in comment.	9 years ago
Fabian Reinartz	7e1b39c682	Fix startup/teardown order, add documentation	9 years ago
beorn7	4221c7de5c	Improve handling of series file truncation ... If only very few chunks are to be truncated from a very large series file, the rewrite of the file is a lorge overhead. With this change, a certain ratio of the file has to be dropped to make it happen. While only causing disk overhead at about the same ratio (by default 10%), it will cut down I/O by a lot in above scenario.	9 years ago
Fabian Reinartz	37d80c4b25	Fix premature rule evaluation ... This commit prevents rule evaluation from starting until after the storage is ready.	9 years ago
Richard Hartmann	7da42eee6e	main.go: Remove warning about external_labels	9 years ago
Julius Volz	87d1831f12	Document INFLUXDB_PW env var in username flag ... Fixes https://github.com/prometheus/prometheus/issues/1281	9 years ago
Fabian Reinartz	62075aa037	Reduce noisy no-alertmanager warning	9 years ago
Fabian Reinartz	52e5224f5a	Refactor rules/ package	9 years ago
Fabian Reinartz	2c8a96ecdc	Adjust notification handler flags	9 years ago
Fabian Reinartz	e114ce0ff7	Refactor notification handler	9 years ago
Fabian Reinartz	a542cc8609	Remove -web.use-local-assets	9 years ago
Corentin Chary	a2e4439086	Add support for remote storage on Graphite ... Allows to use graphite over tcp or udp. Metrics labels and values are used to construct a valid Graphite path in a way that will allow us to eventually read them back and reconstruct the metrics. For example, this metric: model.Metric{ model.MetricNameLabel: "test:metric", "testlabel": "test:value", "testlabel2": "test:value", ) Will become: test:metric.testlabel=test:value.testlabel2=test:value escape.go takes care of escaping values to match Graphite character set, it basically uses percent-encoding as a fallback wich will work pretty will in the graphite/grafana world. The remote storage module also has an optional 'prefix' parameter to prefix all metrics with a path (for example, 'prometheus.'). Graphite URLs are simply in the form tcp://host:port or udp://host:port.	9 years ago
Fabian Reinartz	e3b6ec9784	Switch to common/log	9 years ago
Julius Volz	dac26cef71	Rename global "labels" config option to "external_labels".	9 years ago
Julius Volz	24d0d9190e	Make -web.external-url help string more verbose.	9 years ago
Julius Volz	eeb1da36ac	Fix InfluxDB write support to work with InfluxDB 0.9.x. ... Because the InfluxDB client library currently pulls in multiple MBs of unnecessary dependencies, I have modified and cut up the vendored version to only pull in the few pieces that are actually needed. On InfluxDB's side, this dependency issue is tracked in: https://github.com/influxdb/influxdb/issues/3447 Hopefully, it will be resolved soon. If a password is needed for InfluxDB, it may be supplied via the INFLUXDB_PW environment variable.	9 years ago
Julius Volz	af513468eb	Fix some dead code, missing error checks, shadowings. ... I applied https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567 and was greeted with a deluge of warnings, most of which were not applicable or really fixable realistically. These are some of the first ones I decided to fix.	9 years ago
Julius Volz	d73c8a4f0b	Remove notice about 0.14.x config file format change.	9 years ago
Jimmi Dyson	ec04ba38a2	Kubernetes SD config check	9 years ago
Jimmi Dyson	a1574aa2b3	Move TLS options to scrape config ... Fixes #1013, fixes #989	9 years ago
Fabian Reinartz	d839980fcb	Merge pull request #1051 from prometheus/globallabels ... Change global label handling	9 years ago
Fabian Reinartz	8fa719f778	Attach global labels to remote storage samples	9 years ago
Fabian Reinartz	5fed076a76	Attach global labels to outgoing alerts.	9 years ago
Fabian Reinartz	9bbd9264e2	Add global labels to federation	9 years ago
Silas Snider	b2cb637f97	Add instrumentation around configuration reloads. ... This commit enables automation (and alerting) around attempts to update prometheus server configuration automatically.	9 years ago
Julius Volz	995d3b831d	Fix most golint warnings. ... This is with `golint -min_confidence=0.5`. I left several lint warnings untouched because they were either incorrect or I felt it was better not to change them at the moment.	9 years ago
Julius Volz	274e9d6955	Exit when web server encounters a startup error	9 years ago
Fabian Reinartz	18c0f347a3	Fix loop-reloading on shutdown	9 years ago
Jan Berktold	2bf7048dbb	Add reload handler to web	9 years ago
Fabian Reinartz	73f1cc807d	Check token and cert file existence in promtool	9 years ago
Fabian Reinartz	7a67472fc1	Resolve relative paths on configuration loading ... This moves the concern of resolving the files relative to the config file into the configuration loading itself. It also fixes #921 which did not load the cert and token files relatively.	9 years ago
Fabian Reinartz	7e615dcdf0	cmd/promtool: resolve rule files relative to config file	10 years ago
Fabian Reinartz	feb8a03503	rules: load rule files relative to a base dir	10 years ago
Julius Volz	fcff35b43e	Consolidate external reachability flags into one. ... Besides fixing https://github.com/prometheus/prometheus/issues/805 by making the entire externally reachable server URL configurable, this adds tests for the "globalURL" template function and makes it easier to test other such functions in the future. This breaks the `web.Hostname` flag (and introduces `web.external-url`). This flag is likely only used by few users, so I hope that's justifiable. Fixes https://github.com/prometheus/prometheus/issues/805	10 years ago
Fabian Reinartz	b201725d1c	cmd/prometheus: fix remote storage fanout	10 years ago
Julius Volz	8887f1e1a2	Merge pull request #853 from prometheus/fabxc/help ... cmd/prometheus: improve help output	10 years ago
Fabian Reinartz	525070419b	cmd/prometheus: improve help output	10 years ago
Fabian Reinartz	bcc8101d9e	cmd/promtool: fix missing builddate in version info	10 years ago
Fabian Reinartz	23e77450ff	main: cleanup initialization of remote storage.	10 years ago
Fabian Reinartz	ccbc801d19	Merge pull request #816 from prometheus/fabxc/promctl ... Create promtool command	10 years ago
Fabian Reinartz	890c1a7e74	cmd/promtool: add promtool command. ... The promtool command should bundle multiple commands that help in maintaining a running Prometheus server.	10 years ago
Fabian Reinartz	f97db8d4e5	cmd/prometheus: fix version output	10 years ago
Fabian Reinartz	39edc2df7a	version: move version information into separate package. ... Version information is determined at build-time and thus there is no need to pass it down from main. In its own package it can be used from various other packages.	10 years ago
Fabian Reinartz	de66e32a4d	cmd/prometheus: create new main package.	10 years ago