prometheus

Commit Graph

Author	SHA1	Message	Date
Tom Wilkie	d83879210c	Switch back to protos over HTTP, instead of GRPC. My aim is to support the new grpc generic write path in Frankenstein. On the surface this seems easy - however I've hit a number of problems that make me think it might be better to not use grpc just yet. The explanation of the problems requires a little background. At weave, traffic to frankenstein need to go through a couple of services first, for SSL and to be authenticated. So traffic goes: internet -> frontend -> authfe -> frankenstein - The frontend is Nginx, and adds/removes SSL. Its done this way for legacy reasons, so the certs can be managed in one place, although eventually we imagine we'll merge it with authfe. All traffic from frontend is sent to authfe. - Authfe checks the auth tokens / cookie etc and then picks the service to forward the RPC to. - Frankenstein accepts the reads and does the right thing with them. First problem I hit was Nginx won't proxy http2 requests - it can accept them, but all calls downstream are http1 (see https://trac.nginx.org/nginx/ticket/923). This wasn't such a big deal, so it now looks like: internet --(grpc/http2)--> frontend --(grpc/http1)--> authfe --(grpc/http1)--> frankenstein Next problem was golang grpc server won't accept http1 requests (see https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms). It is possible to link a grpc server in with a normal go http mux, as long as the mux server is serving over SSL, as the golang http client & server won't do http2 over anything other than an SSL connection. This would require making all our service to service comms SSL. So I had a go a writing a grpc http1 server, and got pretty far. But is was a bit of a mess. So finally I thought I'd make a separate grpc frontend for this, running in parallel with the frontend/authfe combo on a different port - and first up I'd need a grpc reverse proxy. Ideally we'd have some nice, generic reverse proxy that only knew about a map from service names -> downstream service, and didn't need to decode & re-encode every request as it went through. It seems like this can't be done with golang's grpc library - see https://github.com/mwitkow/grpc-proxy/issues/1. And then I was surprised to find you can't do grpc from browsers! See http://www.grpc.io/faq/ - not important to us, but I'm starting to question why we decided to use grpc in the first place? It would seem we could have most of the benefits of grpc with protos over HTTP, and this wouldn't preclude moving to grpc when its a bit more mature? In fact, the grcp FAQ even admits as much: > Why is gRPC better than any binary blob over HTTP/2? > This is largely what gRPC is on the wire.	8 years ago
Tobias Schmidt	29ced0090f	Fix common english misspellings	8 years ago
Julius Volz	a88e950d1f	Mark remote write address flag as experimental.	8 years ago
Julius Volz	aa3f2b7216	Generic write cleanups and changes. - fold metric name into labels - return initialization errors back to main - add snappy compression - better context handling - pre-allocation of labels - remove generic naming - other cleanups	8 years ago
Brian Brazil	36d2c4bd0b	Add generic write path using grpc. This uses a new proto format, with scope for multiple samples per timeseries in future. This will allow users to pump samples out to whatever they like without having to change the core Prometheus code. There's also an example receiver to save users figuring out the boilerplate themselves.	8 years ago
Julius Volz	4a866c13be	Fix ApplyConfig() error handling Currently, Prometheus starts up without any error when there is an invalid rule file :-/	8 years ago
Julius Volz	08891beb5f	Merge pull request #1828 from drawks/iss-1821 Error on non-flag commandline arguments	8 years ago
Björn Rabenstein	12709af249	Merge pull request #1838 from prometheus/release-1.0 Explicitly add logging flags to our custom flag set	8 years ago
Dave Rawks	00ea36cdbe	Error on non-flag commandline arguments - Added minor cmdline parsing logic change to bail on unconsumed arguments. Fixes #1821	8 years ago
beorn7	bf6201483c	Improve wording on log flag comment	8 years ago
beorn7	25385aafcb	Explicitly add logging flags to our custom flag set In https://github.com/prometheus/prometheus/pull/1782 , we moved to a custom flag set to avoid getting test flags into the main prometheus binary. However, that removed the logging flags, too. This commit updates the vendoring to a version of the log package that allows adding the log flags to our flag set explicitly.	8 years ago
Dmitry Vorobev	273e457da4	web: return status code and error message for config resource	8 years ago
Fabian Reinartz	59d26e8536	web: add -web.route-prefix flag Fixes #1191	9 years ago
Fabian Reinartz	8c24dfdb86	cmd/prometheus: use own flag set Fixes #1743	9 years ago
Fabian Reinartz	dd57e7ef5c	Merge pull request #1699 from prometheus/fabxc-multiam notifier: dispatch to multiple Alertmanagers	9 years ago
Fabian Reinartz	9baf120cd5	notifier: dispatch to multiple Alertmanagers This commit extends the notifier to dispatch alert batches to multiple Alertmanagers concurrently. It changes the `-alertmanager.url` flag to accept a comma separated list of URLs and/or to be set multiple times.	9 years ago
beorn7	99881ded63	Make the number of fingerprint mutexes configurable With a lot of series accessed in a short timeframe (by a query, a large scrape, checkpointing, ...), there is actually quite a significant amount of lock contention if something similar is running at the same time. In those cases, the number of locks needs to be increased. On the same front, as our fingerprints don't have a lot of entropy, I introduced some additional shuffling. With the current state, anly changes in the least singificant bits of a FP would matter.	9 years ago
beorn7	da8cb10b43	Partition the status tab into items in a dropdown I got feedback from different sources about rules and targets being too heavy in the status tab if their are lots of them. This change also allows for more fine-granular locking.	9 years ago
Steve Durrheimer	399d5c6375	Make version informations consistent between prometheus components	9 years ago
beorn7	865d16f870	Rename Gorilla into varbit	9 years ago
beorn7	8cdced3850	Implement Gorilla-inspired chunk encoding This is not a verbatim implementation of the Gorilla encoding. First of all, it could not, even if we wanted, because Prometheus has a different chunking model (constant size, not constant time). Second, this adds a number of changes that improve the encoding in general or at least for the specific use case of Prometheus (and are partially only possible in the context of Prometheus). See comments in the code for details.	9 years ago
Tobias Schmidt	2f151d02eb	Merge pull request #1456 from prometheus/validate-alertmanager-url Validate alertmanager URL	9 years ago
Tobias Schmidt	7763bbd993	Validate alertmanager URL	9 years ago
beorn7	b6fdb355d7	Move dump-heads into its own tool	9 years ago
beorn7	f193f2b8ef	Add a command to promtool that dumps metadata of heads.db I needed this today for debugging. It can certainly be improved, but it's already quite helpful. I refactored the reading of heads.db files out of persistence, which is an improvement, too. I made minor changes to the cli package to allow outputting via the io.Writer interface.	9 years ago
Fabian Reinartz	bfa8aaa017	Rename notification to notifier	9 years ago
Fabian Reinartz	fce17b41c5	Merge pull request #1408 from prometheus/hostname Log argument parse errors	9 years ago
Fabian Reinartz	e62677d7ba	Log argument parse errors Fixes #1407	9 years ago
Ignacio Carbajo	6a323b1e6d	Fix minor typo	9 years ago
beorn7	ec08c9a391	Rework the way to communicate backpressure (AKA suspended ingestion) This gives up on the idea to communicate throuh the Append() call (by either not returning as it is now or returning an error as suggested/explored elsewhere). Here I have added a Throttled() call, which has the advantage that it can be called before a whole _batch_ of Append()'s. Scrapes will happen completely or not at all. Same for rule group evaluations. That's a highly desired behavior (as discussed elsewhere). The code is even simpler now as the whole ingestion buffer could be removed. Logging of throttled mode has been streamlined and will create at most one message per minute.	9 years ago
Fabian Reinartz	d9f836e5b8	Merge pull request #1340 from prometheus/validate-externa-url Validate URL parameters	9 years ago
beorn7	a2cd479058	Fix calculation of chunks to persist after restart Since we are not overestimating the number of chunks to persist anymore, this commit also adjusts the default value for -storage.local.memory-chunks. Update of documentation will follow.	9 years ago
Tobias Schmidt	122d73858d	Validate URL parameters	9 years ago
Julius Volz	b150c5768c	Add missing word in comment.	9 years ago
Fabian Reinartz	7e1b39c682	Fix startup/teardown order, add documentation	9 years ago
beorn7	4221c7de5c	Improve handling of series file truncation If only very few chunks are to be truncated from a very large series file, the rewrite of the file is a lorge overhead. With this change, a certain ratio of the file has to be dropped to make it happen. While only causing disk overhead at about the same ratio (by default 10%), it will cut down I/O by a lot in above scenario.	9 years ago
Fabian Reinartz	37d80c4b25	Fix premature rule evaluation This commit prevents rule evaluation from starting until after the storage is ready.	9 years ago
Richard Hartmann	7da42eee6e	main.go: Remove warning about external_labels	9 years ago
Julius Volz	87d1831f12	Document INFLUXDB_PW env var in username flag Fixes https://github.com/prometheus/prometheus/issues/1281	9 years ago
Fabian Reinartz	62075aa037	Reduce noisy no-alertmanager warning	9 years ago
Fabian Reinartz	52e5224f5a	Refactor rules/ package	9 years ago
Fabian Reinartz	2c8a96ecdc	Adjust notification handler flags	9 years ago
Fabian Reinartz	e114ce0ff7	Refactor notification handler	9 years ago
Fabian Reinartz	a542cc8609	Remove -web.use-local-assets	9 years ago
Corentin Chary	a2e4439086	Add support for remote storage on Graphite Allows to use graphite over tcp or udp. Metrics labels and values are used to construct a valid Graphite path in a way that will allow us to eventually read them back and reconstruct the metrics. For example, this metric: model.Metric{ model.MetricNameLabel: "test:metric", "testlabel": "test:value", "testlabel2": "test:value", ) Will become: test:metric.testlabel=test:value.testlabel2=test:value escape.go takes care of escaping values to match Graphite character set, it basically uses percent-encoding as a fallback wich will work pretty will in the graphite/grafana world. The remote storage module also has an optional 'prefix' parameter to prefix all metrics with a path (for example, 'prometheus.'). Graphite URLs are simply in the form tcp://host:port or udp://host:port.	9 years ago
Fabian Reinartz	e3b6ec9784	Switch to common/log	9 years ago
Julius Volz	dac26cef71	Rename global "labels" config option to "external_labels".	9 years ago
Julius Volz	24d0d9190e	Make -web.external-url help string more verbose.	9 years ago
Julius Volz	eeb1da36ac	Fix InfluxDB write support to work with InfluxDB 0.9.x. Because the InfluxDB client library currently pulls in multiple MBs of unnecessary dependencies, I have modified and cut up the vendored version to only pull in the few pieces that are actually needed. On InfluxDB's side, this dependency issue is tracked in: https://github.com/influxdb/influxdb/issues/3447 Hopefully, it will be resolved soon. If a password is needed for InfluxDB, it may be supplied via the INFLUXDB_PW environment variable.	9 years ago
Julius Volz	af513468eb	Fix some dead code, missing error checks, shadowings. I applied https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567 and was greeted with a deluge of warnings, most of which were not applicable or really fixable realistically. These are some of the first ones I decided to fix.	9 years ago

1 2

77 Commits (4b850970a26f98407b2f949ed49468db221b8da1)