240 Commits (9fe8bcf4be00a7ebd73932df0cf39cc837cbb993)

Author	SHA1	Message	Date
Fabian Reinartz	2d0e3746ac	rules: remove dependency on promql.Engine	7 years ago
Krasi Georgiev	e2f4850fea	Refactor main.go with oklog/pkg/group actors pattern	7 years ago
Thibault Chataigner	fc4406201e	Tsdb StartTime : Use a simplier way to compute StartTime	7 years ago
Julius Volz	099df0c5f0	Migrate "golang.org/x/net/context" -> "context" (#3333) ... In some places, where ctxhttp or gRPC are concerned, we still need to use the old contexts.	7 years ago
Julius Volz	9d43176ab3	Remove unused printVersion variable (#3335) ... Kingpin now automatically does this via --version.	7 years ago
Julius Volz	82c5b98496	Capitalize Prometheus in startup message (#3332) ... Hey, branding :)	7 years ago
Thibault Chataigner	bf4a279a91	Remote storage reads based on oldest timestamp in primary storage (#3129) ... Currently all read queries are simply pushed to remote read clients. This is fine, except for remote storage for wich it unefficient and make query slower even if remote read is unnecessary. So we need instead to compare the oldest timestamp in primary/local storage with the query range lower boundary. If the oldest timestamp is older than the mint parameter, then there is no need for remote read. This is an optionnal behavior per remote read client. Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>	7 years ago
Julius Volz	5f715f5733	Fix typo in flag description (#3302)	7 years ago
Tobias Schmidt	3589f2f1d4	Merge pull request #3285 from jlevesy/use-testutils-in-cmd-subpackage ... Use testutil assertion helpers in cmd package	7 years ago
Julien Levesy	d7b4fa8d78	use testutil assertions in the cmd/prometheus package	7 years ago
Mathieu Pasquet	38afa507bb	Provide better errors messages in commandline ... Instead or only printing the help message, which is not always helpful. For example, when upgrading from prometheus v1, the retention time value format has changed and now only accepts one unit (e.g. "15d") where it previously allowed more complex strings (e.g. "360h0m0s"). This commit provides the error message as an explanation for the parsing failure.	7 years ago
Marc Sluiter	6a633eece1	Added go-conntrack for monitoring http connections (#3241) ... Added metrics for in- and outgoing traffic with go-conntrack.	7 years ago
Fabian Reinartz	2d0b8e8b94	Merge branch 'master' into dev-2.0	7 years ago
Paul Gier	08af129b4d	cmd/prometheus: don't allow quotes at beginning or end of url ... This prevents accidental copy/paste error where a the web.external-url or alertmanager.url params could have an extra set of quotes. See also: https://github.com/prometheus/prometheus/issues/1229	7 years ago
Paul Gier	f79b55d057	cmd/prometheus: remove govalidator for url validation ... The usage of govalidator is redundant with the call to url.Parse for url validation. Removing it has the following benefits: - The explicit error message is displayed instead of just a generic valid/invalid message - Slightly smaller code with one fewer external dependency - Speed improvement by removing duplicate call to url.Parse (inside govalidator.IsURL() - Resolves issue #2717 The only potential drawback of removing govalidator is that certain URLs will be considered valid which were previously invalid. For example: - URLs with hostnames that start and/or end with an underscore (http://_example.com_) - URLs with hostnames that contain some special characters (http://foo&*bar.org) These are valid URIs according to RFC 3986 and valid domain names per RFC 2181, however they are not valid hostnames per RFC 952.	7 years ago
Fabian Reinartz	7b02bfee0a	web: start web handler while TSDB is starting up	7 years ago
Goutham Veeramachaneni	f5aed810f9	logging: Port to common/promlog ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	d21f149745	*: migrate to go-kit/log	7 years ago
Fabian Reinartz	c70379e1c7	Merge branch 'dev-2.0' of github.com:prometheus/prometheus into dev-2.0	7 years ago
Fabian Reinartz	fffe51fb03	Add mutex and block profiling via envvar	7 years ago
Ben Kochie	59aca4138b	Fix staticcheck issues.	7 years ago
Matt Bostock	64973f5c65	cmd/prometheus: Fix capitalisation in log line (#3123) ... Change 'Ready' to 'ready'.	7 years ago
Mark Adams	77c816b309	Fix pprof endpoints when -web.route-prefix or -web.external-url is used (#3054) ... Whenever a route prefix is applied, the router prepends the prefix to the URL path on the request. For most handlers, this is not an issue because the request's path is only used for routing and is not actually needed by the handler itself. However, Prometheus delegates the handling of the /debug/* endpoints to the http.DefaultServeMux which has it's own routing logic that depends on the url.Path. As a result, whenever a prefix is applied, the prefixed URL is passed to the DefaultServeMux which has no awareness of the prefix and returns a 404. This change fixes the issue by creating a new serveDebug handler which routes requests /debug/* requests to appropriate net/http/pprof handler and removing the net/http/pprof import in cmd/prometheus since it is no longer necessary. Fixes #2183.	7 years ago
Callum Styan	8912f81ffe	check if file_sd files exist in checkConfig	7 years ago
Fabian Reinartz	25f3e1c424	Merge branch 'master' into mergemaster	7 years ago
KalivarapuReshma	686050d816	Change -config.file to --config.file in Readme and error message	7 years ago
emluque	ff54c5c11a	2831 Add Healthy and Ready endpoints	7 years ago
Fabian Reinartz	4d3d8ee229	Merge pull request #2850 from tomwilkie/dev-2.0-remote ... Remote APIs for v2	7 years ago
Julius Volz	cc50aa2c6b	main: Consistently end flag descriptions with periods. (#2977)	7 years ago
Tom Wilkie	2dda5775e3	Initial port of remote storage to v2.	7 years ago
Fabian Reinartz	32226e30f5	Guard reload and quit endpoints by flag	7 years ago
Fabian Reinartz	45ac064669	web: disable Amin APIs by default	7 years ago
Fabian Reinartz	ccf9e62972	*: add admin grpc API	7 years ago
Fabian Reinartz	be32afd6df	cmd/prometheus: add back tsdb.no-lockfile flag	8 years ago
Goutham Veeramachaneni	f9202c6511	Move from .yaml to .yml in update rules ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	e3701077c3	Move promtool to kingpin ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	867b8d108f	cmd/prometheus: cleanup	8 years ago
Fabian Reinartz	34ab7a885a	cmd/prometheus: switch to kingpin	8 years ago
Goutham Veeramachaneni	592cb00c2f	Remove version from RuleGroups ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	37e7b69f56	Merge remote-tracking branch 'upstream/dev-2.0' into rulegroups ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	67dc73fd59	Flag changes for 2.0 ... Fixes: prometheus/prometheus#2087 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	d407bd150c	Consolidate the duration params in CLI ... * All CLI params moved to model.Duration Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	6b70a4d850	Incorporate PR feedback ... * Move fingerprint to Hash() * Move away from tsdb.MultiError * 0777 -> 0666 for files * checkOverflow of extra fields Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	6c1617fd13	Simplify usage string ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	507790a357	Rework logging to use explicitly passed logger ... Mostly cleaned up the global logger use. Still some uses in discovery package. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	dc69645e92	Move back to go-yaml ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	8abb91f656	Move CLI commander to cobra ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	1c08743721	Update check-rules to new format. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	cea1e99f78	Add update-rules command to promtool ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	669075c6b9	Merge branch 'master' into dev-2.0	8 years ago
Chris Goller	42de0ae013	Use log.Logger interface for all discovery services	8 years ago
Conor Broderick	6766123f93	Replace regex with Secret type and remarshal config to hide secrets (#2775)	8 years ago
Fabian Reinartz	4c31061251	Merge branch 'master' into dev-2.0	8 years ago
Fabian Reinartz	d289dc55c3	storage: update TSDB	8 years ago
Shashank Varanasi	dea60bb553	Fix malformed uname string (#2727) ... * Fix malformed uname string * Make fix better * Reformat code for simplicity	8 years ago
Fabian Reinartz	06c2b76cd4	Merge branch 'master' into uptsdb	8 years ago
Shashank Varanasi	61235fd851	Print system information (uname) at Prometheus startup (#2709) ... * Print uname on prom startup * Make uname file linux-only * Add missing license headers Add missing license headers * Print OS when uname is not available * Print only OS name when uname not available * Remove extra space, fix cmd/prometheus/main.go license header * Add fix for int8 and uint8 systems * Better formatting for build tags in cmd/prometheus/uname files * Remove newline	8 years ago
Frederic Branczyk	c50a3eccce	prometheus: default max-block-duration to 10% of retention	8 years ago
Michal Witkowski	4177c35eba	Fixup sighup for P2 TSDB init #2699	8 years ago
Fabian Reinartz	9b175d48cb	Add flag to disable TSDB lock file	8 years ago
Fabian Reinartz	73b8ff0ddc	Merge branch 'master' into dev-2.0	8 years ago
Matt Layher	283756c503	Initial commit of 'promtool check-metrics', promlint package (#2605)	8 years ago
Fabian Reinartz	757cba7c31	cmd/prometheus: Undo GOGC adjustment	8 years ago
beorn7	f20b84e816	flags: Improve doc strings for checkpoint flags	8 years ago
Fabian Reinartz	8ffc851147	Merge branch 'master' into dev-2.0	8 years ago
Julius Volz	589061919a	Merge pull request #2465 from Gouthamve/alert-metrics-2429 ... Better Metrics For Alerts	8 years ago
Goutham Veeramachaneni	f27ce34a13	Use Registerer to Register All Metrics ... * Made Metric a Gauge so that it can be registered.	8 years ago
Goutham Veeramachaneni	0d0c9d5440	Move Registerer to Config Struct in Notifier	8 years ago
Björn Rabenstein	29f05680a2	Merge pull request #2528 from prometheus/beorn7/storage2 ... main.go: Set GOGC to 40 by default	8 years ago
Björn Rabenstein	e63d079b59	Merge pull request #2527 from prometheus/beorn7/storage ... storage: Evict chunks and calculate persistence pressure...	8 years ago
Julius Volz	b5b0e00923	Merge pull request #2499 from prometheus/remote-read ... Remote Read	8 years ago
beorn7	434ab2a6a3	storage: Evict chunks and calculate persistence pressure based on target heap size ... This is a fairly easy attempt to dynamically evict chunks based on the heap size. A target heap size has to be set as a command line flage, so that users can essentially say "utilize 4GiB of RAM, and please don't OOM". The -storage.local.max-chunks-to-persist and -storage.local.memory-chunks flags are deprecated by this change. Backwards compatibility is provided by ignoring -storage.local.max-chunks-to-persist and use -storage.local.memory-chunks to set the new -storage.local.target-heap-size to a reasonable (and conservative) value (both with a warning). This also makes the metrics intstrumentation more consistent (in naming and implementation) and cleans up a few quirks in the tests. Answers to anticipated comments: There is a chance that Go 1.9 will allow programs better control over the Go memory management. I don't expect those changes to be in contradiction with the approach here, but I do expect them to complement them and allow them to be more precise and controlled. In any case, once those Go changes are available, this code has to be revisted. One might be tempted to let the user specify an estimated value for the RSS usage, and then internall set a target heap size of a certain fraction of that. (In my experience, 2/3 is a fairly safe bet.) However, investigations have shown that RSS size and its relation to the heap size is really really complicated. It depends on so many factors that I wouldn't even start listing them in a commit description. It depends on many circumstances and not at least on the risk trade-off of each individual user between RAM utilization and probability of OOMing during a RAM usage peak. To not add even more to the confusion, we need to stick to the well-defined number we also use in the targeting here, the sum of the sizes of heap objects.	8 years ago
beorn7	96a303b348	storage: Use staleness delta as head chunk timeout ... Currently, if a series stops to exist, its head chunk will be kept open for an hour. That prevents it from being persisted. Which prevents it from being evicted. Which prevents the series from being archived. Most of the time, once no sample has been added to a series within the staleness limit, we can be pretty confident that this series will not receive samples anymore. The whole chain as described above can be started after 5m instead of 1h. In the relaxed case, this doesn't change a lot as the head chunk timeout is only checked during series maintenance, and usually, a series is only maintained every six hours. However, there is the typical scenario where a large service is deployed, the deoply turns out to be bad, and then it is deployed again within minutes, and quite quickly the number of time series has tripled. That's the point where the Prometheus server is stressed and switches (rightfully) into rushed mode. In that mode, time series are processed as quickly as possible, but all of that is in vein if all of those recently ended time series cannot be persisted yet for another hour. In that scenario, this change will help most, and it's exactly the scenario where help is most desperately needed.	8 years ago
beorn7	04ccf84559	main.go: Set GOGC to 40 by default ... Rationale: The default value for GOGC is 100, i.e. a garbage collected is initialized once as many heap space has been allocated as was in use after the last GC was done. This ratio doesn't make a lot of sense in Prometheus, as typically about 60% of the heap is allocated for long-lived memory chunks (most of which are around for many hours if not days). Thus, short-lived heap objects are accumulated for quite some time until they finally match the large amount of memory used by bulk memory chunks and a gigantic GC cyle is invoked. With GOGC=40, we are essentially reinstating "normal" GC behavior by acknowledging that about 60% of the heap are used for long-term bulk storage. The median Prometheus production server at SoundCloud runs a GC cycle every 90 seconds. With GOGC=40, a GC cycle is run every 35 seconds (which is still not very often). However, the effective RAM usage is now reduced by about 30%. If settings are updated to utilize more RAM, the time between GC cycles goes up again (as the heap size is larger with more long-lived memory chunks, but the frequency of creating short-lived heap objects does not change). On a quite busy large Prometheus server, the timing changed from one GC run every 20s to one GC run every 12s. In the former case (just changing GOGC, leave everything else as it is), the CPU usage increases by about 10% (on a mid-size referenc server from 8.1 to 8.9). If settings are adjusted, the CPU consumptions increases more drastically (from 8 cores to 13 cores on a large reference server), despite GCs happening more rarely, presumably because a 50% larger set of memory chunks is managed now. Having more memory chunks is good in many regards, and most servers are running out of memory long before they run out of CPU cycles, so the tradeoff is overwhelmingly positive in most cases. Power users can still set the GOGC environment variable as usual, as the implementation in this commit honors an explicitly set variable.	8 years ago
Julius Volz	8fda83ea12	Make rules only read local data	8 years ago
Julius Volz	406b65d0dc	Rename remote.Storage to remote.Writer	8 years ago
Julius Volz	02395a224d	[WIP] Remote Read	8 years ago
Fabian Reinartz	b586781283	*: update tsdb vendoring and add retention flag	8 years ago
Goutham Veeramachaneni	f35816613e	Refactored Notifier to use Registerer ... * Brought metrics back into Notifier Notifier still implements a Collector. Check if that is needed.	8 years ago
Fabian Reinartz	9304179ef7	Merge branch 'master' into dev-2.0	8 years ago
Fabian Reinartz	4397b4d508	*: pass Prometheus registry into storage	8 years ago
Julius Volz	beb3c4b389	Remove legacy remote storage implementations ... This removes legacy support for specific remote storage systems in favor of only offering the generic remote write protocol. An example bridge application that translates from the generic protocol to each of those legacy backends is still provided at: documentation/examples/remote_storage/remote_storage_bridge See also https://github.com/prometheus/prometheus/issues/10 The next step in the plan is to re-add support for multiple remote storages.	8 years ago
Fabian Reinartz	ea3ba338dd	main: add flags for new storage	8 years ago
Fabian Reinartz	5772f1a7ba	retrieval/storage: adapt to new interface ... This simplifies the interface to two add methods for appends with labels or faster reference numbers.	8 years ago
Fabian Reinartz	1d3cdd0d67	Merge branch 'master' into dev-2.0-rebase	8 years ago
Fabian Reinartz	035976b275	retrieval: handle not found error correctly	8 years ago
Bartek Plotka	579e33f19a	Fixed style issues.	8 years ago
Bartek Plotka	d7febe97fa	Fixed regression in -alertmanager.url flag. Basic auth was ignored. ... - Included basic auth parsing while parsing to AlertmanagerConfig - Added test case Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Fabian Reinartz	ad9bc62e4c	storage: extend appender and adapt it	8 years ago
Fabian Reinartz	e631a1260d	retrieval: use separate appender per target	8 years ago
Fabian Reinartz	68dc358496	cmd/prometheus: remove tests for old flags	8 years ago
Fabian Reinartz	f8fc1f5bb2	*: migrate ingestion to new batch Appender	8 years ago
Fabian Reinartz	1becee3f6c	main: remove Alertmanager legacy flag configuration	8 years ago
Fabian Reinartz	15a931dbdb	promql: migrate model types, use tsdb interfaces	8 years ago
Fabian Reinartz	8b84ee5ee6	storage: remove old storage ... This removes all old storage files and only keeps interfaces to still allow the code to compile.	8 years ago
Fabian Reinartz	11a731ba82	remote: remove hard-coded remote storages ... This commit removes the flag-configured remote storage integrations in favor of the generic remote write path.	8 years ago
Erdem Agaoglu	054f8ebbfb	Increase default max-connections	8 years ago
Erdem Agaoglu	e487477a17	LimitListener to limit max number of connections ... This also drops tcp keep-alive in ListenAndServe but it's no longer necessary since we now close idle connections long before that.	8 years ago
Erdem Agaoglu	9986b28380	Set read-timeout for http.Server ... This also specifies a timeout for idle client connections, which may cause "too many open files" errors. See #2238	8 years ago
Fabian Reinartz	3fb4d1191b	config: rename AlertingConfig, resolve file paths	8 years ago