71 Commits (de172049abc9b795e226c2153fcfaea48804bbf9)

Author	SHA1	Message	Date
Fabian Reinartz	61bd698143	web: implement federation for new storage	8 years ago
Fabian Reinartz	f8fc1f5bb2	*: migrate ingestion to new batch Appender	8 years ago
Fabian Reinartz	5817cb5bde	*: migrate from model.* to promql.* types	8 years ago
Fabian Reinartz	cef2e04aa3	web: add error counter for federation responses	8 years ago
Fabian Reinartz	9d68e81b32	web: don't return federation errors over HTTP ... We are writing federation responses streaming. So after the first byte we wrote, the status header is fixed. We cannot return an HTTP error for intermediate error but should just abort and log instead.	8 years ago
Fabian Reinartz	8fa18d564a	storage: enhance Querier interface usage ... This extracts Querier as an instantiateable and closeable object rather than just defining extending methods of the storage interface. This improves composability and allows abstracting query transactions, which can be useful for transaction-level caches, consistent data views, and encapsulating teardown.	8 years ago
Julius Volz	c187308366	storage: Contextify storage interfaces. ... This is based on https://github.com/prometheus/prometheus/pull/1997. This adds contexts to the relevant Storage methods and already passes PromQL's new per-query context into the storage's query methods. The immediate motivation supporting multi-tenancy in Frankenstein, but this could also be used by Prometheus's normal local storage to support cancellations and timeouts at some point.	8 years ago
beorn7	39c4915401	federation: Collapse time series of the same name ... This will avoid duplicate MetricFamilies, thereby shrinking the size of the federation payload and also creating legal text format. Also, add unit tests for federation. They were also needed for the previous state of the code, but were missing.	8 years ago
Julius Volz	3bfec97d46	Make the storage interface higher-level. ... See discussion in https://groups.google.com/forum/#!topic/prometheus-developers/bkuGbVlvQ9g The main idea is that the user of a storage shouldn't have to deal with fingerprints anymore, and should not need to do an individual preload call for each metric. The storage interface needs to be made more high-level to not expose these details. This also makes it easier to reuse the same storage interface for remote storages later, as fewer roundtrips are required and the fingerprint concept doesn't work well across the network. NOTE: this deliberately gets rid of a small optimization in the old query Analyzer, where we dedupe instants and ranges for the same series. This should have a minor impact, as most queries do not have multiple selectors loading the same series (and at the same offset).	8 years ago
beorn7	836f1db04c	Improve MetricsForLabelMatchers ... WIP: This needs more tests. It now gets a from and through value, which it may opportunistically use to optimize the retrieval. With possible future range indices, this could be used in a very efficient way. This change merely applies some easy checks, which should nevertheless solve the use case of heavy rule evaluations on servers with a lot of series churn. Idea is the following: - Only archive series that are at least as old as the headChunkTimeout (which was already extremely unlikely to happen). - Then maintain a high watermark for the last archival, i.e. no archived series has a sample more recent than that watermark. - Any query that doesn't reach to a time before that watermark doesn't have to touch the archive index at all. (A production server at Soundcloud with the aforementioned series churn and heavy rule evaluations spends 50% of its CPU time in archive index lookups. Since rule evaluations usually only touch very recent values, most of those lookup should disappear with this change.) - Federation with a very broad label matcher will profit from this, too. As a byproduct, the un-needed MetricForFingerprint method was removed from the Storage interface.	9 years ago
beorn7	1e13f89039	Return SamplePair istead of *SamplePair consistently ... Formalize ZeroSamplePair as return value for non-existing samples. Change LastSamplePairForFingerprint to return a SamplePair (and not a pointer to it), which saves allocations in a potentially extremely frequent call.	9 years ago
Fabian Reinartz	b35e25a082	Don't federate stale samples	9 years ago
Julius Volz	dac26cef71	Rename global "labels" config option to "external_labels".	9 years ago
beorn7	ca63c05b4e	Fix creation of protobuf fields	9 years ago
Fabian Reinartz	9bbd9264e2	Add global labels to federation	9 years ago
Julius Volz	995d3b831d	Fix most golint warnings. ... This is with `golint -min_confidence=0.5`. I left several lint warnings untouched because they were either incorrect or I felt it was better not to change them at the moment.	9 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	9 years ago
Fabian Reinartz	d6d88f8950	Add missing license headers	9 years ago
Fabian Reinartz	47aa0d536c	Use common/expfmt in federation	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
Fabian Reinartz	39a8254963	web: add basic federation support. ... This commit adds a federation handler on /federate. It accepts `match[]` query parameters containing vector selectors. Their intersection determines the in-memory metrics that are returned in the same way as the /metrics endpoint does (modulo sorting).	10 years ago