prometheus

Commit Graph

Author	SHA1	Message	Date
Tom Wilkie	d41d91388f	Update for new generic remote storage.	8 years ago
Tom Wilkie	a6931b71e8	Rationalise retrieval metrics so we have the state (success/failed) on both samples and batches, in a consistent fashion. Also, report total queue capacity of all queues, i.e. capacity * shards.	8 years ago
Tom Wilkie	ece12bff93	Shard/parrallelise samples by fingerprint in StorageQueueManager By splitting the single queue into multiple queues and flushing each individual queue in serially (and all queues in parallel), we can guarantee to preserve the order of timestampsin samples sent to downstream systems.	8 years ago
Julius Volz	fe29e87824	Merge pull request #1930 from prometheus/generic-write-grpc Generic write via gRPC	8 years ago
Julius Volz	aa3f2b7216	Generic write cleanups and changes. - fold metric name into labels - return initialization errors back to main - add snappy compression - better context handling - pre-allocation of labels - remove generic naming - other cleanups	8 years ago
Brian Brazil	36d2c4bd0b	Add generic write path using grpc. This uses a new proto format, with scope for multiple samples per timeseries in future. This will allow users to pump samples out to whatever they like without having to change the core Prometheus code. There's also an example receiver to save users figuring out the boilerplate themselves.	8 years ago
Dan Milstein	ec064c96f6	Add field names to table-driven test fixtures	8 years ago
Dan Milstein	ac8788aca6	Convert to table-driven test and inline helper func	8 years ago
Dan Milstein	f50f656a66	Fix double-delta unmarshaling to respect actual min header size Turns out its valid to have an overall chunk which is smaller than the full doubleDeltaHeaderBytes size -- if it has a single sample, it doesn't fill the whole header. Updated unmarshalling check to respect this.	8 years ago
Dan Milstein	b815956341	Catch errors when unmarshalling delta/doubleDelta encoded chunks This is (hopefully) a fix for #1653 Specifically, this makes it so that if the length for the stored delta/doubleDelta is somehow corrupted to be too small, the attempt to unmarshal will return an error. The current (broken) behavior is to return a malformed chunk, which can then lead to a panic when there is an attempt to read header values. The referenced issue proposed creating chunks with a minimum length -- I instead opted to just error on the attempt to unmarshal, since I'm not clear on how it could be safe to proceed when the length is incorrect/unknown. The issue also talked about possibly "quarantining series", but I don't know the surrounding code well enough to understand how to make that happen.	8 years ago
Matt Bostock	e618af5d0b	Storage: Add crash recovery metric 'started_dirty' ...to indicate when crash recovery was invoked during Prometheus startup. Fixes #1918.	8 years ago
Dan Milstein	764ceaa939	Add timeout to test, cap waiting at 1 second	8 years ago
Dan Milstein	007907b410	Fix one of the tests for a remote storage QueueManager Specifically, the TestSpawnNotMoreThanMaxConcurrentSendsGoroutines was failing on a fresh checkout of master. The test had a race condition -- it would only pass if one of the spawned goroutines happened to very quickly pull a set of samples off an internal queue. This patch rewrites the test so that it deterministically waits until all samples have been pulled off that queue. In case of errors, it also now reports on the difference between what it expected and what it found. I verified that, if the code under test is deliberately broken, the test successfully reports on that.	8 years ago
Julius Volz	3bfec97d46	Make the storage interface higher-level. See discussion in https://groups.google.com/forum/#!topic/prometheus-developers/bkuGbVlvQ9g The main idea is that the user of a storage shouldn't have to deal with fingerprints anymore, and should not need to do an individual preload call for each metric. The storage interface needs to be made more high-level to not expose these details. This also makes it easier to reuse the same storage interface for remote storages later, as fewer roundtrips are required and the fingerprint concept doesn't work well across the network. NOTE: this deliberately gets rid of a small optimization in the old query Analyzer, where we dedupe instants and ranges for the same series. This should have a minor impact, as most queries do not have multiple selectors loading the same series (and at the same offset).	8 years ago
beorn7	fc6737b7fb	storage: improve index lookups tl;dr: This is not a fundamental solution to the indexing problem (like tindex is) but it at least avoids utilizing the intersection problem to the greatest possible amount. In more detail: Imagine the following query: nicely:aggregating:rule{job="foo",env="prod"} While it uses a nicely aggregating recording rule (which might have a very low cardinality), Prometheus still intersects the low number of fingerprints for `{__name__="nicely:aggregating:rule"}` with the many thousands of fingerprints matching `{job="foo"}` and with the millions of fingerprints matching `{env="prod"}`. This totally innocuous query is dead slow if the Prometheus server has a lot of time series with the `{env="prod"}` label. Ironically, if you make the query more complicated, it becomes blazingly fast: nicely:aggregating:rule{job=~"foo",env=~"prod"} Why so? Because Prometheus only intersects with non-Equal matchers if there are no Equal matchers. That's good in this case because it retrieves the few fingerprints for `{__name__="nicely:aggregating:rule"}` and then starts right ahead to retrieve the metric for those FPs and checking individually if they match the other matchers. This change is generalizing the idea of when to stop intersecting FPs and go into "retrieve metrics and check them individually against remaining matchers" mode: - First, sort all matchers by "expected cardinality". Matchers matching the empty string are always worst (and never used for intersections). Equal matchers are in general consider best, but by using some crude heuristics, we declare some better than others (instance labels or anything that looks like a recording rule). - Then go through the matchers until we hit a threshold of remaining FPs in the intersection. This threshold is higher if we are already in the non-Equal matcher area as intersection is even more expensive here. - Once the threshold has been reached (or we have run out of matchers that do not match the empty string), start with "retrieve metrics and check them individually against remaining matchers". A beefy server at SoundCloud was spending 67% of its CPU time in index lookups (fingerprintsForLabelPairs), serving mostly a dashboard that is exclusively built with recording rules. With this change, it spends only 35% in fingerprintsForLabelPairs. The CPU usage dropped from 26 cores to 18 cores. The median latency for query_range dropped from 14s to 50ms(!). As expected, higher percentile latency didn't improve that much because the new approach is _occasionally_ running into the worst case while the old one was _systematically_ doing so. The 99th percentile latency is now about as high as the median before (14s) while it was almost twice as high before (26s).	8 years ago
Dmitry Vorobev	273e457da4	web: return status code and error message for config resource	8 years ago
Björn Rabenstein	0622304244	Merge pull request #1798 from prometheus/beorn7/storage2 Crash recovery: Fix an edge case.	8 years ago
beorn7	2a75b15328	Crash recovery: Fix an edge case. If the chunks of a series in the checkpoint are all older then the latest chunk on disk, the head chunk is persisted and therefore has to be declared closed. It would be great to have a test for this, but that would require more plumbing, subject of #447.	9 years ago
beorn7	064b57858e	Consistently use the `Seconds()` method for conversion of durations This also fixes one remaining case of recording integral numbers of seconds only for a metric, i.e. this will probably fix #1796.	9 years ago
Julius Volz	91401794fa	storage: Make MemorySeriesStorage a public type See https://twitter.com/fabxc/status/748032597876482048	9 years ago
Fabian Reinartz	425736a377	*: remove last remainers of non-second metrics	9 years ago
Julius Volz	b7b6717438	Separate query interface out of local.Storage. PromQL only requires a much narrower interface than local.Storage in order to run queries. Narrower interfaces are easier to replace and test, too. We could also change the web interface to use local.Querier, except that we'll probably use appending functions from there in the future.	9 years ago
Jan van Valburg	68f3df49d0	stoarge: fix 'access denied' error on Windows On Windows, it is not possible to rename or delete a file that is currerntly open. This change closes the file in dropAndPersistChunks before it tries to delete it, or rename the temporary file to it.	9 years ago
beorn7	b274c7aaa7	Update doc comments	9 years ago
beorn7	99881ded63	Make the number of fingerprint mutexes configurable With a lot of series accessed in a short timeframe (by a query, a large scrape, checkpointing, ...), there is actually quite a significant amount of lock contention if something similar is running at the same time. In those cases, the number of locks needs to be increased. On the same front, as our fingerprints don't have a lot of entropy, I introduced some additional shuffling. With the current state, anly changes in the least singificant bits of a FP would matter.	9 years ago
Tobias Schmidt	4c439b4b45	Merge pull request #1646 from prometheus/beorn7/valuecomparison Correctly identify no-op appends if the value is NaN	9 years ago
beorn7	a308c76292	Improve TestAppendOutOfOrder It did not test the returned error so far. Also, add tests for the NaN case broken before https://github.com/prometheus/common/pull/40	9 years ago
beorn7	b2ef4dc52d	Correctly identify no-op appends if the value is NaN This requires an updating of the vendored commen.model package, which I will do once https://github.com/prometheus/common/pull/40 is merged.	9 years ago
Dmitry Vorobev	bd2a770015	storage/remote: Spawn not more than "maxConcurrentSends" goroutines.	9 years ago
Dmitry Savintsev	7fdb62c253	fix several minor golint style issues	9 years ago
Steve Durrheimer	399d5c6375	Make version informations consistent between prometheus components	9 years ago
beorn7	07a294ac15	Doc comment fixes	9 years ago
beorn7	20cba1ed8f	Initialize metric vectors in memorySeriesStorage	9 years ago
beorn7	d566808d40	Bring back logging of discarded samples But only on DEBUG level. Also, count and report the two cases of out-of-order timestamps on the one hand and same timestamp but different value on the other hand separately.	9 years ago
beorn7	db16acd7fb	Never drop a still open head chunk.	9 years ago
beorn7	a90d645378	Checkpoint fingerprint mappings only upon shutdown Before, we checkpointed after every newly detected fingerprint collision, which is not a problem as long as collisions are rare. However, with a sufficient number of metrics or particular nature of the data set, there might be a lot of collisions, all to be detected upon the first set of scrapes, and then the checkpointing after each detection will take a quite long time (it's O(n²), essentially). Since we are rebuilding the fingerprint mapping during crash recovery, the previous, very conservative approach didn't even buy us anything. We only ever read from the checkpoint file after a clean shutdown, so the only time we need to write the checkpoint file is during a clean shutdown.	9 years ago
Jonathan Boulle	38098f8c95	Add missing license headers Prometheus is Apache 2 licensed, and most source files have the appropriate copyright license header, but some were missing it without apparent reason. Correct that by adding it.	9 years ago
Fabian Reinartz	a18639dc2d	Merge pull request #1454 from prometheus/beorn7/fix-test Give TestEvictAndLoadChunkDescs more time to actually evict	9 years ago
Tobias Schmidt	e82ef154ee	Remove unused code leftovers	9 years ago
beorn7	d09ca03e10	Work around compiler bug Benchmarks don't show any significant changes.	9 years ago
beorn7	507f550cd4	Merge branch 'master' into beorn7/storage7	9 years ago
beorn7	865d16f870	Rename Gorilla into varbit	9 years ago
Julius Volz	d3b53bd7f0	Fix comment about Graphite mapping of dimensions.	9 years ago
beorn7	4b574e8a61	Switch chunk encoding to type 2 where it was hardcoded type 1 before The chunk encoding was hardcoded there because it mostly doesn't matter what encoding is chosen in that test. Since type 1 is battle-hardened enough, I'm switching to type 2 here so that we can catch unexpected problems as a byproduct. My expectation is that the chunk encoding doesn't matter anyway, as said, but then "unexpected problems" contains the word "unexpected".	9 years ago
beorn7	c72979e3ed	Remove a redundancy from Gorilla-style chunks So far, the last sample in a chunk was saved twice. That's required for adding more samples as we need to know the last sample added to add more samples without iterating through the whole chunk. However, once the last sample was added to the chunk before it's full, there is no need to save it twice. Thus, the very last sample added to a chunk can _only_ be saved in the header fields for the last sample. The chunk has to be identifiable as closed, then. This information has been added to the flags byte.	9 years ago
beorn7	b6dbb826ae	Improve fuzz testing and fix a bug exposed This improves fuzz testing in two ways: (1) More realistic time stamps. So far, the most common case in practice was very rare in the test: Completely regular increases of the timestamp. (2) Verify samples by scanning through the whole relevant section of the series. For Gorilla-like chunks, this showed two things: (1) With more regularly increasing time stamps, BenchmarkFuzz is essentially as fast as with the traditional chunks: ``` BenchmarkFuzzChunkType0-8 2 972514684 ns/op 83426196 B/op 2500044 allocs/op BenchmarkFuzzChunkType1-8 2 971478001 ns/op 82874660 B/op 2512364 allocs/op BenchmarkFuzzChunkType2-8 2 999339453 ns/op 76670636 B/op 2366116 allocs/op ``` (2) There was a bug related to when and how the chunk footer is overwritten to make use for the last sample. This wasn't exposed by random access as the last sample of a chunk is retrieved from the values in the header in that case.	9 years ago
beorn7	9d8fbbe822	Review improvements	9 years ago
beorn7	8cdced3850	Implement Gorilla-inspired chunk encoding This is not a verbatim implementation of the Gorilla encoding. First of all, it could not, even if we wanted, because Prometheus has a different chunking model (constant size, not constant time). Second, this adds a number of changes that improve the encoding in general or at least for the specific use case of Prometheus (and are partially only possible in the context of Prometheus). See comments in the code for details.	9 years ago
beorn7	8e64e8dfca	Fix return statement.	9 years ago
Björn Rabenstein	98c8560851	Merge pull request #1477 from prometheus/beorn7/storage7 Solve the series churn problem...	9 years ago

1 2 3 4 5 ...

654 Commits (373025539279d3892a1f4ba9384af9bf84c5b4ed)