prometheus

Commit Graph

Author	SHA1	Message	Date
Fabian Reinartz	1535ef1457	Replace metric.SamplePair with model.SamplePair	9 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	9 years ago
beorn7	699946bf32	Fix chunk desc loading. If all samples in consecutive chunks have the same timestamp, the way we used to load chunks will fail. With this change, the persist watermark is used to load the right amount of chunkDescs from disk. This bug is a possible reason for the rare storage corruption we have observed.	10 years ago
Fabian Reinartz	6bfb4549a6	storage: add LastSamplePairForFingerprint method	10 years ago
Fabian Reinartz	b105e26f4d	storage: remove global flags	10 years ago
beorn7	3b9c421a69	Weed out all the [Gg]et* method names. The only exception is getNumChunksToPersist to avoid naming the struct member numChunksToPersist in a weird way.	10 years ago
beorn7	cd5574bf8a	Make chunk and series iterators more efficient.	10 years ago
beorn7	b02d900e61	Improve chunk and chunkDesc loading. Also, clean up some things in the code (especially introduction of the chunkLenWithHeader constant to avoid the same expression all over the place). Benchmark results: BEFORE BenchmarkLoadChunksSequentially 5000 283580 ns/op 152143 B/op 312 allocs/op BenchmarkLoadChunksRandomly 20000 82936 ns/op 39310 B/op 99 allocs/op BenchmarkLoadChunkDescs 10000 110833 ns/op 15092 B/op 345 allocs/op AFTER BenchmarkLoadChunksSequentially 10000 146785 ns/op 152285 B/op 315 allocs/op BenchmarkLoadChunksRandomly 20000 67598 ns/op 39438 B/op 103 allocs/op BenchmarkLoadChunkDescs 20000 99631 ns/op 12636 B/op 192 allocs/op Note that everything is obviously loaded from the page cache (as the benchmark runs thousands of times with very small series files). In a real-world scenario, I expect a larger impact, as the disk operations will more often actually hit the disk. To load ~50 sequential chunks, this reduces the iops from 100 seeks and 100 reads to 1 seek and 1 read.	10 years ago
beorn7	5bea942d8e	Improve various things around chunk encoding. A number of mostly minor things: - Rename chunk type -> chunk encoding. - After all, do not carry around the chunk encoding to all parts of the system, but just have one place where the encoding for new chunks is set based on the flag. The new approach has caveats as well, but the polution of so many method signatures is worse. - Use the default chunk encoding for new chunks of existing series. (Previously, only new _series_ would get chunks with the default encoding.) - Use an enum for chunk encoding. (But keep the version number for the flag, for reasons discussed previously.) - Add encoding() to the chunk interface (so that a chunk knows its own encoding - no need to have that in a different top-level function). - Got rid of newFollowUpChunk (which would keep the existing encoding for all chunks of a time series). Now only use newChunk(), which will create a chunk encoding according to the flag. - Simplified transcodeAndAdd. - Reordered methods of deltaEncodedChunk and doubleDeltaEncoded chunk to match the order in the chunk interface. - Only transcode if the chunk is not yet half full. If more than half full, add a new chunk instead.	10 years ago
beorn7	13fcf1ddbc	Implement double-delta encoded chunks.	10 years ago
Bjoern Rabenstein	5859b74f1b	Clean up license issues. - Move CONTRIBUTORS.md to the more common AUTHORS. - Added the required NOTICE file. - Changed "Prometheus Team" to "The Prometheus Authors". - Reverted the erroneous changes to the Apache License.	10 years ago
Bjoern Rabenstein	fee88a7a77	Remove the remaining races, new and old. Also, resolve a few other TODOs. Change-Id: Icb39b5a5e8ca22ebcb48771cd8951c5d9e112691	10 years ago
Bjoern Rabenstein	7d11019aa2	Squash a few trivial TODOs. - Delete unneeded file view_adapter.go. - Assessed that we still need the fingerprints in nodes (to create iterators). - Turned numMemChunkDescs into a metric. Change-Id: I29be963c795a075ec00c095f76bf26405535609d	10 years ago
Bjoern Rabenstein	14bda4180c	Changes after pair code review. Change-Id: Ib72d40f8e9027818cfbbd32a7a7201eebda07455	10 years ago
Bjoern Rabenstein	bb42cc2e2d	Evict based on memory pressure. Evict recently used chunks last. Change-Id: Ie6168f0cdb3917bdc63b6fe15585dd70c1e42afe	10 years ago
Bjoern Rabenstein	a617269b12	Avoid unnecessary cloning of the head chunk. Change-Id: I5da774515d5493166a197b5814d0a720628cfaff	10 years ago
Bjoern Rabenstein	74c9b34a5e	Improve storage instrumentation even more. Add gauge for chunks and chunkdescs in memory (backed by a global variable to be used later not only for instrumentation but also for memory management). Refactored instrumentation code once more (instrumentation.go is back :). Change-Id: Ife39947e22a48cac4982db7369c231947f446e17	10 years ago
Bjoern Rabenstein	443dd33805	Improve instrumentation in storage. Also, fix some other minor bugs. Change-Id: If72f1c058b0f47d3e378fdf80228d7e9a8db06c7	10 years ago
Bjoern Rabenstein	096fa0f8b2	Squash a number of TODOs. - Staleness delta is no a proper function parameter and not replicated from package ast. - Named type 'chunks' replaced by explicit '[]chunk' to avoid confusion. - For the same reason, replaced 'chunkDescs' by '[]*chunkDescs'. - Verified that math.Modf is not a speed enhancement over conversion (actually 5x slower). - Renamed firstTimeField, lastTimeField into chunkFirstTime and chunkLastTime. - Verified unpin() is sufficiently goroutine-safe. - Decided not to update archivedFingerprintToTimeRange upon series truncation and added a rationale why. Change-Id: I863b8d785e5ad9f71eb63e229845eacf1bed8534	10 years ago
Bjoern Rabenstein	71206dbc06	More code cleanups. Add license text everywhere. And others.... Change-Id: I11ccde267a2ef7eb366c4788ba7aeae14ba7545c	10 years ago
Bjoern Rabenstein	f5f9f3514a	Major code cleanup. - Make it go-vet and golint clean. - Add comments, TODOs, etc. Change-Id: If1392d96f3d5b4cdde597b10c8dff1769fcfabe2	10 years ago
Julius Volz	8dfaa5ecd2	Remove use of freelists for chunk bufs. Change-Id: Ib887fdb61e1d96da0cd32545817b925ba88831c1	10 years ago
Bjoern Rabenstein	d742edfe0d	Fix precision loss. Large delta values often imply a difference between a large base value and the large delta value, potentially resulting in small numbers with a huge precision error. Since large delta values need 8 bytes anyway, we are not even saving memory. As a solution, always save the absoluto value rather than a delta once 8 bytes would be needed for the delta. Timestamps are then saved as 8 byte integers, while values are always saved as float64 in that case. Change-Id: I01100d600515e16df58ce508b50982ffd762cc49	10 years ago
Julius Volz	e7ed39c9a6	Initial experimental snapshot of next-gen storage. Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d	10 years ago

24 Commits (91436b2974e44f10ec7bfd6d3158a8f58176ad29)