prometheus

Commit Graph

Author	SHA1	Message	Date
Julius Volz	630b5a087a	Also consider on-disk fingerprints during purge. This reintroduces LevelDB iterators so that we can iterate through all the on-disk fingerprints. Change-Id: I007ee4638d038d2a4461bbda27f30fcaad411474	10 years ago
Bjoern Rabenstein	f5f9f3514a	Major code cleanup. - Make it go-vet and golint clean. - Add comments, TODOs, etc. Change-Id: If1392d96f3d5b4cdde597b10c8dff1769fcfabe2	10 years ago
Bjoern Rabenstein	3592dc2359	Implement series eviction. Change-Id: I7a503e0ba78aae3761d032851b06f2807122b085	10 years ago
Bjoern Rabenstein	bbf49200ab	Implement methods in persistence.go. Change-Id: I804cdd0b30420e171825fd86fe1281eca0d5e638	10 years ago
Bjoern Rabenstein	5a128a04a9	Major reorganization of the storage. Most important, the heads file will now persist all the chunk descs, too. Implicitly, it will serve as the persisted form of the fp-to-series map. Change-Id: Ic867e78f2714d54c3b5733939cc5aef43f7bd08d	10 years ago
Bjoern Rabenstein	e7cb9ddb9f	Use a sync.pool for the staging buffer in codec.go. Change-Id: I1aae6847f77b5a7c75582b07c199b1943cf90552	10 years ago
Bjoern Rabenstein	4770cf76a4	Make index package more self-contained. Moved interna from diskPersistence into the indexer. TotalIndexer now called diskIndexer. Change-Id: I6c8c62cb171f12bbd8a5474773af7786d71ba388	10 years ago
Bjoern Rabenstein	89f10e8eb2	Move to using the standard library interfaces for encoding/decoding. BinaryMarshaler instead of encodable. BinaryUnmarshaler instead of decodable. Left 'codable' in place for lack of a better word. Change-Id: I8a104be7d6db916e8dbc47ff95e6ff73b845ac22	10 years ago
Bjoern Rabenstein	af77d5ef0b	Added a few missing implementations in index.go. Also, added closing of persistence and mem storage. Change-Id: Iacf0d22c3520dd2584d9546984c1f8a5ed6cd54e	10 years ago
Bjoern Rabenstein	70e4837650	Cut v0.8.0. Change-Id: Ie8d49793e78f10bdeb7ebe19cc2dc729ff7ef590	10 years ago
Bjoern Rabenstein	e0a6cb281e	Fix the accept header. A '/' is a separator and has to be in a quoted string. Change-Id: If7a3a847f84f8f709074d05dc98b5b21e954030c	10 years ago
Julius Volz	cca7ebe906	Some more cleanups / obsolete code removals. Change-Id: I584144ceeeedafdb114266d8a6d2513e67b1d010	10 years ago
Julius Volz	7e85711df0	Beginnings of a tiered index implementation. This reintroduces a LevelDB-based metrics index. Change-Id: I4111540301c52255a07b2f570761707a32f72c05	10 years ago
Julius Volz	8dfaa5ecd2	Remove use of freelists for chunk bufs. Change-Id: Ib887fdb61e1d96da0cd32545817b925ba88831c1	10 years ago
Julius Volz	7b35e0f0b8	Use constants from math package instead of literals. Change-Id: I55427ba32c2cbb32ee42ec1e3153160965ab8b3c	10 years ago
Julius Volz	15929eece2	Unpin any already loaded chunks upon preloading error. Change-Id: Ib451136e3ef21bce8b814c21b66eaab727ab341b	10 years ago
Julius Volz	fd01d07589	Check that chunk buffer length fits in 16 bit. Change-Id: Id086a54aa8a1990c1979e747c1c02e53bed6d447	10 years ago
Bjoern Rabenstein	1ca7f24137	Remove float diff tolerance altogether. Change-Id: I9ea9683a4665d5800fca75560bb4b8a8b4406d55	10 years ago
Bjoern Rabenstein	d742edfe0d	Fix precision loss. Large delta values often imply a difference between a large base value and the large delta value, potentially resulting in small numbers with a huge precision error. Since large delta values need 8 bytes anyway, we are not even saving memory. As a solution, always save the absoluto value rather than a delta once 8 bytes would be needed for the delta. Timestamps are then saved as 8 byte integers, while values are always saved as float64 in that case. Change-Id: I01100d600515e16df58ce508b50982ffd762cc49	10 years ago
Bjoern Rabenstein	dc2e463a97	Improvements after review. Change-Id: I484359282d4c7113518bbbb131f4f18383c08fdb	10 years ago
Bjoern Rabenstein	52c9dc43a3	Improve testing. In particular, create a fuzz test for time series. Change-Id: I523a17912405a0b6b46bd395c781d201dfe55036	10 years ago
Julius Volz	3b25867d61	Add chunk persistence tests, fix storage tests. Change-Id: Id0b8f5382e99efa839cc0f826e92bbda985fe9a9	10 years ago
Bjoern Rabenstein	ecdf5ab14f	Index-persistence switched from gob to a hand-coded solution. Change-Id: Ib4ec42535bd08df16d34d4774bb638e35c5a1841	10 years ago
Julius Volz	e7ed39c9a6	Initial experimental snapshot of next-gen storage. Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d	10 years ago
Julius Volz	134bd8fe34	Format changelog properly. Change-Id: I62c5bf8c5b880272d207da564a3fc45490c5db5e	10 years ago
Brian Brazil	5edf689133	Stagger scrapes to spread out load. Change-Id: Ib141b271e4adfb817886871f86051c207b05cf35	10 years ago
Julius Volz	1aee992800	Prometheus version 0.7.0. Change-Id: I73468f72b43654f4bf57627c2f49fe802b18f637	10 years ago
Julius Volz	c6e9f085a3	Update used Go version to 1.3. Go downloads moved to a different URL and require following redirects (curl's '-L' option) now. Go 1.3 deliberately randomizes ranges over maps, which uncovered some bugs in our tests. These are fixed too. Change-Id: Id2d9e185d8d2379a9b7b8ad5ba680024565d15f4	10 years ago
Julius Volz	85497e3f38	Add function to drop common labels in a vector. This fixes https://github.com/prometheus/prometheus/issues/384. Change-Id: I2973c4baeb8a4618ec3875fb11c6fcf5d111784b	10 years ago
Julius Volz	3fdb74e571	Add more topk() / bottomk() tests. Test what happens if k > number of input elements. Change-Id: Ie724b850939e297ebf085f0a5a3522e9cfcc6534	10 years ago
Julius Volz	c582ae73c2	Implement topk() and bottomk() functions. To achieve O(log n * k) runtime, this uses a heap to track the current bottom-k or top-k elements while iterating over the full set of available elements. It would be possible to reuse more code between topk and bottomk, but I decided for some more duplication for the sake of clarity. This fixes https://github.com/prometheus/prometheus/issues/399 Change-Id: I7487ddaadbe7acb22ca2cf2283ba6e7915f2b336	10 years ago
Bjoern Rabenstein	1909686789	Make metrics exported by the Prometheus server itself more consistent. - Always spell out the time unit (e.g. milliseconds instead of ms). - Remove "_total" from the names of metrics that are not counters. - Make use of the "Namespace" and "Subsystem" fields in the options. - Removed the "capacity" facet from all metrics about channels/queues. These are all fixed via command line flags and will never change during the runtime of a process. Also, they should not be part of the same metric family. I have added separate metrics for the capacity of queues as convenience. (They will never change and are only set once.) - I left "metric_disk_latency_microseconds" unchanged, although that metric measures the latency of the storage device, even if it is not a spinning disk. "SSD" is read by many as "solid state disk", so it's not too far off. (It should be "solid state drive", of course, but "metric_drive_latency_microseconds" is probably confusing.) - Brian suggested to not mix "failure" and "success" outcome in the same metric family (distinguished by labels). For now, I left it as it is. We are touching some bigger issue here, especially as other parts in the Prometheus ecosystem are following the same principle. We still need to come to terms here and then change things consistently everywhere. Change-Id: If799458b450d18f78500f05990301c12525197d3	10 years ago
Brian Brazil	4a2b96f848	Remove backoff on scrape failure. Having metrics with variable timestamps inconsistently spaced when things fail will make it harder to write correct rules. Update status page, requires some refactoring to insert a function. Change-Id: Ie1c586cca53b8f3b318af8c21c418873063738a8	10 years ago
Julius Volz	00b9489f1c	Fix time() behavior. time() should return the timestamp for which the query is executed, not the actual current time. Change-Id: I430a45cabad7785cd58f95b1028a71dff4c87710	10 years ago
Julius Volz	c5984f1818	Add abs() and over-time aggregation functions. This implements aggregation functions over time as request in https://github.com/prometheus/prometheus/issues/383. Change-Id: Ifd69b850de8cfdf6e7a6c0e042056fa4c672410e	10 years ago
Julius Volz	1bb7074fec	Fix HTTP connection leak upon non-OK status. Change-Id: Ie7fbd7dcc089b8306b40631be3e3d736c23c1cd3	10 years ago
Brian Brazil	144d5bb9fd	Add 'tmpl', a 'template' for non-string literal names. Change-Id: I6a03a5c5d20029cf414562efa7745ed6c53b2731	10 years ago
Brian Brazil	f525ca5d9e	Let consoles get graph links from experssions. Rename ConsoleLinkFromExpression, as we now have consoles. Change-Id: I7ed2c9c83863adb390b51121dd9736845f7bcdfc	10 years ago
Brian Brazil	eba205fcac	Expose path used to get to console to console. Change-Id: I72386a2d4e53863da302ecc5c7e44d6c310197e0	10 years ago
Johannes 'fish' Ziemke	aed1d384a9	Build prometheus tools as well Change-Id: I49d5ca4d6ff715e8a6631caf052de309b91b0b1b	10 years ago
Brian Brazil	eb5d928da7	Fix console handler. This was accidnetally broken in `2128d9d811`. Change-Id: I50ea1fdb8ae4d28ae4555410bee97e5037692aa5	10 years ago
Bjoern Rabenstein	bacc31d5cc	Remove work-around that required copying all bytes of a scrape. Now that the subtle bug in matttproud/golang_protobuf_extensions is fixed, we do not need to copy the bytes of a scrape into a buffer first before starting to parse it. Change-Id: Ib73ecae16173ddd219cda56388a8f853332f8853	10 years ago
Julius Volz	74de633a3a	Prometheus version 0.6.0. Change-Id: I50f6b69cca952eedf9a62b9a8f58e0fb633a83ed	10 years ago
Julius Volz	80b3d3bf34	Speed up disk flushes by removing unnecessary sort. The first sort in groupByFingerprint already ensures that all resulting sample lists contain only one fingerprint. We also already assume that all samples passed into AppendSamples (and thus groupByFingerprint) are chronologically sorted within each fingerprint. The extra chronological sort is thus superfluous. Furthermore, this second sort didn't only sort chronologically, but also compared all metric fingerprints again (although we already know that we're only sorting within samples for the same fingerprint). This caused a huge memory and runtime overhead. In a heavily loaded real Prometheus, this brought down disk flush times from ~9 minutes to ~1 minute. OLD: BenchmarkLevelDBAppendRepeatingValues 5 331391808 ns/op 44542953 B/op 597788 allocs/op BenchmarkLevelDBAppendsRepeatingValues 5 329893512 ns/op 46968288 B/op 3104373 allocs/op NEW: BenchmarkLevelDBAppendRepeatingValues 5 299298635 ns/op 43329497 B/op 567616 allocs/op BenchmarkLevelDBAppendsRepeatingValues 20 92204601 ns/op `1779454` B/op 70975 allocs/op Change-Id: Ie2d8db3569b0102a18010f9e106e391fda7f7883	10 years ago
Julius Volz	21cafe6cd7	Only evict memory series after they are on disk. This fixes the problem where samples become temporarily unavailable for queries while they are being flushed to disk. Although the entire flushing code could use some major refactoring, I'm explicitly trying to do the minimal change to fix the problem since there's a whole new storage implementation in the pipeline. Change-Id: I0f5393a30b88654c73567456aeaea62f8b3756d9	10 years ago
Bjoern Rabenstein	8956faeccb	Migrate to new client_golang. This change will only be submitted when the new client_golang has been moved to the new version. Change-Id: Ifceb59333072a08286a8ac910709a8ba2e3a1581	10 years ago
Bjoern Rabenstein	814e479723	Treat non-200 HTTP response as error. Change-Id: I2a9f3b47012b3c4839be53aa44c66d16dd41a24a	10 years ago
Brian Brazil	e27447da5c	Remove the broken "User Dashboard" link. Due to the lack of a </a>, this makes the entire header render badly. Accordingly it's safe to assume noone is using it, so remove it. With the new console template support, we'll need to something a bit more nuanced later. Change-Id: I3424bed6aea18cbd4c63ad48f98808098dadc3ad	10 years ago
Brian Brazil	2f76f434a5	Add humanizeDuration function. This attempts to reasonably handle things from weekly cronjobs, to rpcs taking ns to things that are usually ms but jump to over a second. For consistency, stop putting spaces before prefixes. Change-Id: I6407879187b25680b323cd70254e205315b5fc3c	10 years ago
Brian Brazil	960ede66dc	Use html/template for console templates and add template libary support. Add a function to bypass the new auto-escaping. Add a function to workaround go's templates only allowing passing in one argument. Change-Id: Id7aa3f95e7c227692dc22108388b1d9b1e2eec99	10 years ago

1 2 3 4 5 ...

1004 Commits (630b5a087a14784797dce8bba1402d613a36ad86) All Branches Search

1004 Commits (630b5a087a14784797dce8bba1402d613a36ad86)

All Branches