prometheus

Commit Graph

Author	SHA1	Message	Date
Bjoern Rabenstein	d215e013b7	Fix the weird chunkDesc shuffling bug. The root cause was that after chunkDesc eviction, the offset between memory representation of chunk layout (via chunkDescs in memory) was shiftet against chunks as layed out on disk. Keeping the offset up to date is by no means trivial, so this commit is pretty involved. Also, found a race that for some reason didn't bite us so far: Persisting chunks was completel unlocked, so if chunks were purged on disk at the same time, disaster would strike. However, locking the persisting of chunk revealed interesting dead locks. Basically, never queue under the fp lock. Change-Id: I1ea9e4e71024cabbc1f9601b28e74db0c5c55db8	10 years ago
Bjoern Rabenstein	a617269b12	Avoid unnecessary cloning of the head chunk. Change-Id: I5da774515d5493166a197b5814d0a720628cfaff	10 years ago
Bjoern Rabenstein	f1de5b0c4e	Run checkpointing of in-memory metrics and head chunks periodically. Checkpointing interval is now a command line flag. Along the way, several things were refactored. - Restructure the way the storage is started and stopped.. - Number of series in checkpoint is now a uint64, not a varint. (Breaks old checkpoints, needs wipe!) - More consistent naming and order of methods. Change-Id: I883d9170c9a608ee716bb0ab3d0ded8ca03760d9	10 years ago
Bjoern Rabenstein	74c9b34a5e	Improve storage instrumentation even more. Add gauge for chunks and chunkdescs in memory (backed by a global variable to be used later not only for instrumentation but also for memory management). Refactored instrumentation code once more (instrumentation.go is back :). Change-Id: Ife39947e22a48cac4982db7369c231947f446e17	10 years ago
Julius Volz	c3fcea45e3	Support finer time resolutions than 1 second. Change-Id: I4c5f1d6d2361e841999b23283d1961b1bd0c2859	10 years ago
Julius Volz	0712d738d1	Allow alternative "by"-clause position in grammar. In addition to the existing by-clause syntax: sum(<expression>) by (<labels>) [keeping_extra] ...this allows the following new syntax: sum by (<labels>) [keeping_extra] (<expression>) Both orderings may be used in a single expression. It is up to the users to establish guidelines around their usage. Change-Id: Iba10c9cc5fb6ac62edfcf246d281473e82467992	10 years ago
Brian Brazil	f114bbd4e7	Make query_range more robust. Gracefully handle decimal values, by truncating them. Limit amount of steps, to avoid accidentally pulling too much data. This limit returns up to ~500kB per timeseries, and allows for 60s granularity for a week and 1h granularity for a year. Change-Id: Ie549fc24deb2eecbc6c5d1b6088a548a6b02e849	10 years ago
Brian Brazil	75e37db55b	Don't alert() when a query is aborted, such as when you change the range. Change-Id: I574504f97446ac5f3dda737fe054ae83f17dbbc2	10 years ago
Julius Volz	0e48c18bbf	Allow omitting the metric name in queries. This allows the following expression syntaxes for selecting timeseries: foo (already valid before) foo{} (already valid before) {job="prometheus"} (new, select all timeseries for job "prometheus") Omitting both the metric name and any label matchers ("" or "{}") will still yield a syntax error. To get all timeseries, you could do: {__name__=~"."} or, without relying on knowledge about __metric__: {job=~"."} Change-Id: Ifee000b9ac0184ef6ced18411069c7f2699a2dda	10 years ago
Bjoern Rabenstein	443dd33805	Improve instrumentation in storage. Also, fix some other minor bugs. Change-Id: If72f1c058b0f47d3e378fdf80228d7e9a8db06c7	10 years ago
Julius Volz	351e66c5d2	Remove obsolete "coding" directory. Change-Id: I9e71098fd526336b4af7f7eab6d2a43504079658	10 years ago
Bjoern Rabenstein	1936a40e75	Minor loging improvement. Change-Id: I7875d1a58ef9c5ff149f18e36f65959a4712fea2	10 years ago
Bjoern Rabenstein	192bf52c41	Evict chunkDescs, too. Change-Id: I8b70f22fbf1dfcbc49f9ec391985144649e6ce9c	10 years ago
Bjoern Rabenstein	95f392fb2c	Prevent an indexing death spiral. Change-Id: I86b20cd0830d02f87b2f020767257e2d3fb2033c	10 years ago
Bjoern Rabenstein	40354eaa29	Reduce directory depth by one. Change-Id: I7f89df61135ff19169ed97633a662685d414c448	10 years ago
Bjoern Rabenstein	096fa0f8b2	Squash a number of TODOs. - Staleness delta is no a proper function parameter and not replicated from package ast. - Named type 'chunks' replaced by explicit '[]chunk' to avoid confusion. - For the same reason, replaced 'chunkDescs' by '[]*chunkDescs'. - Verified that math.Modf is not a speed enhancement over conversion (actually 5x slower). - Renamed firstTimeField, lastTimeField into chunkFirstTime and chunkLastTime. - Verified unpin() is sufficiently goroutine-safe. - Decided not to update archivedFingerprintToTimeRange upon series truncation and added a rationale why. Change-Id: I863b8d785e5ad9f71eb63e229845eacf1bed8534	10 years ago
Bjoern Rabenstein	427c8d53a5	Fix handling of empty chunkDescs while preloading chunks. Change-Id: I73ce89fe0ef90c6eda78218e5be2cbfa0207c364	10 years ago
Bjoern Rabenstein	ecee5d8281	Fix head chunk persisting and a chunkDesc race condition. - Head chunk persisting only happens in evictOlderThan, so do it there. (With the previous code, it would never happen.) - Raw accesses to chunkDesc.chunk are now done via isEvicted (with locking). Change-Id: I48b07b56dfea4899b50df159b4ea566954396fcd	10 years ago
Bjoern Rabenstein	9c3ecc2134	Remove unused flags. Change-Id: Ie1bcbb0743d65e92072628811706d49753023205	10 years ago
Bjoern Rabenstein	6b37e47f9e	Remove unused metrics. Change-Id: Icf03ba4ce92a5e38daf12930f9661daba79c83bb	10 years ago
Bjoern Rabenstein	2b4ff620aa	Return a nop iterator for series that have been purged completely. Change-Id: I6e92cac4472486feefdecba8593c17867e8c710d	10 years ago
Bjoern Rabenstein	6e3a366f91	Only archive a time series when none of its chunks is pinned. Change-Id: I7e4b67c34b417b8980173bc5dc3b213bd7d698e5	10 years ago
Julius Volz	bfa64248b7	Deal with missing series in preloading. Change-Id: Ibf3a57b329f40a3d5e0b98464a2f45d2f1bd07bf	10 years ago
Bjoern Rabenstein	ca42a22e20	Add safety panic to seriesMap.put. Change-Id: I4d4d2e45cc0f908a33eb1ae6e3ee6796adfcbd1e	10 years ago
Bjoern Rabenstein	83b4fa868d	Fix GetBoundaryValues. Change-Id: I8f8bbdb88e9b24e4c37ff869126ed9343f261ce2	10 years ago
Bjoern Rabenstein	4fc8ad6677	Fix retrieval unit tests. Change-Id: I299b71406b59539230e5182ccc37bc8a83af60b3	10 years ago
Bjoern Rabenstein	b3ed9aa7a2	Clean up start-up and shut-down. Change-Id: Idff4bbb0a15a9f879bfbb3da5b1025179cab5e2c	10 years ago
Bjoern Rabenstein	4447708c9f	Fix a race in target.go. Also, fix problems in shutdown. Starting serving and shutdown still has to be cleaned up properly. It's a mess. Change-Id: I51061db12064e434066446e6fceac32741c4f84c	10 years ago
Bjoern Rabenstein	fd6600850a	Fix race in chunkDesc. Change-Id: Id7bae115d75886e10d44184a690a76777b1531fe	10 years ago
Bjoern Rabenstein	1c53c09558	Treat empty chunkDescs properly in preloadChunksForRange. Change-Id: Ida1bd3fe1f9fb0ea2d5dbb9704be926f0824f873	10 years ago
Bjoern Rabenstein	934d09f738	Fix race during shutdown. Change-Id: I2f8bf48d92a14f1e5ecde27c1b138734d7653394	10 years ago
Bjoern Rabenstein	38fc24d0ed	Fix targetpool_test.go and other tests. Change-Id: I91a4dd1d39e01f174e1aaae653ce1ed7aecaa624	10 years ago
Brian Brazil	fd34e4061d	Add back consoles link. Goes in index.html in consoles or else user data, if present. Change-Id: I5303d30aa24ca0c20d2e0f49121e04a260b9c4f4	10 years ago
Andres Suarez	e389e63684	Focus expression after selection from dropdown Change-Id: Id7f67e558e3611ab4c7188cc428c342d8d3e67db	10 years ago
Andres Suarez	86a447fc0e	Allow selecting metric from Insert Metric Change-Id: I99e0539cab2749a8aeabc0a13015889ff45834f7	10 years ago
Julius Volz	7f5d3c2c29	Fix and improve the fp locker. Benchmark: $ go test -bench 'Fingerprint' -test.run 'Fingerprint' -test.cpu=1,2,4 OLD BenchmarkFingerprintLockerParallel 500000 3618 ns/op BenchmarkFingerprintLockerParallel-2 100000 12257 ns/op BenchmarkFingerprintLockerParallel-4 500000 10164 ns/op BenchmarkFingerprintLockerSerial 10000000 283 ns/op BenchmarkFingerprintLockerSerial-2 10000000 284 ns/op BenchmarkFingerprintLockerSerial-4 10000000 288 ns/op NEW BenchmarkFingerprintLockerParallel 1000000 1018 ns/op BenchmarkFingerprintLockerParallel-2 1000000 1164 ns/op BenchmarkFingerprintLockerParallel-4 2000000 910 ns/op BenchmarkFingerprintLockerSerial 50000000 56.0 ns/op BenchmarkFingerprintLockerSerial-2 50000000 47.9 ns/op BenchmarkFingerprintLockerSerial-4 50000000 54.5 ns/op Change-Id: I3c65a43822840e7e64c3c3cfe759e1de51272581	10 years ago
Bjoern Rabenstein	7ad55ef83c	Actually close the iterator channels. Change-Id: I6f6a2aef5ff55c6b2d21ad91d02ae6b0ecba4ae8	10 years ago
Bjoern Rabenstein	8fba3302bc	Bold changes to concurrency. (WIP. Probably doesn't work yet.) Change-Id: Id1537dfcca53831a1d428078a5863ece7bdf4875	10 years ago
Bjoern Rabenstein	fcdf5a8ee7	Fix bugs in chunk evict code. Also, simplify code by re-looking up metric in metric map. Change-Id: Ib2092f9184374e5a543e87d3a9f4a74fda64b193	10 years ago
Bjoern Rabenstein	7e6a03fbf9	Fix a few concurrency issues before starting to use the new fp locker. Change-Id: I8615e8816e79ef0882e123163ee590c739b79d12	10 years ago
Julius Volz	db92620163	Instrument eviction and purge durations. Change-Id: Ia5b2319363ad2644674c9b7a94162a89bcc296fb	10 years ago
Julius Volz	e0ee7ec7ab	Add fingerprintLocker for locking individual fingerprints. Change-Id: Id41ba555715229edf7d6543f56736b82f6eff1ef	10 years ago
Julius Volz	df1b2a2422	Fix indexing latency instrumentation. Change-Id: I532c170121cd2996d1a378adbb1fd551cd5a4e38	10 years ago
Bjoern Rabenstein	01dd618a20	Fix a locking bug. Change-Id: I183780785991d0b4165ce9186f53eb8201fb3ed5	10 years ago
Julius Volz	358f97791d	Minor cleanups. Change-Id: Ia8685d8439a421fe2143d9ec7120d5bb5ab88d78	10 years ago
Julius Volz	a746fbb8bc	Instrument indexing: queue length, batch sizes and latencies. Change-Id: I60bcbd24b160e47d418a485d8cffa39344a257c6	10 years ago
Bjoern Rabenstein	aea32b0b4b	Avoid redundant fingerprint calculation. Change-Id: Ief8a165dcfa5030226953346ec9dfe4a7787df1f	10 years ago
Bjoern Rabenstein	e9ff29c547	Comment/code cleanup. Change-Id: I38736e3d0fec79759a2bafa35aecf914480ff810	10 years ago
Bjoern Rabenstein	0031a448e2	Add WaitForIndexing. Change-Id: I5a5c975c4246632f937413322c855bbe63d00802	10 years ago
Bjoern Rabenstein	c7aad110fb	Add an indexing queue and batch the ops. Some other improvements on the way, in particular codec -> codable renaming and addition of LookupSet methods. Change-Id: I978f8f3f84ca8e4d39a9d9f152ae0ad274bbf4e2	10 years ago

1 2 3 4 5 ...

1056 Commits (d215e013b78255152598475938302e42444614f6) All Branches Search

1056 Commits (d215e013b78255152598475938302e42444614f6)

All Branches