prometheus

Commit Graph

Author	SHA1	Message	Date
Julius Volz	33702da8a8	Use simple Now() func in API instead of utility.Time.	10 years ago
Julius Volz	3f2686d0b3	Remove unused fields from MetricsService.	10 years ago
beorn7	12ae6e9203	Increase resilience of the storage against data corruption - step 4. Step 4: Add a configurable sync'ing of series files after modification.	10 years ago
beorn7	11bd9ce1bd	Increase resilience of the storage against data corruption - step 3. Step 3: Remember the mtime of series files and make use of it to detect series files that are not the one the checkpoint thinks they are.	10 years ago
beorn7	e25cca823c	Increase resilience of the storage against data corruption - step 2. Step 2: Add a flag -storage.local.pedantic-checks to check every series file. Also, remove countPersistedHeadChunks channel, which is unused.	10 years ago
beorn7	da7c0461c6	Rename persist queue len/cap to num/max chunks to persist. Remove deprecated flag storage.incoming-samples-queue-capacity.	10 years ago
beorn7	be11cb2b07	Remove the sample ingestion channel. The one central sample ingestion channel has caused a variety of trouble. This commit removes it. Targets and rule evaluation call an Append method directly now. To incorporate multiple storage backends (like OpenTSDB), storage.Tee forks the Append into two different appenders. Note that the tsdb queue manager had its own queue anyway. It was a queue after a queue... Much queue, so overhead... Targets have their own little buffer (implemented as a channel) to avoid stalling during an http scrape. But a new scrape will only be started once the old one is fully ingested. The contraption of three pipelined ingesters was removed. A Target is an ingester itself now. Despite more logic in Target, things should be less confusing now. Also, remove lint and vet warnings in ast.go.	10 years ago
beorn7	0056eaeb4f	Redesign series maintenance and chunk persistence.	10 years ago
beorn7	5bea942d8e	Improve various things around chunk encoding. A number of mostly minor things: - Rename chunk type -> chunk encoding. - After all, do not carry around the chunk encoding to all parts of the system, but just have one place where the encoding for new chunks is set based on the flag. The new approach has caveats as well, but the polution of so many method signatures is worse. - Use the default chunk encoding for new chunks of existing series. (Previously, only new _series_ would get chunks with the default encoding.) - Use an enum for chunk encoding. (But keep the version number for the flag, for reasons discussed previously.) - Add encoding() to the chunk interface (so that a chunk knows its own encoding - no need to have that in a different top-level function). - Got rid of newFollowUpChunk (which would keep the existing encoding for all chunks of a time series). Now only use newChunk(), which will create a chunk encoding according to the flag. - Simplified transcodeAndAdd. - Reordered methods of deltaEncodedChunk and doubleDeltaEncoded chunk to match the order in the chunk interface. - Only transcode if the chunk is not yet half full. If more than half full, add a new chunk instead.	10 years ago
beorn7	66e768f05e	Improve docstring for chunk type flag.	10 years ago
beorn7	13fcf1ddbc	Implement double-delta encoded chunks.	10 years ago
beorn7	af91fb8e31	Improve persisting chunks to disk. This is done by bucketing chunks by fingerprint. If the persisting to disk falls behind, more and more chunks are in the queue. As soon as there are "double hits", we will now persist both chunks in one go, doubling the disk throughput (assuming it is limited by disk seeks). Should even more pile up so that we end wit "triple hits", we will persist those first, and so on. Even if we have millions of time series, this will still help, assuming not all of them are growing with the same speed. Series that get many samples and/or are not very compressable will accumulate chunks faster, and they will soon get double- or triple-writes. To improve the chance of double writes, -storage.local.persistence-queue-capacity could be set to a higher value. However, that will slow down shutdown a lot (as the queue has to be worked through). So we leave it to the user to set it to a really high value. A more fundamental solution would be to checkpoint not only head chunks, but also chunks still in the persist queue. That would be quite complicated for a rather limited use-case (running many time series with high ingestion rate on slow spinning disks).	10 years ago
beorn7	8a1c195b54	Move emptiness check to the receivers.	10 years ago
beorn7	11b3c2387c	Improvements after review. - Increase samplesQueueCapacity. - Improve docstring for the above. - Accept a short waiting period for the ingest channel to become ready. This should depend on the http timeout, but 100ms is probably good enough to cushion bursts bigger than samplesQueueCapacity, while it is unlikely that anybody ever will set an HTTP timeout similarly short.	10 years ago
beorn7	d2ab49c396	Make the persist queue length configurable. Also, set a much higher default value. Chunk persist requests can be quite spiky. If you collect a large number of time series that are very similar, they will tend to finish up a chunk at about the same time. There is no reason we need to back up scraping just because of that. The rationale of the new default value is "1/8 of the chunks in memory".	10 years ago
Bjoern Rabenstein	5859b74f1b	Clean up license issues. - Move CONTRIBUTORS.md to the more common AUTHORS. - Added the required NOTICE file. - Changed "Prometheus Team" to "The Prometheus Authors". - Reverted the erroneous changes to the Apache License.	10 years ago
juliusv	cca2e58f20	Merge pull request #442 from prometheus/beorn7/fix-crash-recovery Fix ALL the crash-recovery related problems.	10 years ago
Bjoern Rabenstein	0851945054	Add a heuristics to checkpoint early if there are many "dirty" series..	10 years ago
Julius Volz	d6b9e97655	Remove extraction.Result type, simplify code.	10 years ago
Bjoern Rabenstein	b1e4956142	Apply a giant code cleanup. Essentially: - Remove unused code. - Make it 'go vet' clean. The only remaining warnings are in generated code. - Make it 'golint' clean. The only remaining warnings are in gerenated code. - Smoothed out same minor things. Change-Id: I3fe5c1fbead27b0e7a9c247fee2f5a45bc2d42c6	10 years ago
Bjoern Rabenstein	ea86f7e8f8	Fix weird things after merge. And I swear I'll never use 'rebase' to 'clean something up' ever agin, even if Julius tells me to do so... Change-Id: Ifeabab20445279bf693c95f062da769b60fe195f	10 years ago
Bjoern Rabenstein	3a17aeabfd	Merge branch 'beorn/storage-ng-with-commit-history-cleaned-up' Conflicts: Makefile Makefile.INCLUDE VERSION main.go notification/notification.go retrieval/target.go retrieval/target_test.go retrieval/targetmanager.go retrieval/targetmanager_test.go retrieval/targetpool.go retrieval/targetpool_test.go rules/ast/functions.go rules/rules_test.go storage/metric/interface.go storage/metric/tiered/curator.go storage/metric/tiered/end_to_end_test.go storage/metric/tiered/leveldb.go storage/metric/tiered/memory.go storage/metric/tiered/memory_test.go storage/metric/tiered/tiered.go storage/remote/queue_manager.go templates/templates.go templates/templates_test.go web/api/query.go web/consoles.go web/web.go Change-Id: I96e6312b51e877d4434fe96c494e9558fe2e1d16	10 years ago
Bjoern Rabenstein	14bda4180c	Changes after pair code review. Change-Id: Ib72d40f8e9027818cfbbd32a7a7201eebda07455	10 years ago
Bjoern Rabenstein	bb42cc2e2d	Evict based on memory pressure. Evict recently used chunks last. Change-Id: Ie6168f0cdb3917bdc63b6fe15585dd70c1e42afe	10 years ago
Bjoern Rabenstein	74c143c4c9	Improve scraper shutdown time. - Stop target pools in parallel. - Stop individual scrapers in goroutines, too. - Timing tweaks. Change-Id: I9dff1ee18616694f14b04408eaf1625d0f989696	10 years ago
Bjoern Rabenstein	3f61d304ce	Reorganize maintenance loop. Change-Id: Iac10f988ba3e93ffb188f49c30f92e0b6adce5a3	10 years ago
Bjoern Rabenstein	a5f56639b8	Instrument unwritten samples queue. Change-Id: Id77387314d340a5118490cf08e7bbc37c7366b25	10 years ago
Bjoern Rabenstein	904acd43da	Add crash recovery. Fix the behavior if preload for non-existent series is requested. Instead of returning an error (which triggers a panic further up), simply count those incidents. They can happen regularly, we just want to know if they happen too frequently because that would mean the indexing is behind or broken. Change-Id: I4b2d1b93c4146eeea897d188063cb9574a270f8b	10 years ago
Bjoern Rabenstein	f1de5b0c4e	Run checkpointing of in-memory metrics and head chunks periodically. Checkpointing interval is now a command line flag. Along the way, several things were refactored. - Restructure the way the storage is started and stopped.. - Number of series in checkpoint is now a uint64, not a varint. (Breaks old checkpoints, needs wipe!) - More consistent naming and order of methods. Change-Id: I883d9170c9a608ee716bb0ab3d0ded8ca03760d9	10 years ago
Bjoern Rabenstein	9c3ecc2134	Remove unused flags. Change-Id: Ie1bcbb0743d65e92072628811706d49753023205	10 years ago
Bjoern Rabenstein	b3ed9aa7a2	Clean up start-up and shut-down. Change-Id: Idff4bbb0a15a9f879bfbb3da5b1025179cab5e2c	10 years ago
Bjoern Rabenstein	4447708c9f	Fix a race in target.go. Also, fix problems in shutdown. Starting serving and shutdown still has to be cleaned up properly. It's a mess. Change-Id: I51061db12064e434066446e6fceac32741c4f84c	10 years ago
Bjoern Rabenstein	934d09f738	Fix race during shutdown. Change-Id: I2f8bf48d92a14f1e5ecde27c1b138734d7653394	10 years ago
Bjoern Rabenstein	8fba3302bc	Bold changes to concurrency. (WIP. Probably doesn't work yet.) Change-Id: Id1537dfcca53831a1d428078a5863ece7bdf4875	10 years ago
Julius Volz	a746fbb8bc	Instrument indexing: queue length, batch sizes and latencies. Change-Id: I60bcbd24b160e47d418a485d8cffa39344a257c6	10 years ago
Bjoern Rabenstein	f5f9f3514a	Major code cleanup. - Make it go-vet and golint clean. - Add comments, TODOs, etc. Change-Id: If1392d96f3d5b4cdde597b10c8dff1769fcfabe2	10 years ago
Bjoern Rabenstein	3592dc2359	Implement series eviction. Change-Id: I7a503e0ba78aae3761d032851b06f2807122b085	10 years ago
Bjoern Rabenstein	af77d5ef0b	Added a few missing implementations in index.go. Also, added closing of persistence and mem storage. Change-Id: Iacf0d22c3520dd2584d9546984c1f8a5ed6cd54e	10 years ago
Julius Volz	e7ed39c9a6	Initial experimental snapshot of next-gen storage. Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d	10 years ago
Brian Brazil	5edf689133	Stagger scrapes to spread out load. Change-Id: Ib141b271e4adfb817886871f86051c207b05cf35	10 years ago
Bjoern Rabenstein	8956faeccb	Migrate to new client_golang. This change will only be submitted when the new client_golang has been moved to the new version. Change-Id: Ifceb59333072a08286a8ac910709a8ba2e3a1581	10 years ago
Brian Brazil	3b3ec604c3	Stagger scrapes to spread out load. Change-Id: Ib141b271e4adfb817886871f86051c207b05cf35	10 years ago
Bjoern Rabenstein	2128d9d811	Migrate to new client_golang. This change will only be submitted when the new client_golang has been moved to the new version. Change-Id: Ifceb59333072a08286a8ac910709a8ba2e3a1581	11 years ago
Brian Brazil	e041c0cd46	Add console and alert templates with access to all data. Move rulemanager to it's own package to break cicrular dependency. Make NewTestTieredStorage available to tests, remove duplication. Change-Id: I33b321245a44aa727bfc3614a7c9ae5005b34e03	11 years ago
Julius Volz	01f652cb4c	Separate storage implementation from interfaces. This was initially motivated by wanting to distribute the rule checker tool under `tools/rule_checker`. However, this was not possible without also distributing the LevelDB dynamic libraries because the tool transitively depended on Levigo: rule checker -> query layer -> tiered storage layer -> leveldb This change separates external storage interfaces from the implementation (tiered storage, leveldb storage, memory storage) by putting them into separate packages: - storage/metric: public, implementation-agnostic interfaces - storage/metric/tiered: tiered storage implementation, including memory and LevelDB storage. I initially also considered splitting up the implementation into separate packages for tiered storage, memory storage, and LevelDB storage, but these are currently so intertwined that it would be another major project in itself. The query layers and most other parts of Prometheus now have notion of the storage implementation anymore and just use whatever implementation they get passed in via interfaces. The rule_checker is now a static binary :) Change-Id: I793bbf631a8648ca31790e7e772ecf9c2b92f7a0	11 years ago
Matt T. Proud	2064f32662	Clean up quitting behavior and add quit trigger. The closing of Prometheus now using a sync.Once wrapper to prevent any accidental multiple invocations of it, which could trigger corruption or a race condition. The shutdown process is made more verbose through logging. A not-enabled by default web handler has been provided to trigger a remote shutdown if requested for debugging purposes. Change-Id: If4fee75196bbff1fb1e4a4ef7e1cfa53fef88f2e	11 years ago
Matt T. Proud	81367893fd	Use idiomatic one-to-many one-time signal pattern. The idiomatic pattern for signalling a one-time message to multiple consumers from a single producer is as follows: ``` c := make(chan struct{}) w := new(sync.WaitGroup) // Boilerplate to ensure synchronization. for i := 0; i < 1000; i++ { w.Add(1) go func() { defer w.Done() for { select { case _, ok := <- c: if !ok { return } default: // Do something here. } } }() } close(c) // Signal the one-to-many single-use message. w.Wait() ``` Change-Id: I755f73ba4c70a923afd342a4dea63365bdf2144b	11 years ago
Matt T. Proud	1d01435d4d	Make curation semaphore behavior idiomatic. Idiomatic semaphore usage in Go, unless it is wrapping a concrete type, should use anonymous empty structs (``struct{}``). This has several features that are worthwhile: 1. It conveys that the object in the channel is likely used for resource limiting / semaphore use. This is by idiom. 2. Due to magic under the hood, empty structs have a width of zero, meaning they consume little space. It is presumed that slices, channels, and other values of them can be represented specially with alternative optimizations. Dmitry Vyukov has done investigations into improvements that can be made to the channel design and Go and concludes that there are already nice short circuiting behaviors at work with this type. This is the first change of several that apply this type of change to suitable places. In this one change, we fix a bug in the previous revision, whereby a semaphore can be acquired for curation and never released back for subsequent work: http://goo.gl/70Y2qK. Compare that versus the compaction definition above. On top of that, the use of the semaphore in the mode better supports system shutdown idioms through the closing of channels. Change-Id: Idb4fca310f26b73c9ec690bbdd4136180d14c32d	11 years ago
Julius Volz	817d9b0e97	"go fmt" fixup. Change-Id: I262bb462281bc2610819c822fc7a0768c6ce3d8d	11 years ago
Julius Volz	2279fcbac4	Compact everything to the same sample group size. Change-Id: Ibb4f3a5d76173d64de916ef1eb41ab5d7900c97b	11 years ago

1 2 3

135 Commits (440d5df6e2e2f8771d5df5c7a0dd24471bc30ac0)