Commit Graph

14671 Commits (af2a1cb10c89de496cd4309ac532624b34112d74)

Author SHA1 Message Date
juliusv 1958063cc7 Merge pull request #262 from prometheus/feature/storage/fix-grouping
Spawn grouping of fingerprints with free semaphore.
2013-05-21 07:17:51 -07:00
Matt T. Proud 74a66fd938 Spawn grouping of fingerprints with free semaphore.
The previous implementation spawned N goroutines to group samples
together and would not start work until the semaphore unblocked.
While this didn't leak, it polluted the scheduling space.  Thusly,
the routine only starts after a semaphore has been acquired.
2013-05-21 16:11:35 +02:00
Matt T. Proud 053f4296d0 Merge pull request #260 from prometheus/feature/show-scrape-errors
Remember and display last scrape errors in web UI.
2013-05-21 06:40:39 -07:00
Julius Volz 081191afb8 Remember and display last scrape errors in web UI. 2013-05-21 15:31:27 +02:00
Matt T. Proud 32c3510e66 Merge pull request #259 from prometheus/optimize/fingerprinting-time
Optimize fingerprinting time and repointerize fingerprint usage
2013-05-21 05:44:33 -07:00
Julius Volz 5b105c77fc Repointerize fingerprints. 2013-05-21 14:28:14 +02:00
Julius Volz 259a0ea467 Optimize fingerprint comparison time. 2013-05-21 14:00:52 +02:00
Matt T. Proud da0257bb0e Merge pull request #258 from prometheus/feature/fingerprint-indexing
Benchmark memory arena; simplify map generation.
2013-05-21 00:50:36 -07:00
Matt T. Proud ec5b5bae28 Fuck you, Travis. 2013-05-21 09:42:00 +02:00
Matt T. Proud e5ac91222b Benchmark memory arena; simplify map generation.
The one-off keys have been replaced with ``model.LabelPair``, which is
indexable.  The performance impact is negligible, but it represents
a cognitive simplification.
2013-05-21 09:39:12 +02:00
juliusv 360477f66c Merge pull request #257 from prometheus/feature/better-memory-behaviors
Pointerize memorySeriesArena.
2013-05-16 07:36:40 -07:00
Matt T. Proud e1f20de2e9 Pointerize memorySeriesArena. 2013-05-16 17:09:28 +03:00
juliusv e5d077e146 Merge pull request #254 from prometheus/feature/refactor/serialization
Several Refactorings: Simplified Encoder / Reduction of Naked Return Values
2013-05-16 07:01:27 -07:00
Matt T. Proud 8f4c7ece92 Destroy naked returns in half of corpus.
The use of naked return values is frowned upon.  This is the first
of two bulk updates to remove them.
2013-05-16 10:53:25 +03:00
Matt T. Proud 4e0c932a4f Simplify Encoder's encoding signature.
The reality is that if we ever try to encode a Protocol Buffer and it
fails, it's likely that such an error is ultimately not a runtime error
and should be fixed forthwith.  Thusly, we should rename
``Encoder.Encode`` to ``Encoder.MustEncode`` and drop the error return
value.
2013-05-16 00:54:18 +03:00
juliusv e54edb16a5 Merge pull request #252 from prometheus/feature/larger-in-memory-arena
Expand the in-memory arena size.
2013-05-14 09:00:56 -07:00
Matt T. Proud b063338ae6 Expand the in-memory arena size.
We need to exercise this code path sooner versus later.  This will
be a cheap way of doing so.
2013-05-14 17:59:52 +02:00
juliusv 516101f015 Merge pull request #250 from prometheus/refactor/drop-unused-storage-setting
Drop unused writeMemoryInterval
2013-05-14 08:45:59 -07:00
juliusv 9ff00b651d Merge pull request #251 from prometheus/fix/memory-metric-mutability
Fix GetMetricForFingerprint() metric mutability.
2013-05-14 08:12:45 -07:00
Bernerd Schaefer 63d9988b9c Drop unused writeMemoryInterval 2013-05-14 17:03:03 +02:00
Bernerd Schaefer aa96c7d141 Fix rules_test.go
This is smelly, but for now we copy a helper method from the metric
tests into rules.
2013-05-14 16:55:18 +02:00
Julius Volz 83c60ad43a Fix GetMetricForFingerprint() metric mutability.
Some users of GetMetricForFingerprint() end up modifying the returned metric
labelset. Since the memory storage's implementation of
GetMetricForFingerprint() returned a pointer to the metric (and maps are
reference types anyways), the external mutation propagated back into the memory
storage.

The fix is to make a copy of the metric before returning it.
2013-05-14 16:46:30 +02:00
Matt T. Proud dab017e8cf Merge pull request #249 from prometheus/fix/test-flag-pollution
Rename test helper files to helpers_test.go
2013-05-14 07:33:03 -07:00
Bernerd Schaefer 428d91c86f Rename test helper files to helpers_test.go
This ensures that these files are properly included only in testing.
2013-05-14 16:30:47 +02:00
juliusv 98e512d755 Merge pull request #246 from prometheus/fix/interval-value-extraction
Fix and optimize getValuesAtIntervalOp data extraction.
2013-05-14 05:55:22 -07:00
Matt T. Proud 23a85654c7 Merge pull request #247 from prometheus/feature/database-statistics-exposition
Include forgotten databases.html.
2013-05-14 05:54:32 -07:00
Matt T. Proud 1a95406b81 Include forgotten databases.html. 2013-05-14 14:50:54 +02:00
Matt T. Proud 9a8fb58fa6 Merge pull request #245 from prometheus/feature/runtime/go1.1
¡ DONOTMERGE — Update to go1.1. !
2013-05-14 05:37:18 -07:00
Julius Volz 71a3172abb Fix and optimize getValuesAtIntervalOp data extraction.
- only the data extracted in the last loop iteration of ExtractSamples() was
  emitted as output
- if e.g. op interval < sample interval, there were situations where the same
  sample was added multiple times to the output
2013-05-14 13:55:17 +02:00
Matt T. Proud 8956ebe0c6 Include "Travis Helper". 2013-05-14 13:39:52 +02:00
Matt T. Proud 244a4a9cdb Update to go1.1.
This commit updates the documentation, Makefiles, formatting, and
code semantics to support the 1.1. runtime, which includes ...

1. ``make advice``,

2. ``make format``, and

3. ``go fix`` on various targets.
2013-05-14 12:39:08 +02:00
Matt T. Proud 2ea2e89c97 Merge pull request #244 from prometheus/feature/database-statistics-exposition
Simplify compaction and expose database sizes.
2013-05-14 03:31:15 -07:00
Matt T. Proud b224251981 Simplify compaction and expose database sizes.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals. Secondarily it
introduces the database size estimation mechanisms.

Include database health and help interfaces.

Add database statistics; remove status goroutines.

This commit kills the use of Go routines to expose status throughout
the web components of Prometheus. It also dumps raw LevelDB status
on a separate /databases endpoint.
2013-05-14 12:29:53 +02:00
juliusv 92ad65ff13 Merge pull request #232 from prometheus/optimize/granular-storage-locking
Synchronous memory appends and more fine-grained storage locks.
2013-05-13 10:11:57 -07:00
Matt T. Proud dad20c15e7 Merge pull request #243 from prometheus/feature/size-exposition
Simplify compaction and expose database sizes.
2013-05-13 04:19:28 -07:00
Matt T. Proud 1f7f89b4e3 Simplify compaction and expose database sizes.
This commit simplifies the way that compactions across a database's
keyspace occur due to reading the LevelDB internals.  Secondarily it
introduces the database size estimation mechanisms.
2013-05-13 13:15:35 +02:00
Matt T. Proud 5b183f6434 Merge pull request #242 from prometheus/feature/storage/long-tail-deletion
Include long-tail data deletion mechanism.
2013-05-13 02:05:44 -07:00
Matt T. Proud d538b0382f Include long-tail data deletion mechanism.
This commit introduces the long-tail deletion mechanism, which will
automatically cull old sample values.  It is an acceptable
hold-over until we get a resampling pipeline implemented.

Kill legacy OS X documentation, too.
2013-05-13 10:54:36 +02:00
Julius Volz ce1ee444f1 Synchronous memory appends and more fine-grained storage locks.
This does two things:

1) Make TieredStorage.AppendSamples() write directly to memory instead of
   buffering to a channel first. This is needed in cases where a rule might
   immediately need the data generated by a previous rule.

2) Replace the single storage mutex by two new ones:
   - memoryMutex - needs to be locked at any time that two concurrent
                   goroutines could be accessing (via read or write) the
                   TieredStorage memoryArena.
   - memoryDeleteMutex - used to prevent any deletion of samples from
                         memoryArena as long as renderView is running and
                         assembling data from it.
   The LevelDB disk storage does not need to be protected by a mutex when
   rendering a view since renderView works off a LevelDB snapshot.

The rationale against adding memoryMutex directly to the memory storage: taking
a mutex does come with a small inherent time cost, and taking it is only
required in few places. In fact, no locking is required for the memory storage
instance which is part of a view (and not the TieredStorage).
2013-05-10 17:15:52 +02:00
Matt T. Proud 2a047fe416 Merge pull request #236 from prometheus/feature/tools/compactor
Expose interfaces for pruner and make pruner tool.
2013-05-10 08:09:04 -07:00
Matt T. Proud fa6a1f97d0 Expose interfaces for pruner and make pruner tool.
In order to run database cleanups and diagnostics, we should have
a means for pruning a database---even if LevelDB does this for us.
2013-05-10 17:07:03 +02:00
juliusv 76521c3ff0 Merge pull request #234 from prometheus/feature/storage/long-tail-deletion
Include deletion processor for long-tail values.
2013-05-10 03:26:13 -07:00
Matt T. Proud 161c8fbf9b Include deletion processor for long-tail values.
This commit extracts the model.Values truncation behavior into the actual
tiered storage, which uses it and behaves in a peculiar way—notably the
retention of previous elements if the chunk were to ever go empty.  This is
done to enable interpolation between sparse sample values in the evaluation
cycle.  Nothing necessarily new here—just an extraction.

Now, the model.Values TruncateBefore functionality would do what a user
would expect without any surprises, which is required for the
DeletionProcessor, which may decide to split a large chunk in two if it
determines that the chunk contains the cut-off time.
2013-05-10 12:19:12 +02:00
juliusv 8d062ebb27 Merge pull request #219 from prometheus/refactor/version-protocol-buffer
Include generated Protocol Buffers artifacts.
2013-05-09 12:29:42 -07:00
juliusv 0dc15d28cb Merge pull request #233 from prometheus/feature/count-by
Implement a COUNT ... BY aggregation operator.
2013-05-09 12:26:31 -07:00
Matt T. Proud 38d839d810 Include generated Protocol Buffers artifacts.
This commit reduces the general compile time dependencies to omit
the Protocol Buffer compiler and the Go Protocol Buffer generator
tool.  The build steps to furnish them still remain, but they can
optionally be called if data.proto or config.proto are under work.
2013-05-08 21:16:39 +02:00
Julius Volz 0877680761 Implement a COUNT ... BY aggregation operator.
This also removes the now obsolete scalar count() function and corrects the
expressions test naming (broken in
2202cd71c9 (L6R59))
so that the expression tests will actually run.
2013-05-08 16:35:16 +02:00
Bernerd Schaefer 1c9b5b6b8d Merge pull request #221 from prometheus/fix/race-conditions
Fix/document race conditions
2013-05-07 09:20:16 -07:00
Bernerd Schaefer a07b89d411 Document go-1.1 race detector with prometheus 2013-05-07 18:15:17 +02:00
Bernerd Schaefer cdde766f39 Embed mutex on web status handler 2013-05-07 18:15:17 +02:00