github
/
prometheus
mirror of https://github.com/prometheus/prometheus
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
204 Commits
232 Branches
442 Tags
620 MiB
Commit Graph

51 Commits (3910b913bd0ddd587cb92c28a515121e6ec0ea1d)

Author SHA1 Message Date
Fabian Reinartz 2841499212 Simplify and split up iterators 
This simplifies some of the iterators by loading chunks from the
ChunkReader earlier, filtering of chunks vs filtering or series is
split into separate iterators for easier testing
 2017-03-07 11:30:35 +01:00
Fabian Reinartz 87805fb83f Remove Partitioned* code 2017-03-06 17:34:49 +01:00
Fabian Reinartz 55a9b5428a Add separate head mutex 
Introduce a seperate mutex for the head blocks to avoid a race where
a post-compaction reload may run between switching the DB's base mutex
to create a new head block in an appender.
 2017-03-04 17:41:01 +01:00
Fabian Reinartz 78780cd2ba Segment chunk file 
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
 2017-02-23 10:50:22 +01:00
Fabian Reinartz a3d042b54e Support multiple chunk files in read path 2017-02-18 17:33:20 +01:00
Fabian Reinartz c32a94d409 Unexport HeadBlock, export Block interface 2017-01-10 15:41:57 +01:00
Fabian Reinartz 4c4e0c614e Simplify position mapper updating 2017-01-09 19:24:05 +01:00
Fabian Reinartz 8c31c6e934 Make concurrent head chunk reads safe, fix misc races 
This adds a 4 sample buffer to every head chunk. The XOR
compression scheme may edit bytes in place. The minimum size
of a sample is 2 bits. So keeping the last 4 samples in an in-memory
buffer makes it safe to query the preceeding ones while samples
are added
 2017-01-09 16:51:39 +01:00
Fabian Reinartz 63e12807da Don't update head postings mapper on every append 2017-01-06 16:43:18 +01:00
Fabian Reinartz 96c2bd249f Handle compaction trigger and reinitializing in DB 2017-01-06 13:03:23 +01:00
Fabian Reinartz 3ed2c2a14b Rename Partition to regular DB, DB to PartitionedDB 2017-01-06 11:40:09 +01:00
Fabian Reinartz 4590b61343 Rename shard to partition 2017-01-06 08:08:02 +01:00
Fabian Reinartz 9790aa98ac Add postings wrapper that emits head postings in label set order 
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.

Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
 2017-01-05 16:05:42 +01:00
Fabian Reinartz 40cf215fba Hotfix missing sorting of head block series index 2017-01-04 09:47:20 +01:00
Fabian Reinartz ec99f99d3d Fix and test bug in shardSeriesSet 2017-01-03 19:02:42 +01:00
Fabian Reinartz 62b8ded0a5 Rename iterator value getters to At() 2017-01-02 13:33:06 +01:00
Fabian Reinartz 5c45a1cc6f Remove shardSeriesSet caching 2017-01-02 12:05:52 +01:00
Fabian Reinartz beb842a856 Change block interface 
This changes the block interface to directly expose index
and series readers rather than the direct querier
 2017-01-02 11:12:28 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction 
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
 2016-12-31 16:24:04 +01:00
Fabian Reinartz a009247ab7 Add matching of empty label 2016-12-30 19:36:28 +01:00
Fabian Reinartz 3a5468f251 Lock shards during querying and shutdown 2016-12-28 11:41:44 +01:00
Fabian Reinartz 286293802b Add merge postings 2016-12-28 11:02:19 +01:00
Fabian Reinartz dd72b52098 Add postings intersection 2016-12-27 11:32:10 +01:00
Fabian Reinartz f8111cef0e Fix chunk series iterator seeking 2016-12-26 16:55:44 +01:00
Fabian Reinartz 5efe1d178e labels: add Compare and String methods 2016-12-24 13:49:35 +01:00
Fabian Reinartz 869cccf080 Test and fixes for buffered iterator 2016-12-21 16:06:33 +01:00
Fabian Reinartz 0a94f58f1a Fix test import of labels, simplify constructor names 2016-12-21 15:12:26 +01:00
Fabian Reinartz da2beb3e6d Fix zero division, add buffer series iterator 2016-12-21 13:04:51 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz cddc29fa17 Fix labels comparison, fetch correct labels 2016-12-20 14:54:52 +01:00
Fabian Reinartz ce7f4106c2 Reda correct label number, fix buffered iterator panic 2016-12-20 14:21:50 +01:00
Fabian Reinartz d9ca4b47f5 Fix offset errors, fix persisted postings order 2016-12-20 13:14:55 +01:00
Fabian Reinartz 282d9ae6e2 Implement label value queries in all layers. 2016-12-19 12:26:25 +01:00
Fabian Reinartz aabb21f4b9 Add shard series set test 2016-12-19 11:44:11 +01:00
Fabian Reinartz b08f82fa4e Pre-select relevant chunks on series access. 
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.

This is especially relevant for series that come and go often and larger
files.
 2016-12-16 12:13:17 +01:00
Fabian Reinartz bd77103a49 Add stats serialization, load querier of all blocks 2016-12-15 16:14:33 +01:00
Fabian Reinartz 1a35e54450 Add chained iterator, skipping seek added 2016-12-15 15:23:15 +01:00
Fabian Reinartz 5424a0cf75 Rename SeriesShard to Shard 2016-12-15 08:36:09 +01:00
Fabian Reinartz d56b281006 Rename Iterator to Postings 2016-12-15 07:05:29 +01:00
Fabian Reinartz c1acd3fe85 Advance buffered iterator correctly on seek 2016-12-14 21:43:27 +01:00
Fabian Reinartz e561c91d53 Implement proper buffered iterator 
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.

It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
 2016-12-14 21:14:44 +01:00
Fabian Reinartz ca89080128 Misc fixes for initial Prometheus integration 2016-12-14 18:38:46 +01:00
Fabian Reinartz 725385ea05 Fix compareLabels, add test 2016-12-14 15:47:05 +01:00
Fabian Reinartz fc992fafc2 Change querier interface, initial implementations 2016-12-14 15:39:23 +01:00
Fabian Reinartz 6eeb0ef01c Add queriers and partial mocks 2016-12-13 15:26:58 +01:00
Fabian Reinartz 9b400b4c58 Add chunk based series iterator 2016-12-12 19:12:55 +01:00
Fabian Reinartz b334c3ade8 Write chunk skiplist and add series reader 2016-12-12 15:39:55 +01:00
Fabian Reinartz 14dbc59f2b cleanup and switching removal of unsafe calls. 2016-12-10 18:09:57 +01:00
Fabian Reinartz 6f93a699e6 wipe 2016-12-02 17:49:05 +01:00
Fabian Reinartz 3ad4db56aa Remove Prometheus internal metric type from interface 2016-11-15 11:43:34 +01:00