Fabian Reinartz
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
7 years ago
Fabian Reinartz
1d5f85817d
Fix various races
7 years ago
Fabian Reinartz
c2916736be
Re-use slices on deocding the WAL. Fix tests.
7 years ago
Fabian Reinartz
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
7 years ago
Fabian Reinartz
970bffec8d
Fix WAL errors and add tests for it
7 years ago
Fabian Reinartz
81222849bc
Filter WAL data in Head, misc fixes
7 years ago
Fabian Reinartz
33e9bdf403
WAL refactoring and truncation fixes and test
7 years ago
Fabian Reinartz
c36d574290
Replace single head lock with granular locks
...
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
7 years ago
Fabian Reinartz
1ddedf2b30
Change series ID from uint32 to uint64
7 years ago
Fabian Reinartz
af2c2f9674
Re-enable head block delet test
7 years ago
Goutham Veeramachaneni
1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
9f41d9fd3c
Instrument chunks on level 1 compactions
7 years ago
Fabian Reinartz
893b6ec506
Add tests for GC and chunk truncation
7 years ago
Fabian Reinartz
4f037da462
Remove defer statement in hot path
7 years ago
Fabian Reinartz
4cc37eecab
Refactor and add tests for compactor
7 years ago
Fabian Reinartz
5cf2662074
Refactor WAL into Head and misc improvements
7 years ago
Fabian Reinartz
8209e3ec23
Add various metrics
7 years ago
Fabian Reinartz
3901b6e70b
Remove multiple heads
...
This changes the structure to a single WAL backed by a single head
block.
Parts of the head block can be compacted. This relieves us from any head
amangement and greatly simplifies any consistency and isolation concerns
by just having a single head.
7 years ago
Fabian Reinartz
0fe67df9f2
Merge pull request #135 from alin-amana/fix_eval_blocking_on_fsync
...
Don't do blocking disk I/O under mutex blocking scrape and eval
7 years ago
Alin Sinpalean
8f7934d025
Take the fdatasync() syscall out from under mutex lock to (mostly) avoid scrape and eval from blocking on disk I/O.
7 years ago
Fabian Reinartz
215a856f34
Just check for general error in WAL test
...
Go 1.9 changed the returned error. Checking the exact message is too
fragile.
7 years ago
Fabian Reinartz
1b86bbb48a
Merge pull request #130 from mdlayher/crc32-once
...
Add newCRC32 function to simplify hash initialization
7 years ago
Matt Layher
78b15c3434
Add newCRC32 function to simplify hash initialization
7 years ago
Goutham Veeramachaneni
e2e2947ee8
Merge pull request #128 from Gouthamve/expose-types
...
Expose types for easier interface implementations.
7 years ago
Goutham Veeramachaneni
c463b0c8c8
Expose NewMergedSeriesSet for merging SeriesSets
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Goutham Veeramachaneni
5b242f35ba
Expose a Querier with manually passed in readers.
...
Allows people to not copy the querying code.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Goutham Veeramachaneni
7438ed7035
Expose Intervals type for use by TombstoneReader.
...
TombstoneReader is exposed but Intervals is not.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Goutham Veeramachaneni
79dc0ed420
Merge pull request #126 from ankon/pr/issue-125
...
Log the directory when the WAL cannot be opened
7 years ago
Andreas Kohn
4003da0fdd
Log the directory when the WAL cannot be opened
...
Fixes #125
7 years ago
Fabian Reinartz
c4ca881685
Merge pull request #120 from Gouthamve/fix-0-division
...
Fix panic caused by 0 division
7 years ago
Goutham Veeramachaneni
178f840295
Fix panic caused by 0 division
...
Introduced in #108
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
e0aca4bee9
Merge pull request #119 from prometheus/allocsave
...
Allocsave
7 years ago
Fabian Reinartz
f9f439c6db
Merge pull request #118 from prometheus/allocsave2
...
Avoid chunk allocations and refactor compactions
7 years ago
Fabian Reinartz
912302877b
Merge pull request #116 from prometheus/allocsave
...
Persist series without allocating the full set
7 years ago
Fabian Reinartz
905af27cf9
Refactor compactor
7 years ago
Fabian Reinartz
66ff7b12e9
Pool Chunk objects during compaction
7 years ago
Fabian Reinartz
2644c8665c
Don't allocate ChunkMetas, reuse postings slices
7 years ago
Fabian Reinartz
96d7f540d4
Persist series without allocating the full set
...
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.
As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
7 years ago
Fabian Reinartz
1875d05e00
Merge pull request #115 from prometheus/compactionfix
...
Fix compactions
7 years ago
Fabian Reinartz
3951d8cc29
Fix selection of compactable blocks
7 years ago
Fabian Reinartz
fa04455cd7
Properly detect compactable heads
...
The fullness of the wrong head was being tracked, causing wrong
lists of compactable heads.
7 years ago
Julius Volz
15baaa5677
Merge pull request #112 from joehandzik/patch-3
...
Fix typo in index.md
7 years ago
Joe Handzik
e6cb86c029
Fix typo in index.md
...
Minor change, monotinically -> monotonically.
Signed-Off-By: Joe Handzik <joseph.t.handzik@hpe.com>
7 years ago
Goutham Veeramachaneni
1373a0ea9f
Merge pull request #111 from dilyevsky/add-prefix-matcher
...
Add prefix label matcher.
7 years ago
Dmitry Ilyevsky
37194b7a30
Add prefix label matcher.
...
Implement labels.PrefixMatcher and use interface conversion in querier
to optimize label tuples search.
[unit-tests]: Fix bug and populate label index for mock index.
Signed-off-by: Dmitry Ilyevsky <ilyevsky@gmail.com>
7 years ago
Goutham Veeramachaneni
1378338aab
Merge pull request #110 from WJTian/master
...
Fix DefaultOptions
7 years ago
tianwenjie01
b64054365d
Fix DefaultOptions
7 years ago
Fabian Reinartz
8d3a398113
Merge pull request #109 from prometheus/fixes
...
Fix compactions
7 years ago
Fabian Reinartz
74f67e8271
Fix compacting disable/enable
...
Enabling and disabling compaction no longer blocks are potentially
causes panics.
7 years ago
Fabian Reinartz
3065be97d8
Fix and document locking order for DB
7 years ago