Fabian Reinartz
7efb830d70
wal: parallelize sample processing
7 years ago
Fabian Reinartz
d3682d701c
wal: decode and process in separate threads.
7 years ago
Fabian Reinartz
cd2e26b7fc
Load postings in batch on startup
...
This allows to insert IDs to postings out of order until
a trigger function is called. This avoids the insertion sort we usually
do which can be very costly since WAL entries are more out of order than
regular adds.
7 years ago
Goutham Veeramachaneni
4a7c39d9d8
Merge pull request #160 from dim/fix/snapshot-test
...
Restore snapshot functionality
7 years ago
Fabian Reinartz
27f1b8aac3
Merge pull request #162 from BasPH/fsync-duration
...
Instrument WAL fsync
7 years ago
Bas Harenslak
5e1c258a98
Instrument WAL fsync
7 years ago
Goutham Veeramachaneni
da565f975e
Merge pull request #161 from prometheus/fileutil
...
Remove dependency on etcd/pkg/fileutil
7 years ago
Goutham Veeramachaneni
203012169a
snapshot: Remove truncation check to restore func.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
f04ec031eb
compact: sync temporary directory
7 years ago
Fabian Reinartz
665d1fd451
Merge pull request #158 from prometheus/cachesym
...
Allocate and cache strings for persisted blocks
7 years ago
Fabian Reinartz
bbe72dccb9
Remove dependency on etcd/pkg/fileutil
7 years ago
Dimitrij Denissenko
c9fc2af6c0
Add test for snapshot
7 years ago
Goutham Veeramachaneni
3b7e71fee9
Merge pull request #159 from dim/fix/wal-flush
...
Use configurable WAL flush interval
7 years ago
Dimitrij Denissenko
83c984d9a3
Use configurable WAL flush inteval
7 years ago
Fabian Reinartz
e19b648377
Merge pull request #155 from Gouthamve/level-log
...
Add levels to all log lines.
7 years ago
Fabian Reinartz
78df406dac
Allocate and cache strings for persisted blocks
...
This change loads the full symbol table when we open a persisted block
and allocates a string for each. This ensures that strings retrieved
through the index can be used after the block was closed.
Before we backed the strings by the mmap'd byte regions which would
segfault in this case.
Also remove an inconsistency in the disk format and move both offset
tables to the end (breaking change).
7 years ago
Goutham Veeramachaneni
c521ac495f
Merge pull request #152 from criteo-forks/master
...
Add db.Head() method
7 years ago
Goutham Veeramachaneni
c35d3a65bd
Add levels to all log lines.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Thibault Chataigner
0ff8f91ec8
Add db.Head() method
...
Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
7 years ago
Goutham Veeramachaneni
87c01dd5fb
Merge pull request #134 from criteo-forks/master
...
Expose `Blocks()` on DB
7 years ago
Fabian Reinartz
69f105f4f9
Merge pull request #151 from prometheus/waltrunc
...
Use boolean function instead of postings to drop WAL series
7 years ago
Fabian Reinartz
1e88ba06b4
Use boolean function instead of postings to drop WAL series
...
There is not guarantee or requirement for WAL writers to only add
series entries in increasing order of IDs. A postings list cannot look
back and thus unordered WAL entries would skip over IDs to not truncate
from the WAL.
We replace it with a simple boolean check function that does not require
order.
7 years ago
Thibault Chataigner
5ca18837e9
Make DB blocks list public.
...
This enable computing the first timestamp in the DB
only accessing public interfaces.
Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
7 years ago
Fabian Reinartz
5fa1c993b9
Merge pull request #150 from prometheus/postingssort
...
Ensure postings are always sorted
7 years ago
Fabian Reinartz
6ee254e353
Ensure postings are always sorted
...
IDs for new series are handed out before the postings are locked. Thus
series are not indexed in order of their IDs, which could result in only
partially sorted postings list.
Iterating over those silently skipped elements as the sort invariant was
violated.
7 years ago
Goutham Veeramachaneni
f39388c9af
Merge pull request #147 from prometheus/deadlock
...
Simplify series create logc in head
7 years ago
Fabian Reinartz
162a48e4f2
Create series with ID recorded in WAL when reading it back
7 years ago
Fabian Reinartz
7ada9cd805
Simplify series create logic in head
7 years ago
Goutham Veeramachaneni
2187388292
Merge pull request #146 from prometheus/deadlock
...
Add missing unlock on early return
7 years ago
Fabian Reinartz
ab8d9b9706
Add missing unlock on early return
7 years ago
Fabian Reinartz
99d39174f6
Merge branch 'master' of github.com:prometheus/tsdb
7 years ago
Fabian Reinartz
8214dc82a7
Remove infinite block in benchmark
7 years ago
Fabian Reinartz
643563068b
Merge pull request #144 from Gouthamve/misc-
...
Expose NewIndexReader() and cleanups
7 years ago
Goutham Veeramachaneni
8919baef03
Expose NewIndexReader() and cleanups
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
e45bb1d328
Merge pull request #142 from krasi-georgiev/107-swap-cobra-with-kingpin
...
replaced cobra with kingpin for the tsdb cli tool
7 years ago
Fabian Reinartz
3870ec285c
Merge pull request #140 from prometheus/locks
...
Fix various races
7 years ago
Fabian Reinartz
3d8be398d6
Merge branch 'locks' of github.com:prometheus/tsdb into locks
7 years ago
Fabian Reinartz
24362567b9
Fix test flakes
7 years ago
Fabian Reinartz
30d29b889c
Merge pull request #141 from prometheus/dectmpstr
...
Fixup allocation regression during compaction
7 years ago
Fabian Reinartz
bfe75cec35
Merge pull request #138 from Gouthamve/chunk-compress
...
Compress the series chunk details in index.
7 years ago
Krasi Georgiev
92d0414993
replaced cobra with kingpin
...
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
7 years ago
Fabian Reinartz
b09d90c79c
Add decoding method to retrieve unsafe strings
...
When decoding data from mmaped blocks, we would like to retrieve
a string backed by the mmaped region. As the underlying byte slice
never changes, this is safe.
7 years ago
Fabian Reinartz
f904cd385f
Do not build a superflous 'all' postings
7 years ago
Fabian Reinartz
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
7 years ago
Goutham Veeramachaneni
afaf12fe45
Compress the series chunk details in index.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
1d5f85817d
Fix various races
7 years ago
Fabian Reinartz
c2916736be
Re-use slices on deocding the WAL. Fix tests.
7 years ago
Fabian Reinartz
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
7 years ago
Fabian Reinartz
970bffec8d
Fix WAL errors and add tests for it
7 years ago
Fabian Reinartz
81222849bc
Filter WAL data in Head, misc fixes
7 years ago