Brad Walker
3348930df5
Replace fileutil.ReadDir with ioutil.ReadDir ( #7029 ) ( #7033 )
...
* tsdb: Replace fileutil.ReadDir with ioutil.ReadDir (#7029 )
Signed-off-by: Brad Walker <brad@bradmwalker.com>
* tsdb: Remove fileutil.ReadDir (#7029 )
Signed-off-by: Brad Walker <brad@bradmwalker.com>
5 years ago
Josh Soref
91d76c8023
Spelling ( #6517 )
...
* spelling: alertmanager
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: attributes
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: autocomplete
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: bootstrap
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: caught
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: chunkenc
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: compaction
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: corrupted
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: deletable
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: expected
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: fine-grained
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: initialized
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: iteration
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: javascript
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: multiple
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: number
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: overlapping
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: possible
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: postings
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: procedure
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: programmatic
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: queuing
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: querier
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: repairing
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: received
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: reproducible
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: retention
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: sample
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: segements
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: semantic
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: software [LICENSE]
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: staging
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: timestamp
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: unfortunately
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: uvarint
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: subsequently
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
* spelling: ressamples
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
5 years ago
Tom Wilkie
de0a772b8e
Port tsdb to use pkg/labels. ( #6326 )
...
* Port tsdb to use pkg/labels.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Get tests passing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Remove useless cast.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Appease linters.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
5 years ago
陈谭军
103f26d188
fix the wrong word ( #6069 )
...
Signed-off-by: chentanjun <2799194073@qq.com>
5 years ago
Callum Styan
3344bb5c33
Move WAL watcher code to tsdb/wal package. ( #5999 )
...
* Move WAL watcher code to tsdb/wal package.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Fix tests after moving WAL watcher code.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Lint fixes.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
5 years ago
Bartek Plotka
f0863a604e
Removed extra tsdb/testutil after merge.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
5 years ago
Ganesh Vernekar
5ecef3542d
Cleanup after merging tsdb into prometheus
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
5 years ago
Ganesh Vernekar
7cf09b0395
Moving tsdb into its own subdirectory
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
5 years ago
Chris Marchbanks
b40cc43958
Provide option to compress WAL records ( #609 )
...
In running Prometheus instances, compressing the records was shown to
reduce disk usage by half while incurring a negligible CPU cost.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
6 years ago
Krasi Georgiev
288f67efbf
fix some unchecked errors and remove unused vars. ( #592 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. ( #549 )
...
Testing that createBlock creates blocks that can be opened.
and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.
Many missing .Close() calls were added for fixing failing os.RemoveAll
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
48c439d26d
fix statick check errors ( #475 )
...
fix the tests for `check_license` and `staticcheck`
the static check also found some actual bugs.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
2962202ed3
fix windows tests ( #469 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Ben
a8351dc9d0
Using filepath.Join() instead of strings with slashes ( #428 )
...
fixes: https://github.com/prometheus/tsdb/issues/426
Using `filepath.Join()` instead of strings containing forward slash path delimiters (needed for non-*nix OSes), as suggested by @krasi-georgiev
6 years ago
Chris Marchbanks
2db59a71a6
Fix assert order being backwards in a few places ( #388 )
...
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
6 years ago
Fabian Reinartz
22fd3ef24e
Deal with zero-length segments
...
Signed-off-by: Fabian Reinartz <freinartz@google.com>
6 years ago
Fabian Reinartz
1a5573b4ce
Migrate write ahead log
...
On startup, rewrite the old write ahead log into the new format once.
Signed-off-by: Fabian Reinartz <freinartz@google.com>
6 years ago
Mario Trangoni
c0e888e82b
fix megacheck issues: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.
7 years ago
Krasi Georgiev
eb6cdbcb5b
some test funcitons didn't cleanup after themselves.
7 years ago
Nipun
6537ee832c
Incorporated review comments
7 years ago
Nipun
cadae47522
fixed an issue in TestWALRestoreCorrupted
7 years ago
Fabian Reinartz
adb6be73dd
move label read function int labels pkg
7 years ago
Callum Styan
8326e410d0
decouple tsdb main.go (and tests) from prometheus/pkg
7 years ago
Callum Styan
ca4e817372
move test utils to it's own package so we can use it in tests for other packages in tsdb repo
7 years ago
Callum Styan
6c8bbf3f61
use test utils in wal_test
7 years ago
ranbochen
a27cf34a36
fix bugs on platform windows to pass all test case. ( #192 )
...
* fix bugs on platform windows to pass all test case.
* fix bugs on platform windows to pass all test case
* clean up codes
7 years ago
ranbo chen
19340a382d
clean up codes
7 years ago
ranbo chen
b82b2d0c1b
fix bugs on platform windows to pass all test case.
7 years ago
Fabian Reinartz
80055bb95b
Truncate segments on broken header
7 years ago
Fabian Reinartz
d3682d701c
wal: decode and process in separate threads.
7 years ago
Bas Harenslak
5e1c258a98
Instrument WAL fsync
7 years ago
Fabian Reinartz
1e88ba06b4
Use boolean function instead of postings to drop WAL series
...
There is not guarantee or requirement for WAL writers to only add
series entries in increasing order of IDs. A postings list cannot look
back and thus unordered WAL entries would skip over IDs to not truncate
from the WAL.
We replace it with a simple boolean check function that does not require
order.
7 years ago
Fabian Reinartz
24362567b9
Fix test flakes
7 years ago
Fabian Reinartz
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
7 years ago
Fabian Reinartz
c2916736be
Re-use slices on deocding the WAL. Fix tests.
7 years ago
Fabian Reinartz
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
7 years ago
Fabian Reinartz
970bffec8d
Fix WAL errors and add tests for it
7 years ago
Fabian Reinartz
33e9bdf403
WAL refactoring and truncation fixes and test
7 years ago
Fabian Reinartz
1ddedf2b30
Change series ID from uint32 to uint64
7 years ago
Goutham Veeramachaneni
1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
5cf2662074
Refactor WAL into Head and misc improvements
7 years ago
Fabian Reinartz
215a856f34
Just check for general error in WAL test
...
Go 1.9 changed the returned error. Checking the exact message is too
fragile.
7 years ago
Goutham Veeramachaneni
7438ed7035
Expose Intervals type for use by TombstoneReader.
...
TombstoneReader is exposed but Intervals is not.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Goutham Veeramachaneni
44e9ae38b5
Incorporate PR feedback.
...
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
8 years ago
Goutham Veeramachaneni
bacb143b7e
Simplify tombstone and WAL Delete formats.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
8 years ago
Goutham Veeramachaneni
f29fb62fba
Make TombstoneReader a Getter.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
8 years ago
Goutham Veeramachaneni
3eb4119ab1
Make HeadBlock use WAL.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
8 years ago
Fabian Reinartz
fb5c5535fc
Misc cleanup
8 years ago
Fabian Reinartz
4862b261d0
Abstract WAL into interface
8 years ago
Fabian Reinartz
535532ca02
Export refdSample
...
The type was part of a exported method signatures and should therefore
be exported as well.
8 years ago