yuxiaobo
7850f1b35c
new world spelling mistake
...
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
5 years ago
陈谭军
103f26d188
fix the wrong word ( #6069 )
...
Signed-off-by: chentanjun <2799194073@qq.com>
5 years ago
Callum Styan
3344bb5c33
Move WAL watcher code to tsdb/wal package. ( #5999 )
...
* Move WAL watcher code to tsdb/wal package.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Fix tests after moving WAL watcher code.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
* Lint fixes.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
5 years ago
Bartek Plotka
f0863a604e
Removed extra tsdb/testutil after merge.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
5 years ago
Ganesh Vernekar
5ecef3542d
Cleanup after merging tsdb into prometheus
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
5 years ago
Ganesh Vernekar
7cf09b0395
Moving tsdb into its own subdirectory
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
5 years ago
Krasi Georgiev
6f9bbc7253
Open db in Read only mode ( #588 )
...
* Added db read only open mode and use it for the tsdb cli.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
5 years ago
naivewong
b05ca5c36e
fix two benchmarks ( #631 )
...
* fix two benchmarks
Signed-off-by: naivewong <867245430@qq.com>
6 years ago
Krasi Georgiev
882162d5b9
fix the "failed compaction" metric. ( #613 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
288f67efbf
fix some unchecked errors and remove unused vars. ( #592 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
naivewong
7ab060c864
selectOverlappingDirs selects wrong dirs where there are 2 disjoint sets of overlapping dirs
...
Signed-off-by: naivewong <867245430@qq.com>
6 years ago
Krasi Georgiev
c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. ( #549 )
...
Testing that createBlock creates blocks that can be opened.
and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.
Many missing .Close() calls were added for fixing failing os.RemoveAll
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Alec
0861a9b14f
[bugfix] update 'last' variable in chunks.MergeOverlappingChunks() ( #539 )
...
* update
Signed-off-by: naivewong <867245430@qq.com>
* update comment
Signed-off-by: naivewong <867245430@qq.com>
* Update failing test.
Signed-off-by: naivewong <867245430@qq.com>
* Update comment
Signed-off-by: naivewong <867245430@qq.com>
* Add changelog entry.
Signed-off-by: naivewong <867245430@qq.com>
* Update CHANGELOG.md
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
6 years ago
Ganesh Vernekar
c59ed492b2
Vertical query merging and compaction ( #370 )
...
* Vertical series iterator
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Select overlapped blocks first in compactor Plan()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Added vertical compaction
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Code cleanup and comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix tests
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Add benchmark for compaction
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Perform vertical compaction only when blocks are overlapping.
Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Benchmark for vertical compaction
* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Benchmark for query iterator and seek for non overlapping blocks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Vertical query merge only for overlapping blocks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Simplify logging in Compact(...)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Updated CHANGELOG.md
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Calculate overlapping inside populateBlock
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* MinTime and MaxTime for BlockReader.
Using this to find overlapping blocks in populateBlock()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Sort blocks w.r.t. MinTime in reload()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Log about overlapping in LeveledCompactor.write() instead of returning bool
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Log about overlapping inside LeveledCompactor.populateBlock()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Refactor createBlock to take optional []Series
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* review1
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
* Updated CHANGELOG and minor nits
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* nits
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Updated CHANGELOG
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Refactor iterator and seek benchmarks for Querier.
Also has as overlapping blocks.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Additional test case
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Split genSeries into genSeries and populateSeries
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Check error in benchmark
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Warn about overlapping blocks in reload()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
6 years ago
Krasi Georgiev
bf79c767f0
new line
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
07df4fd383
nits
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
0f8f5027ef
remove nested for if
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
da9da9fbee
fix the sleep logic
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
457534d5c4
simplify nesting.
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
752ab86e4e
change the test block series for more stable tests
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
6c34eb8b63
nits
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
1603222bbc
small refactor of openTestDB to handle errors properly.
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
b521559c3b
do a proper cleanup for a failed reload after a compaction
...
a failed reload immediately after a compaction should delete the
resulting block to avoid creating blocks with the same time range.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
fa1c00f9e2
no backgorund
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
1b0d85bbf2
use a db copy instead of creating it again.
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
8ffd705346
fix the misleading log during compaction cancelation
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
9638c13ec8
test compares normal vs canceled compaction times
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
9874377ead
nits
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
921f82cfc1
WIP
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
a2779cc901
fix flaky tests: TestDisableAutoCompactions,TestBlockRanges ( #472 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
db127a60e0
changelog
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
fced260a24
test
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
bd5ccee5c1
use context to cancel compactions
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
bac9cbed2e
no overlapping on compaction when an existing block is not within default boundaries. ( #461 )
...
closes https://github.com/prometheus/prometheus/issues/4643
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
b75d702ceb
fix flaky compaction test ( #458 )
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Krasi Georgiev
7f00217d77
Allow manual compaction for tests when compaction is disabled globally. ( #412 )
...
for tests we need to control when a compaction happens so with this
change automated compaction can be disabled, but allow to run it
manually it tests.
fixes failing tests in : https://github.com/prometheus/tsdb/pull/374
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Bartek Płotka
047b1b1357
compact: Verify for chunks outside of compacted time range. Added unit test for populateBlocs. ( #349 )
...
* compact: Verify for chunks outside of compacted time range.
Unit test for populateBlocs.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>
6 years ago
Bartek Plotka
2aceac0a12
Fixed some comments.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
7 years ago
Bartek Plotka
328c0ff5b0
Simplified the flow and tests.
...
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
7 years ago
Bartek Plotka
483da43660
compact: Exclude last block with shortest range instead of newest one by ULID.
...
Fixes https://github.com/prometheus/prometheus/issues/3943 issue.
Added tests.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
7 years ago
Krasi Georgiev
eb6cdbcb5b
some test funcitons didn't cleanup after themselves.
7 years ago
Callum Styan
ca4e817372
move test utils to it's own package so we can use it in tests for other packages in tsdb repo
7 years ago
Callum Styan
fa26e2b6c3
use test utils in compact_test
7 years ago
Goutham Veeramachaneni
1fc94a02d1
Don't retry failed compactions.
...
Fixes prometheus/prometheus#3487
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
4cc37eecab
Refactor and add tests for compactor
7 years ago
Goutham Veeramachaneni
178f840295
Fix panic caused by 0 division
...
Introduced in #108
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago
Fabian Reinartz
905af27cf9
Refactor compactor
7 years ago
Fabian Reinartz
3951d8cc29
Fix selection of compactable blocks
7 years ago
Fabian Reinartz
5d7b5994d6
Fix compaction range selection
7 years ago
Goutham Veeramachaneni
9a2ab732b5
Choose ranges by retention and min-block-duration
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
7 years ago