mirror of https://github.com/prometheus/prometheus
main
beorn7/metrics
nhcb-scrape-optimize2
dependabot/npm_and_yarn/web/ui/react-app/bootstrap-5.0.0
keep-multiple-separators
labels-utf8-fix
no-ignore-identifying-attributes
release-3.0
cedwards/nhcb-wal-wbl
alexg/settable-user-agent
dependabot/npm_and_yarn/web/ui/tanstack/react-query-5.60.6
dependabot/npm_and_yarn/web/ui/codemirror/language-6.10.3
dependabot/npm_and_yarn/web/ui/reduxjs/toolkit-2.3.0
release-2.53
release-2.55
otel-3
bwplotka/2_18_cmp
fionaliao/3.0-migration-guide-fixes
bwplotka/2_0_cmp
alexg/fix-fallback-scrape
cedwards/ooo-native-histograms-promql-tests
dependabot/npm_and_yarn/web/ui/react-router-dom-6.27.0
dependabot/npm_and_yarn/web/ui/uiw/react-codemirror-4.23.6
dependabot/npm_and_yarn/web/ui/typescript-eslint/eslint-plugin-8.12.2
dependabot/npm_and_yarn/web/ui/lezer/common-1.2.3
dependabot/npm_and_yarn/web/ui/uplot-1.6.31
dependabot/npm_and_yarn/web/ui/testing-library/jest-dom-6.6.3
dependabot/npm_and_yarn/web/ui/types/jest-29.5.14
dependabot/npm_and_yarn/web/ui/eslint/compat-1.2.2
dependabot/npm_and_yarn/web/ui/eslint/js-9.14.0
dependabot/npm_and_yarn/web/ui/vitejs/plugin-react-4.3.3
dependabot/npm_and_yarn/web/ui/rollup/plugin-node-resolve-15.3.0
dependabot/npm_and_yarn/web/ui/mantine/code-highlight-7.13.4
dependabot/npm_and_yarn/web/ui/sanitize-html-2.13.1
dependabot/npm_and_yarn/web/ui/globals-15.11.0
dependabot/npm_and_yarn/web/ui/types/react-18.3.12
dependabot/npm_and_yarn/web/ui/types/lodash-4.17.13
dependabot/npm_and_yarn/web/ui/eslint-plugin-react-hooks-5.1.0-rc.0
ooo-nh-fix
jvp/refactor-append-to-have-hints
bwplotka/nhcb-wal-wbl
jvp/refactor-append-to-have-hints-as-value
jvp/refactor-append-to-have-hints-and-single-appendcall
redesign-parser-interface
imprct
textparsebench-manik1
manik2
rel-2.55-backport-14947
bisect-series-base
callum-rw-split-intern
catch-sighup-early
njpm/fix-mockIndex
trim-3.0changelog-for-2.55
njpm/tsdb-utf8-mixed-querying
callum-vt20
julius/y-axis-zero
superq/help_link
jvp/enable-otlp-endpoint-by-default
krajo/testing-pr14831
revert-14809-patch-mem-issue
cut-2.54.2
release-2.54
arthursens/appendct-prwv2
arthursens/ct-in-wal
callum-rw-intern-v3
callum-rw-intern-unique
pull/14654/head
release-2.54+dedupelabels
superq/update_storage_docs
manik1
revert-14513-fix-histogram-overlap-compaction-bug
v2.54.0+dedupelabels
jvp/metadata-poc
mantine-ui-uplot
superq/betteralign_promql
callum-live-segment-test
remote-write-2.0
rw20-nhcb
nhcb
njpm/rw2-main-sync-conflicts
rw20-helpers-gen
rw20-gen
release-2.45
callum-filter-benchmark
jvp/range-duration-with-open-intervals
callum-simplify-ts-filtering
update-2.53-to-correct-place
beorn7/lookback
release-2.52
faster-exemplars
alexg-remove-HEAD-1
rw20-config
alexg-content-negotiation-dont-advertise
release-2.51
revert-13498-alexg-content-negotiation-2
alexg-content-negotiation-2
release-2.51+dedupelabels
krajo/engine-immutable-histograms
release-2.50
feature/new-prom-ui
owilliams/utf8-03-reads
njpm/rw2-sync-main-conflicts
superq/v2
alexg-content-negotiation
release-v2.50
owilliams/utf8-02-mimir
owilliams/utf8
release-2.49
csmarchbanks/native-histogram-compact-format
csmarchbanks/native-histogram-json-format
csmarchbanks/native-histogram-verbose-format
njpm/metadata-remote-write-wiring-2-update
prw-shard-test
arct2
release-2.48
vtproto-test
bwplotka/prw-initial
test-rw-compression
callum-rw-min-format
callum-rw-e2e-dashboard-1
release-2.47
callum-opportunistic-labels-cleanup
feature/shutdownscrape
bwplotka/config-watch
release-2.37
release-2.46
remote-write-checkpoint
native-histogram-charts
prom-common-update-0.43
release-2.44
release-2.43+stringlabels
release-2.43
callum-track-series-segment
callum-remote-proto-2
callum-watcher-poll-backoff
release-2.42
histogram-charts
csmarchbanks/optmize-arrow-responses
csmarchbanks/arrow-responses
csmarchbanks/arrow-native-histograms
release-2.41
release-2.40
release-2.39
add-body-size-limit-global
pre-ooo
release-2.38
remove-circleci-windows-tests
nexucis/sd-optimisation
revert-10878-jvp/replay-wait-for-idle-dont-always-sleep
histogram-json-experiment
release-2.36
nexucis/sofisticat-query
wip-play-prombench
release-2.35
release-2.34
release-2.33
rmremote_storage
release-2.32
release-2.31
release-2.30
beorn7/polymorphism
superq/fieldalignment
mem/fix_8551
release-2.30-merge
release-2.29
richih/remove_indirect_deps
expand
release-2.28
proposalprocess
release-docs-formatting-fix
merge-main
release-2.26
release-2.27
bp/append-docs
release-2.25
revert-8489-append
superq/drop_vendor
post-processing
exemplar-storage-separate-mvp
release-2.24
multi-err-cause
partial-response
lazy-unsymbolize
release-2.23
release-2.22
for-cortex-1-4-1
lbl-optimize
write-req-content-size
release-2.21
fix-2.21-latency
promselect
fix-2.21-iter-latency
segfault-chunk-iter
release-2.20
scrape-bytes-scraped
release-2.19
tsdb-import
tsdb-import-csv
fix-sym-7373-3
fix-sym-7373-2
fix-sym-7373
release-2.18
release-2.17
superq/test
series-hint
add-chunkiterator-interface
rule-page-followups
shutdown-select-bug
revert-6907-prom/rules_race_condition
release-2.16
revert-6605-master
join
nav-settings
release-2.15
f17-test
react-travis-test-branch
release-2.14
release-2.13
release-2.12
release-2.11
rr
remove-alertmanager
rr-proto
integrate-react
release-2.10
allow-swappable-engine
release-2.9
fix/binop-output-labels
release-2.8
release-2.7
resharding
release-2.6
release-2.5
release-2.4
release-2.3
release-2.2
richih/examples
release-2.1
release-2.0
release-1.8.2
release-1.8
release-1.7
release-1.6
release-1.5
release-1.4
release-1.3
release-1.2
release-1.1
release-0.20
release-0.18
release-0.16
v0.300.0
v3.0.0
v0.300.0-rc.1
v3.0.0-rc.1
v0.55.1
v2.55.1
v0.53.3
v2.53.3
v0.300.0-beta.1
v3.0.0-rc.0
v0.300.0-rc.0
v0.55.0
v2.55.0
v2.55.0-rc.1
v2.55.0-rc.0
v0.300.0-beta.0
v3.0.0-beta.0
v0.54.1
v2.54.1
v0.53.2
v2.53.2
v2.54.0+dedupelabels
v0.54.0
v2.54.0
v2.54.0-rc.1
v2.54.0-rc.0+dedupelabels
v0.54.0-rc.0
v2.54.0-rc.0
v0.53.1
v2.53.1
v0.45.6
v2.45.6
v0.53.0
v2.53.0
v0.53.0-rc.1
v2.53.0-rc.1
v0.53.0-rc.0
v2.53.0-rc.0
v0.52.1
v0.52.0
v2.52.0
v0.52.0-rc.1
v2.52.0-rc.1
v0.45.5
v2.45.5
v0.52.0-rc.0
v2.52.0-rc.0
v0.51.2
v2.51.2
v0.51.1
v2.51.1
v2.51.0+dedupelabels
v0.51.0
v2.51.0
v0.45.4
v2.45.4
v2.51.0-rc.0+dedupelabels
v2.51.0-rc.0
v0.50.1
v2.50.1
v0.50.0
v2.50.0
v0.50.0-rc.1
v2.50.0-rc.1
v0.50.0-rc.0
v2.50.0-rc.0
v0.45.3
v2.45.3
v0.49.1
v0.49.0
v2.49.1
v2.49.0
v0.49.0-rc.2
v2.49.0-rc.2
v0.49.0-rc.1
v2.49.0-rc.1
v0.45.2
v2.45.2
v0.49.0-rc.0
v2.49.0-rc.0
v0.48.1
v2.48.1
v0.48.0
v2.48.0
v0.48.0-rc.2
v2.48.0-rc.2
v0.48.0-rc.1
v2.48.0-rc.1
v0.48.0-rc.0
v2.48.0-rc.0
v0.47.2
v2.47.2
v0.47.1
v2.47.1
v0.45.1
v2.45.1
v0.47.0
v2.47.0
v0.47.0-rc.0
v2.47.0-rc.0
v0.37.9
v2.37.9
v0.46.0
v2.46.0
v0.46.0-rc.0
v2.46.0-rc.0
v0.45.0
v2.45.0
v0.45.0-rc.1
v2.45.0-rc.1
v0.45.0-rc.0
v2.45.0-rc.0
v0.44.0
v2.44.0
v0.44.0-rc.1
v0.44.0-rc.2
v2.44.0-rc.2
v0.37.8
v2.37.8
v0.43.1
v2.43.1
v2.43.1+stringlabels
v2.44.0-rc.1
v0.37.7
v2.37.7
v0.44.0-rc.0
v2.44.0-rc.0
v0.43.0
v2.43.0
v2.43.0+stringlabels
v2.43.0-rc.1+stringlabels
v0.43.0-rc.1
v2.43.0-rc.1
v0.43.0-rc.0
v2.43.0-rc.0
v0.37.6
v2.37.6
v0.42.0
v2.42.0
v2.42.0-rc.0
v0.41.0
v2.41.0
v0.41.0-rc.0
v2.41.0-rc.0
v0.40.7
v2.40.7
v0.40.6
v2.40.6
v0.37.5
v2.37.5
v0.40.5
v2.40.5
v2.40.4
v0.40.4
v2.37.4
v0.37.4
v0.40.3
v2.40.3
v0.37.3
v2.37.3
v0.40.2
v2.40.2
v0.40.1
v2.40.1
v0.39.2
v2.39.2
v0.40.0
v2.40.0
v0.37.2
v2.37.2
v0.40.0-rc.0
v2.40.0-rc.0
v0.39.1
v2.39.1
v0.39.0
v2.39.0
v2.39.0-rc.0
v0.37.1
v2.37.1
v0.38.0
v2.38.0
v0.38.0-rc.0
v2.38.0-rc.0
v0.37.0
v2.37.0
v0.37.0-rc.1
v2.37.0-rc.1
v0.37.0-rc.0
v2.37.0-rc.0
v0.36.2
v2.36.2
v0.36.1
v2.36.1
v0.36.0
v2.36.0
v0.36.0-rc0
v2.36.0-rc0
v0.35.0
v2.35.0
v1.99.0
v1.99.0-retract
v2.35.0-retract
v0.35.0-rc1
v2.35.0-rc1
v2.35.0-rc0
v2.34.0
v2.34.0-rc.2
v2.34.0-rc.1
v2.33.5
v2.34.0-rc.0
v2.33.4
v2.33.3
v2.33.2
v2.33.1
v2.33.0
v2.33.0-rc.1
v2.33.0-rc.0
v2.32.1
v2.32.0
v2.31.2
v2.30.4
v2.32.0-rc.1
v2.32.0-rc.0
v2.32.0-beta.0
v2.31.1
v2.31.0
v2.31.0-rc.1
v2.31.0-rc.0
v2.30.3
v2.30.2
v2.30.1
v2.30.0
v2.30.0-rc.0
v2.29.2
v2.29.1
v2.29.0
v2.29.0-rc.2
v2.29.0-rc.1
v2.29.0-rc.0
v2.28.1
v2.28.0
v2.28.0-rc.0
v2.27.1
v2.26.1
v2.27.0
v2.27.0-rc.0
v2.26.0
v2.26.0-rc.0
v2.25.2
v2.25.1
v2.25.0
v2.25.0-rc.0
v2.24.1
v2.24.0
v2.24.0-rc.0
v2.23.0
v2.23.0-rc.0
v2.22.2
v2.22.1
v2.22.0
v2.22.0-rc.0
v2.21.0
v2.21.0-rc.1
v2.21.0-rc.0
v2.20.1
v2.19.3
v2.20.0
v2.20.0-rc.1
v2.20.0-rc.0
v2.19.2
v2.19.1
v2.19.0
v2.18.2
v2.19.0-rc.0
v2.18.1
v2.18.0
v2.18.0-rc.1
v2.18.0-rc.0
v2.17.2
v2.17.1
v2.17.0
v2.17.0-rc.4
v2.17.0-rc.3
v2.17.0-rc.2
v2.17.0-rc.1
v2.17.0-rc.0
v2.16.0
v2.16.0-rc.1
v2.16.0-rc.0
v2.15.2
v2.15.1
v2.15.0
v2.15.0-rc.0
v2.14.0
v2.14.0-rc.0
v2.13.1
v2.13.0
v2.12.0
v2.12.0-rc.0
v2.11.2
v2.11.1
v2.11.0
v2.11.0-rc.0
v2.10.0
v2.10.0-rc.0
v2.9.2
v2.9.1
v2.9.0
v2.9.0-rc.0
v2.8.1
v2.8.0
v2.8.0-rc.0
v2.7.2
v2.7.1
v2.7.0
v2.7.0-rc.2
v2.7.0-rc.1
v2.7.0-rc.0
v2.6.1
v2.6.0
v2.6.0-rc.1
v2.6.0-rc.0
v2.5.0
v2.5.0-rc.2
v2.5.0-rc.1
v2.5.0-rc.0
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.4.0-rc.0
v2.3.2
v2.3.1
v2.2.0-rc.0
v2.1.0
v2.0.0
v2.0.0-rc.3
v1.8.2
v1.8.1
v1.8.0
v2.0.0-rc.0
v1.7.2
v1.7.1
v1.7.0
v1.6.1
v1.6.0
v1.5.2
v1.5.1
v1.5.0
v1.3.1
v1.2.3
v1.2.2
v1.2.1
v1.2.0
v1.0.1
0.1.0
0.10.0
0.11.0
0.11.1
0.12.0
0.13.0
0.13.0rc2
0.13.1
0.13.2
0.13.3
0.13.4
0.14.0
0.14.0rc1
0.14.0rc2
0.14.0rc3
0.15.0
0.15.0rc1
0.15.0rc2
0.15.0rc3
0.15.1
0.16.0
0.16.0rc1
0.16.0rc2
0.16.1
0.16.2
0.17.0
0.17.0rc1
0.17.0rc2
0.18.0
0.18.0rc1
0.19.0
0.19.1
0.19.2
0.19.3
0.2.0
0.2.1
0.20.0
0.3.0
0.4.0
0.5.0
0.6.0
0.7.0
0.8.0
0.9.0
0.9.0rc1
0.9.0rc2
0.9.0rc3
0.9.0rc4
0.9.0rc5
all-tests-pass
checkout
dev
discovery-handle-discoverer-updates
v1.0.0
v1.0.0-rc.0
v1.0.2
v1.1.0
v1.1.1
v1.1.2
v1.1.3
v1.3.0
v1.3.0-beta.0
v1.4.0
v1.4.1
v1.5.3
v1.6.2
v1.6.3
v2.0.0-alpha.0
v2.0.0-alpha.1
v2.0.0-alpha.2
v2.0.0-alpha.3
v2.0.0-beta.0
v2.0.0-beta.1
v2.0.0-beta.2
v2.0.0-beta.3
v2.0.0-beta.4
v2.0.0-beta.5
v2.0.0-rc.1
v2.0.0-rc.2
v2.13.0-rc.0
v2.2.0
v2.2.0-rc.1
v2.2.1
v2.3.0
v3.0.0-beta.1
${ noResults }
72 Commits (68e59374741f104eab20a0a9eb0aaba8ea437cbc)
Author | SHA1 | Message | Date |
---|---|---|---|
Annanay | 7f98a744e5 |
Add context to Appender interface
Signed-off-by: Annanay <annanayagarwal@gmail.com> |
4 years ago |
Julien Pivotto |
cfe30a7b62
|
TSDB: Use t.Cleanup to delete temporary files (#7620)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> |
4 years ago |
Ganesh Vernekar |
1c99adb9fd
|
Callbacks for lifecycle of series in TSDB (#7159)
* Callbacks for lifecycle of series in TSDB Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add more comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |
Ganesh Vernekar |
d4b9fe801f
|
M-map full chunks of Head from disk (#6679)
When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory Prom startup now happens in these stages - Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks. - Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series. If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss. [Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks. [The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files. In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file. **Prombench results** _WAL Replay_ 1h Wal reply time 30% less wal reply time - 4m31 vs 3m36 2h Wal reply time 20% less wal reply time - 8m16 vs 7m _Memory During WAL Replay_ High Churn: 10-15% less RAM - 32gb vs 28gb 20% less RAM after compaction 34gb vs 27gb No Churn: 20-30% less RAM - 23gb vs 18gb 40% less RAM after compaction 32.5gb vs 20gb Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |
Julien Pivotto | 73228b1b68 |
Those links should not be reverted
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> |
5 years ago |
Julien Pivotto | 653f343547 |
Revert head posting optimization
This reverts commit
|
5 years ago |
Julien Pivotto | 52630ad0c7 |
Make head Postings only return series in time range
benchmark old ns/op new ns/op delta BenchmarkQuerierSelect/Head/1of1000000-8 405805161 120436132 -70.32% BenchmarkQuerierSelect/Head/10of1000000-8 403079620 120624292 -70.07% BenchmarkQuerierSelect/Head/100of1000000-8 404678647 120923522 -70.12% BenchmarkQuerierSelect/Head/1000of1000000-8 403145813 118636563 -70.57% BenchmarkQuerierSelect/Head/10000of1000000-8 405020046 125716206 -68.96% BenchmarkQuerierSelect/Head/100000of1000000-8 426305002 175808499 -58.76% BenchmarkQuerierSelect/Head/1000000of1000000-8 619002108 567013003 -8.40% BenchmarkQuerierSelect/SortedHead/1of1000000-8 1276316086 120281094 -90.58% BenchmarkQuerierSelect/SortedHead/10of1000000-8 1282631170 121836526 -90.50% BenchmarkQuerierSelect/SortedHead/100of1000000-8 1325824787 121174967 -90.86% BenchmarkQuerierSelect/SortedHead/1000of1000000-8 1271386268 121025117 -90.48% BenchmarkQuerierSelect/SortedHead/10000of1000000-8 1280223345 130838948 -89.78% BenchmarkQuerierSelect/SortedHead/100000of1000000-8 1271401620 243635515 -80.84% BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 1360256090 1307744674 -3.86% BenchmarkQuerierSelect/Block/1of1000000-8 748183120 707888498 -5.39% BenchmarkQuerierSelect/Block/10of1000000-8 741084129 716317249 -3.34% BenchmarkQuerierSelect/Block/100of1000000-8 722157273 735624256 +1.86% BenchmarkQuerierSelect/Block/1000of1000000-8 727587744 731981838 +0.60% BenchmarkQuerierSelect/Block/10000of1000000-8 727518578 726860308 -0.09% BenchmarkQuerierSelect/Block/100000of1000000-8 765577046 757382386 -1.07% BenchmarkQuerierSelect/Block/1000000of1000000-8 1126722881 1084779083 -3.72% benchmark old allocs new allocs delta BenchmarkQuerierSelect/Head/1of1000000-8 4000018 24 -100.00% BenchmarkQuerierSelect/Head/10of1000000-8 4000036 82 -100.00% BenchmarkQuerierSelect/Head/100of1000000-8 4000216 625 -99.98% BenchmarkQuerierSelect/Head/1000of1000000-8 4002016 6028 -99.85% BenchmarkQuerierSelect/Head/10000of1000000-8 4020016 60037 -98.51% BenchmarkQuerierSelect/Head/100000of1000000-8 4200016 600047 -85.71% BenchmarkQuerierSelect/Head/1000000of1000000-8 6000016 6000016 +0.00% BenchmarkQuerierSelect/SortedHead/1of1000000-8 4000055 28 -100.00% BenchmarkQuerierSelect/SortedHead/10of1000000-8 4000073 87 -100.00% BenchmarkQuerierSelect/SortedHead/100of1000000-8 4000253 630 -99.98% BenchmarkQuerierSelect/SortedHead/1000of1000000-8 4002053 6036 -99.85% BenchmarkQuerierSelect/SortedHead/10000of1000000-8 4020053 60054 -98.51% BenchmarkQuerierSelect/SortedHead/100000of1000000-8 4200053 600074 -85.71% BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 6000053 6000053 +0.00% BenchmarkQuerierSelect/Block/1of1000000-8 6000021 6000021 +0.00% BenchmarkQuerierSelect/Block/10of1000000-8 6000057 6000057 +0.00% BenchmarkQuerierSelect/Block/100of1000000-8 6000417 6000417 +0.00% BenchmarkQuerierSelect/Block/1000of1000000-8 6004017 6004017 +0.00% BenchmarkQuerierSelect/Block/10000of1000000-8 6040017 6040017 +0.00% BenchmarkQuerierSelect/Block/100000of1000000-8 6400017 6400017 +0.00% BenchmarkQuerierSelect/Block/1000000of1000000-8 10000018 10000018 +0.00% benchmark old bytes new bytes delta BenchmarkQuerierSelect/Head/1of1000000-8 176001177 1392 -100.00% BenchmarkQuerierSelect/Head/10of1000000-8 176002329 4368 -100.00% BenchmarkQuerierSelect/Head/100of1000000-8 176013849 33520 -99.98% BenchmarkQuerierSelect/Head/1000of1000000-8 176129056 321456 -99.82% BenchmarkQuerierSelect/Head/10000of1000000-8 177281049 3427376 -98.07% BenchmarkQuerierSelect/Head/100000of1000000-8 188801049 35055408 -81.43% BenchmarkQuerierSelect/Head/1000000of1000000-8 304001059 304001049 -0.00% BenchmarkQuerierSelect/SortedHead/1of1000000-8 229192188 2488 -100.00% BenchmarkQuerierSelect/SortedHead/10of1000000-8 229193340 5568 -100.00% BenchmarkQuerierSelect/SortedHead/100of1000000-8 229204860 35536 -99.98% BenchmarkQuerierSelect/SortedHead/1000of1000000-8 229320060 345104 -99.85% BenchmarkQuerierSelect/SortedHead/10000of1000000-8 230472060 3894672 -98.31% BenchmarkQuerierSelect/SortedHead/100000of1000000-8 241992060 40511632 -83.26% BenchmarkQuerierSelect/SortedHead/1000000of1000000-8 357192060 357192060 +0.00% BenchmarkQuerierSelect/Block/1of1000000-8 227201516 227201506 -0.00% BenchmarkQuerierSelect/Block/10of1000000-8 227203057 227203041 -0.00% BenchmarkQuerierSelect/Block/100of1000000-8 227217161 227217165 +0.00% BenchmarkQuerierSelect/Block/1000of1000000-8 227358279 227358289 +0.00% BenchmarkQuerierSelect/Block/10000of1000000-8 228769485 228769475 -0.00% BenchmarkQuerierSelect/Block/100000of1000000-8 242881487 242881477 -0.00% BenchmarkQuerierSelect/Block/1000000of1000000-8 384001705 384001705 +0.00% Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> |
5 years ago |
Bartlomiej Plotka | c0a9ab0829 |
Close db properly in tests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> |
5 years ago |
Bartlomiej Plotka | aadffd1360 |
Finally found a fix for the bug I was chasing for 2h...
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> |
5 years ago |
Bartlomiej Plotka | 34426766d8 |
Unify Iterator interfaces. All point to storage now.
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> |
5 years ago |
Thor |
17d8c49919
|
made stripe size configurable (#6644)
Signed-off-by: Thor <thansen@digitalocean.com> |
5 years ago |
Ganesh Vernekar |
e0733a99e3
|
Expose DB.Compact()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |
Brian Brazil | 0482d93fe6 |
Add contexts to index writer to fix test races.
With recent speed improvements to populate block, the cancellation test now fails regularly on CI. Use contexts to get the index writer to shut down much faster, and that allows us to make the cancellation test faster too. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> |
5 years ago |
Brian Brazil | 971dafdfbe |
Coalesce series reads where we can.
When compacting rather than doing a read of all series in the index per label name, do many at once but only when it won't use (much) more ram than writing the special all index does. original in-memory postings: BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1202383447 ns/op 158936496 B/op 1031511 allocs/op BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1141792706 ns/op 154453408 B/op 1093453 allocs/op BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1169288829 ns/op 161072336 B/op 1110021 allocs/op BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 1115700103 ns/op 149480472 B/op 1129180 allocs/op BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 1283813141 ns/op 162937800 B/op 1202771 allocs/op before: BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1145195941 ns/op 131749984 B/op 834400 allocs/op BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1233526345 ns/op 127889416 B/op 897033 allocs/op BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1821942296 ns/op 131665648 B/op 914836 allocs/op BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 8035568665 ns/op 123811832 B/op 934312 allocs/op BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 71325926267 ns/op 140722648 B/op 1016824 allocs/op after: BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 1 1101429174 ns/op 129063496 B/op 832571 allocs/op BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 1 1074466374 ns/op 124154888 B/op 894875 allocs/op BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 1 1166510282 ns/op 128790648 B/op 912931 allocs/op BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 1 1075013071 ns/op 120570696 B/op 933511 allocs/op BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 1 1231673790 ns/op 138754288 B/op 1022791 allocs/op Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> |
5 years ago |
Brian Brazil |
4df814f509
|
Don't buffer up tables in memory on compaction. (#6422)
We can instead write it as we go, and then go back and write in the length at the end. Also fix the compaction benchmark, which indicates no changes. For the benchmark, this brings maximum memory usage of the buffers from ~200kB down to 128B. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> |
5 years ago |
Tom Wilkie | de0a772b8e |
Port tsdb to use pkg/labels. (#6326)
* Port tsdb to use pkg/labels. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Get tests passing. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Remove useless cast. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Appease linters. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |
yuxiaobo | 7850f1b35c |
new world spelling mistake
Signed-off-by: yuxiaobo <yuxiaobogo@163.com> |
5 years ago |
陈谭军 | 103f26d188 |
fix the wrong word (#6069)
Signed-off-by: chentanjun <2799194073@qq.com> |
5 years ago |
Callum Styan | 3344bb5c33 |
Move WAL watcher code to tsdb/wal package. (#5999)
* Move WAL watcher code to tsdb/wal package. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix tests after moving WAL watcher code. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Lint fixes. Signed-off-by: Callum Styan <callumstyan@gmail.com> |
5 years ago |
Bartek Plotka | f0863a604e |
Removed extra tsdb/testutil after merge.
Signed-off-by: Bartek Plotka <bwplotka@gmail.com> |
5 years ago |
Ganesh Vernekar |
5ecef3542d
|
Cleanup after merging tsdb into prometheus
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |
Ganesh Vernekar |
7cf09b0395
|
Moving tsdb into its own subdirectory
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> |
5 years ago |