mirror of https://github.com/prometheus/prometheus
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
main
release-3.0
cedwards/nhcb-wal-wbl
keep-multiple-separators
labels-utf8-fix
alexg/settable-user-agent
beorn7/metrics
dependabot/npm_and_yarn/web/ui/tanstack/react-query-5.60.6
dependabot/npm_and_yarn/web/ui/codemirror/language-6.10.3
dependabot/npm_and_yarn/web/ui/reduxjs/toolkit-2.3.0
release-2.53
release-2.55
otel-3
bwplotka/2_18_cmp
fionaliao/3.0-migration-guide-fixes
bwplotka/2_0_cmp
alexg/fix-fallback-scrape
cedwards/ooo-native-histograms-promql-tests
dependabot/npm_and_yarn/web/ui/react-router-dom-6.27.0
dependabot/npm_and_yarn/web/ui/uiw/react-codemirror-4.23.6
dependabot/npm_and_yarn/web/ui/typescript-eslint/eslint-plugin-8.12.2
dependabot/npm_and_yarn/web/ui/lezer/common-1.2.3
dependabot/npm_and_yarn/web/ui/uplot-1.6.31
dependabot/npm_and_yarn/web/ui/testing-library/jest-dom-6.6.3
dependabot/npm_and_yarn/web/ui/types/jest-29.5.14
dependabot/npm_and_yarn/web/ui/eslint/compat-1.2.2
dependabot/npm_and_yarn/web/ui/eslint/js-9.14.0
dependabot/npm_and_yarn/web/ui/vitejs/plugin-react-4.3.3
dependabot/npm_and_yarn/web/ui/rollup/plugin-node-resolve-15.3.0
dependabot/npm_and_yarn/web/ui/mantine/code-highlight-7.13.4
dependabot/npm_and_yarn/web/ui/sanitize-html-2.13.1
dependabot/npm_and_yarn/web/ui/globals-15.11.0
dependabot/npm_and_yarn/web/ui/types/react-18.3.12
dependabot/npm_and_yarn/web/ui/types/lodash-4.17.13
dependabot/npm_and_yarn/web/ui/eslint-plugin-react-hooks-5.1.0-rc.0
ooo-nh-fix
jvp/refactor-append-to-have-hints
bwplotka/nhcb-wal-wbl
jvp/refactor-append-to-have-hints-as-value
jvp/refactor-append-to-have-hints-and-single-appendcall
redesign-parser-interface
imprct
textparsebench-manik1
manik2
rel-2.55-backport-14947
bisect-series-base
callum-rw-split-intern
catch-sighup-early
njpm/fix-mockIndex
trim-3.0changelog-for-2.55
njpm/tsdb-utf8-mixed-querying
callum-vt20
julius/y-axis-zero
superq/help_link
jvp/enable-otlp-endpoint-by-default
krajo/testing-pr14831
revert-14809-patch-mem-issue
cut-2.54.2
release-2.54
arthursens/appendct-prwv2
arthursens/ct-in-wal
callum-rw-intern-v3
callum-rw-intern-unique
pull/14654/head
release-2.54+dedupelabels
superq/update_storage_docs
manik1
revert-14513-fix-histogram-overlap-compaction-bug
v2.54.0+dedupelabels
jvp/metadata-poc
mantine-ui-uplot
superq/betteralign_promql
callum-live-segment-test
remote-write-2.0
rw20-nhcb
nhcb
njpm/rw2-main-sync-conflicts
rw20-helpers-gen
rw20-gen
release-2.45
callum-filter-benchmark
jvp/range-duration-with-open-intervals
callum-simplify-ts-filtering
update-2.53-to-correct-place
beorn7/lookback
release-2.52
faster-exemplars
alexg-remove-HEAD-1
rw20-config
alexg-content-negotiation-dont-advertise
release-2.51
revert-13498-alexg-content-negotiation-2
alexg-content-negotiation-2
release-2.51+dedupelabels
krajo/engine-immutable-histograms
release-2.50
feature/new-prom-ui
owilliams/utf8-03-reads
njpm/rw2-sync-main-conflicts
superq/v2
alexg-content-negotiation
release-v2.50
owilliams/utf8-02-mimir
owilliams/utf8
release-2.49
csmarchbanks/native-histogram-compact-format
csmarchbanks/native-histogram-json-format
csmarchbanks/native-histogram-verbose-format
njpm/metadata-remote-write-wiring-2-update
prw-shard-test
arct2
release-2.48
vtproto-test
bwplotka/prw-initial
test-rw-compression
callum-rw-min-format
callum-rw-e2e-dashboard-1
release-2.47
callum-opportunistic-labels-cleanup
feature/shutdownscrape
bwplotka/config-watch
release-2.37
release-2.46
remote-write-checkpoint
native-histogram-charts
prom-common-update-0.43
release-2.44
release-2.43+stringlabels
release-2.43
callum-track-series-segment
callum-remote-proto-2
callum-watcher-poll-backoff
release-2.42
histogram-charts
csmarchbanks/optmize-arrow-responses
csmarchbanks/arrow-responses
csmarchbanks/arrow-native-histograms
release-2.41
release-2.40
release-2.39
add-body-size-limit-global
pre-ooo
release-2.38
remove-circleci-windows-tests
nexucis/sd-optimisation
revert-10878-jvp/replay-wait-for-idle-dont-always-sleep
histogram-json-experiment
release-2.36
nexucis/sofisticat-query
wip-play-prombench
release-2.35
release-2.34
release-2.33
rmremote_storage
release-2.32
release-2.31
release-2.30
beorn7/polymorphism
superq/fieldalignment
mem/fix_8551
release-2.30-merge
release-2.29
richih/remove_indirect_deps
expand
release-2.28
proposalprocess
release-docs-formatting-fix
merge-main
release-2.26
release-2.27
bp/append-docs
release-2.25
revert-8489-append
superq/drop_vendor
post-processing
exemplar-storage-separate-mvp
release-2.24
multi-err-cause
partial-response
lazy-unsymbolize
release-2.23
release-2.22
for-cortex-1-4-1
lbl-optimize
write-req-content-size
release-2.21
fix-2.21-latency
promselect
fix-2.21-iter-latency
segfault-chunk-iter
release-2.20
scrape-bytes-scraped
release-2.19
tsdb-import
tsdb-import-csv
fix-sym-7373-3
fix-sym-7373-2
fix-sym-7373
release-2.18
release-2.17
superq/test
series-hint
add-chunkiterator-interface
rule-page-followups
shutdown-select-bug
revert-6907-prom/rules_race_condition
release-2.16
revert-6605-master
join
nav-settings
release-2.15
f17-test
react-travis-test-branch
release-2.14
release-2.13
release-2.12
release-2.11
rr
remove-alertmanager
rr-proto
integrate-react
release-2.10
allow-swappable-engine
release-2.9
fix/binop-output-labels
release-2.8
release-2.7
resharding
release-2.6
release-2.5
release-2.4
release-2.3
release-2.2
richih/examples
release-2.1
release-2.0
release-1.8.2
release-1.8
release-1.7
release-1.6
release-1.5
release-1.4
release-1.3
release-1.2
release-1.1
release-0.20
release-0.18
release-0.16
v0.300.0
v3.0.0
v0.300.0-rc.1
v3.0.0-rc.1
v0.55.1
v2.55.1
v0.53.3
v2.53.3
v0.300.0-beta.1
v3.0.0-rc.0
v0.300.0-rc.0
v0.55.0
v2.55.0
v2.55.0-rc.1
v2.55.0-rc.0
v0.300.0-beta.0
v3.0.0-beta.0
v0.54.1
v2.54.1
v0.53.2
v2.53.2
v2.54.0+dedupelabels
v0.54.0
v2.54.0
v2.54.0-rc.1
v2.54.0-rc.0+dedupelabels
v0.54.0-rc.0
v2.54.0-rc.0
v0.53.1
v2.53.1
v0.45.6
v2.45.6
v0.53.0
v2.53.0
v0.53.0-rc.1
v2.53.0-rc.1
v0.53.0-rc.0
v2.53.0-rc.0
v0.52.1
v0.52.0
v2.52.0
v0.52.0-rc.1
v2.52.0-rc.1
v0.45.5
v2.45.5
v0.52.0-rc.0
v2.52.0-rc.0
v0.51.2
v2.51.2
v0.51.1
v2.51.1
v2.51.0+dedupelabels
v0.51.0
v2.51.0
v0.45.4
v2.45.4
v2.51.0-rc.0+dedupelabels
v2.51.0-rc.0
v0.50.1
v2.50.1
v0.50.0
v2.50.0
v0.50.0-rc.1
v2.50.0-rc.1
v0.50.0-rc.0
v2.50.0-rc.0
v0.45.3
v2.45.3
v0.49.1
v0.49.0
v2.49.1
v2.49.0
v0.49.0-rc.2
v2.49.0-rc.2
v0.49.0-rc.1
v2.49.0-rc.1
v0.45.2
v2.45.2
v0.49.0-rc.0
v2.49.0-rc.0
v0.48.1
v2.48.1
v0.48.0
v2.48.0
v0.48.0-rc.2
v2.48.0-rc.2
v0.48.0-rc.1
v2.48.0-rc.1
v0.48.0-rc.0
v2.48.0-rc.0
v0.47.2
v2.47.2
v0.47.1
v2.47.1
v0.45.1
v2.45.1
v0.47.0
v2.47.0
v0.47.0-rc.0
v2.47.0-rc.0
v0.37.9
v2.37.9
v0.46.0
v2.46.0
v0.46.0-rc.0
v2.46.0-rc.0
v0.45.0
v2.45.0
v0.45.0-rc.1
v2.45.0-rc.1
v0.45.0-rc.0
v2.45.0-rc.0
v0.44.0
v2.44.0
v0.44.0-rc.1
v0.44.0-rc.2
v2.44.0-rc.2
v0.37.8
v2.37.8
v0.43.1
v2.43.1
v2.43.1+stringlabels
v2.44.0-rc.1
v0.37.7
v2.37.7
v0.44.0-rc.0
v2.44.0-rc.0
v0.43.0
v2.43.0
v2.43.0+stringlabels
v2.43.0-rc.1+stringlabels
v0.43.0-rc.1
v2.43.0-rc.1
v0.43.0-rc.0
v2.43.0-rc.0
v0.37.6
v2.37.6
v0.42.0
v2.42.0
v2.42.0-rc.0
v0.41.0
v2.41.0
v0.41.0-rc.0
v2.41.0-rc.0
v0.40.7
v2.40.7
v0.40.6
v2.40.6
v0.37.5
v2.37.5
v0.40.5
v2.40.5
v2.40.4
v0.40.4
v2.37.4
v0.37.4
v0.40.3
v2.40.3
v0.37.3
v2.37.3
v0.40.2
v2.40.2
v0.40.1
v2.40.1
v0.39.2
v2.39.2
v0.40.0
v2.40.0
v0.37.2
v2.37.2
v0.40.0-rc.0
v2.40.0-rc.0
v0.39.1
v2.39.1
v0.39.0
v2.39.0
v2.39.0-rc.0
v0.37.1
v2.37.1
v0.38.0
v2.38.0
v0.38.0-rc.0
v2.38.0-rc.0
v0.37.0
v2.37.0
v0.37.0-rc.1
v2.37.0-rc.1
v0.37.0-rc.0
v2.37.0-rc.0
v0.36.2
v2.36.2
v0.36.1
v2.36.1
v0.36.0
v2.36.0
v0.36.0-rc0
v2.36.0-rc0
v0.35.0
v2.35.0
v1.99.0
v1.99.0-retract
v2.35.0-retract
v0.35.0-rc1
v2.35.0-rc1
v2.35.0-rc0
v2.34.0
v2.34.0-rc.2
v2.34.0-rc.1
v2.33.5
v2.34.0-rc.0
v2.33.4
v2.33.3
v2.33.2
v2.33.1
v2.33.0
v2.33.0-rc.1
v2.33.0-rc.0
v2.32.1
v2.32.0
v2.31.2
v2.30.4
v2.32.0-rc.1
v2.32.0-rc.0
v2.32.0-beta.0
v2.31.1
v2.31.0
v2.31.0-rc.1
v2.31.0-rc.0
v2.30.3
v2.30.2
v2.30.1
v2.30.0
v2.30.0-rc.0
v2.29.2
v2.29.1
v2.29.0
v2.29.0-rc.2
v2.29.0-rc.1
v2.29.0-rc.0
v2.28.1
v2.28.0
v2.28.0-rc.0
v2.27.1
v2.26.1
v2.27.0
v2.27.0-rc.0
v2.26.0
v2.26.0-rc.0
v2.25.2
v2.25.1
v2.25.0
v2.25.0-rc.0
v2.24.1
v2.24.0
v2.24.0-rc.0
v2.23.0
v2.23.0-rc.0
v2.22.2
v2.22.1
v2.22.0
v2.22.0-rc.0
v2.21.0
v2.21.0-rc.1
v2.21.0-rc.0
v2.20.1
v2.19.3
v2.20.0
v2.20.0-rc.1
v2.20.0-rc.0
v2.19.2
v2.19.1
v2.19.0
v2.18.2
v2.19.0-rc.0
v2.18.1
v2.18.0
v2.18.0-rc.1
v2.18.0-rc.0
v2.17.2
v2.17.1
v2.17.0
v2.17.0-rc.4
v2.17.0-rc.3
v2.17.0-rc.2
v2.17.0-rc.1
v2.17.0-rc.0
v2.16.0
v2.16.0-rc.1
v2.16.0-rc.0
v2.15.2
v2.15.1
v2.15.0
v2.15.0-rc.0
v2.14.0
v2.14.0-rc.0
v2.13.1
v2.13.0
v2.12.0
v2.12.0-rc.0
v2.11.2
v2.11.1
v2.11.0
v2.11.0-rc.0
v2.10.0
v2.10.0-rc.0
v2.9.2
v2.9.1
v2.9.0
v2.9.0-rc.0
v2.8.1
v2.8.0
v2.8.0-rc.0
v2.7.2
v2.7.1
v2.7.0
v2.7.0-rc.2
v2.7.0-rc.1
v2.7.0-rc.0
v2.6.1
v2.6.0
v2.6.0-rc.1
v2.6.0-rc.0
v2.5.0
v2.5.0-rc.2
v2.5.0-rc.1
v2.5.0-rc.0
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.4.0-rc.0
v2.3.2
v2.3.1
v2.2.0-rc.0
v2.1.0
v2.0.0
v2.0.0-rc.3
v1.8.2
v1.8.1
v1.8.0
v2.0.0-rc.0
v1.7.2
v1.7.1
v1.7.0
v1.6.1
v1.6.0
v1.5.2
v1.5.1
v1.5.0
v1.3.1
v1.2.3
v1.2.2
v1.2.1
v1.2.0
v1.0.1
0.1.0
0.10.0
0.11.0
0.11.1
0.12.0
0.13.0
0.13.0rc2
0.13.1
0.13.2
0.13.3
0.13.4
0.14.0
0.14.0rc1
0.14.0rc2
0.14.0rc3
0.15.0
0.15.0rc1
0.15.0rc2
0.15.0rc3
0.15.1
0.16.0
0.16.0rc1
0.16.0rc2
0.16.1
0.16.2
0.17.0
0.17.0rc1
0.17.0rc2
0.18.0
0.18.0rc1
0.19.0
0.19.1
0.19.2
0.19.3
0.2.0
0.2.1
0.20.0
0.3.0
0.4.0
0.5.0
0.6.0
0.7.0
0.8.0
0.9.0
0.9.0rc1
0.9.0rc2
0.9.0rc3
0.9.0rc4
0.9.0rc5
all-tests-pass
checkout
dev
discovery-handle-discoverer-updates
v1.0.0
v1.0.0-rc.0
v1.0.2
v1.1.0
v1.1.1
v1.1.2
v1.1.3
v1.3.0
v1.3.0-beta.0
v1.4.0
v1.4.1
v1.5.3
v1.6.2
v1.6.3
v2.0.0-alpha.0
v2.0.0-alpha.1
v2.0.0-alpha.2
v2.0.0-alpha.3
v2.0.0-beta.0
v2.0.0-beta.1
v2.0.0-beta.2
v2.0.0-beta.3
v2.0.0-beta.4
v2.0.0-beta.5
v2.0.0-rc.1
v2.0.0-rc.2
v2.13.0-rc.0
v2.2.0
v2.2.0-rc.1
v2.2.1
v2.3.0
v3.0.0-beta.1
${ noResults }
Brian Brazil
d782387f81
Rather than buffer up symbols in RAM, do it one by one during compaction. Then use the reader's symbol handling for symbol lookups during the rest of the index write. There is some slowdown in compaction, due to having to look through a file rather than a hash lookup. This is noise to the overall cost of compacting series with thousands of samples though. benchmark old ns/op new ns/op delta BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 539917175 675341565 +25.08% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 2441815993 2477453524 +1.46% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 3978543559 3922909687 -1.40% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 8430219716 8586610007 +1.86% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 1786424591 1909552782 +6.89% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 5328998202 6020839950 +12.98% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 10085059958 11085278690 +9.92% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 25497010155 27018079806 +5.97% BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 2427391406 2817217987 +16.06% BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 2592965497 2538805050 -2.09% BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 2437388343 2668012858 +9.46% BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 2317095324 2787423966 +20.30% BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 2600239857 2096973860 -19.35% benchmark old allocs new allocs delta BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 500851 470794 -6.00% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 821527 791451 -3.66% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 1141562 1111508 -2.63% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 2141576 2111504 -1.40% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 871466 841424 -3.45% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 1941428 1911415 -1.55% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 3071573 3041510 -0.98% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 6771648 6741509 -0.45% BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 731493 824888 +12.77% BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 793918 887311 +11.76% BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 811842 905204 +11.50% BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 832244 925081 +11.16% BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 921553 1019162 +10.59% benchmark old bytes new bytes delta BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 40532648 35698276 -11.93% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 60340216 53409568 -11.49% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 81087336 72065552 -11.13% BenchmarkCompaction/type=normal,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 142485576 120878544 -15.16% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=101-4 208661368 203831136 -2.31% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=1001-4 347345904 340484696 -1.98% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=2001-4 585185856 576244648 -1.53% BenchmarkCompaction/type=vertical,blocks=4,series=10000,samplesPerSeriesPerBlock=5001-4 1357641792 1358966528 +0.10% BenchmarkCompactionFromHead/labelnames=1,labelvalues=100000-4 126486664 119666744 -5.39% BenchmarkCompactionFromHead/labelnames=10,labelvalues=10000-4 122323192 115117224 -5.89% BenchmarkCompactionFromHead/labelnames=100,labelvalues=1000-4 126404504 119469864 -5.49% BenchmarkCompactionFromHead/labelnames=1000,labelvalues=100-4 119047832 112230408 -5.73% BenchmarkCompactionFromHead/labelnames=10000,labelvalues=10-4 136576016 116634800 -14.60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> |
5 years ago | |
---|---|---|
.. | ||
format | Stream symbols during compaction. (#6468) | 5 years ago |