Without this fix, if snapshots were enabled, and wbl goes missing
between restarts, then TSDB does not recognize that there are ooo
mmap chunks on disk and we cannot query them until those chunks
are compacted into blocks.
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
If the snapshot was enabled with some ooo mmap chunks on disk,
and wbl was removed between restarts, then we should still be able
to query the ooo mmap chunks after a restart. This test shows that
we are not able to query those ooo mmap chunks after a restart
under this situation.
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
M-map chunks replayed on startup are discarded if there
was no WAL and no snapshot loaded, because there is no
series created in the Head that it can map to. So only
load m-map chunks from disk if there is either a snapshot
loaded or there is WAL on disk.
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Scraping targets are synced by creating the full set, then adding/removing any which have changed.
This PR speeds up the process of creating the full set.
I added a benchmark for `TargetsFromGroup`; it uses configuration from a typical Kubernetes SD.
The crux of the change is to do relabeling inside labels.Builder instead of converting to labels.Labels and back again for every rule. The change is broken into several commits for easier review.
This is a breaking change to `scrape.PopulateLabels()`, but `relabel.Process` is left as-is, with a new `relabel.ProcessBuilder` option.
In storage/remote, try converting to RecoverableError using errors.As,
instead of through direct casting.
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
TestConcurrentRangeQueries runs many queries, up to 4 at the same time,
to try to expose any race conditions.
This change stops four of them from running with a thousand or more steps:
`holt_winters(a_X[1d], 0.3, 0.3)`
`changes(a_X[1d])`
`rate(a_X[1d])`
`absent_over_time(a_X[1d])`
Particularly when the test runs with `-race` in CI, this reduces the
time and resources required.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Direct imports:
go.opentelemetry.io/otel v1.14.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.14.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.14.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.14.0
go.opentelemetry.io/otel/sdk v1.14.0
go.opentelemetry.io/otel/trace v1.14.0
These seem to correspond:
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.39.0
Other direct dependencies required to update by Otel:
github.com/stretchr/testify v1.8.2
google.golang.org/grpc v1.53.0
Indirect dependencies required to update by Otel:
cloud.google.com/go/compute v1.15.1
github.com/cncf/xds/go v0.0.0-20230105202645-06c439db220b
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.14.0
go.opentelemetry.io/otel/metric v0.36.0
Also the import of go.opentelemetry.io/otel/semconv had to be updated
to v1.17.0 to match 60f7d42d1e/sdk/resource/process.go (L25)
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Common service discovery mechanisms such as Kubernetes can generate a
lot of target groups, so this function was allocating a lot of memory
which then immediately became garbage. Re-using the structures across
an entire Sync saves effort.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Save work converting to `Labels` then to `Builder`.
`PopulateLabels()` now takes as Builder as input.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Save work converting between Builder and Labels.
Also expose ProcessBuilder, so callers can supply a Builder.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
This lets relabelling work on a `Builder` rather than converting to and
from `Labels` on every rule.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Adds `WALReplayConcurrency` as an option on tsdb `Options` and `HeadOptions`.
If it is not set or set <=0, then `GOMAXPROCS` is used, which matches the previous behaviour.
Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>
The difference is modest, but we've used `slices.Sort` in lots of other
places so why not here.
name old time/op new time/op delta
Builder 1.04µs ± 3% 0.95µs ± 3% -8.27% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Builder 312B ± 0% 288B ± 0% -7.69% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Builder 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.008 n=5+5)
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
This makes the buffer the correct size for the common case that labels
have only been added. It will be too large for the case that labels are
changed, but the current buffer resize logic in `appendLabelTo` doubles
the buffer, so a small over-estimate is better.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Ganesh Vernekar