Browse Source

Remove 4 interval staleness heuristic. (#3244)

This means that if there is no stale marker, only the usual staleness
delta (5m) applies.

It has occured to me that there is an oddity in the heurestic. It works
fine as long as you have 2 points within the last 5m, but breaks down
when the time window advances to the point where you have just 1 point.

Consider you had points at t=0 and t=10. With the heurestic it goes stale
at t=51, up until t=300. However from t=301 until t=310 we only
see the t=10 point and the series comes back to life. That is not
desirable.

I don't see a way to keep this form of heurestic working given this
issue, so thus I'm removing it.
reviewable/pr3248/r1
Brian Brazil 7 years ago committed by GitHub
parent
commit
67274f0794
  1. 14
      promql/engine.go
  2. 6
      promql/testdata/staleness.test

14
promql/engine.go

@ -779,20 +779,6 @@ func (ev *evaluator) vectorSelector(node *VectorSelector) Vector {
if value.IsStaleNaN(v) {
continue
}
// Find timestamp before this point, within the staleness delta.
prevT, _, ok := it.PeekBack(peek)
if ok && prevT >= refTime-durationMilliseconds(LookbackDelta) {
interval := t - prevT
if interval*4+interval/10 < refTime-t {
// It is more than 4 (+10% for safety) intervals
// since the last data point, skip as stale.
//
// We need 4 to allow for federation, as with a 10s einterval an eval
// started at t=10 could be ingested at t=20, scraped for federation at
// t=30 and only ingested by federation at t=40.
continue
}
}
vec = append(vec, Sample{
Metric: node.series[i].Labels(),

6
promql/testdata/staleness.test vendored

@ -13,11 +13,11 @@ eval instant at 30s metric
eval instant at 40s metric
{__name__="metric"} 2
# It goes stale 4 intervals + 10% after the last sample.
eval instant at 71s metric
# It goes stale 5 minutes after the last sample.
eval instant at 330s metric
{__name__="metric"} 2
eval instant at 72s metric
eval instant at 331s metric
# Range vector ignores stale sample.

Loading…
Cancel
Save