Browse Source

Evict chunk descs of all unloaded chunks during maintenance. (#2297)

Keeping these around has two problems:
1) Each desc takes 64 bytes, 10 of them is 640B. This is a lot of
overhead on a 1024 byte chunk.
2) It can take well over a week to reach a point where this and thus
Prometheus memory usage as a whole enters steady state. This makes RAM
estimation very hard for users, and makes it difficult to investigate
things like memory fragmentation.

Instead we'll wipe them during each memory series maintenance cycle, and
if a query pulls them in they'll hang around as cache until the next
cycle.
pull/2306/head
Brian Brazil 8 years ago committed by GitHub
parent
commit
93b70ee4ea
  1. 15
      storage/local/series.go

15
storage/local/series.go

@ -26,13 +26,6 @@ import (
)
const (
// chunkDescEvictionFactor is a factor used for chunk.Desc eviction (as opposed
// to evictions of chunks, see method evictOlderThan. A chunk takes about 20x
// more memory than a chunk.Desc. With a chunkDescEvictionFactor of 10, not more
// than a third of the total memory taken by a series will be used for
// chunkDescs.
chunkDescEvictionFactor = 10
headChunkTimeout = time.Hour // Close head chunk if not touched for that long.
)
@ -283,11 +276,11 @@ func (s *memorySeries) maybeCloseHeadChunk() bool {
return false
}
// evictChunkDescs evicts chunkDescs if there are chunkDescEvictionFactor times
// more than non-evicted chunks. iOldestNotEvicted is the index within the
// current chunkDescs of the oldest chunk that is not evicted.
// evictChunkDescs evicts chunkDescs if the chunk is evicted.
// iOldestNotEvicted is the index within the current chunkDescs of the oldest
// chunk that is not evicted.
func (s *memorySeries) evictChunkDescs(iOldestNotEvicted int) {
lenToKeep := chunkDescEvictionFactor * (len(s.chunkDescs) - iOldestNotEvicted)
lenToKeep := len(s.chunkDescs) - iOldestNotEvicted
if lenToKeep < len(s.chunkDescs) {
s.savedFirstTime = s.firstTime()
lenEvicted := len(s.chunkDescs) - lenToKeep

Loading…
Cancel
Save