prometheus

Commit Graph

Author	SHA1	Message	Date
Brian Brazil	d50b9a5619	Reload after reading the WAL. (#460 ) This causes the head to be GCed at startup, removing any series that were read from the WAL but have since been written to a block. In systems with low ingestion rates, this potentially could be many many hours of data. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	407e12d051	Make MemPostings nested. This saves memory, about a quarter of the size of the postings map itself with high-cardinality labels (not including the post ids). Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	fc99b8bb3a	Make index reader postings nested. This reduces memory by only having to store the string's 16 bytes+map overheard once per label name, rather than duplicating it in every entry for the label value. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	c93e261466	Reduce memory taken up by posting/symbol tables. Reuse the string already allocated for symbols in the posting tables. Use a slice for symbols in v2 format. Move symbol size logic into the index code. Avoid duplication of lookupSymbol logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Tom Wilkie	88ebd749dd	Make newBReader return a struct, not a pointer. (#459 ) This shows up as a hot spot in profiles of queries involving lots of seeks, as each seek creates a new iterator. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Krasi Georgiev	b75d702ceb	fix flaky compaction test (#458 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	7f00217d77	Allow manual compaction for tests when compaction is disabled globally. (#412 ) for tests we need to control when a compaction happens so with this change automated compaction can be disabled, but allow to run it manually it tests. fixes failing tests in : https://github.com/prometheus/tsdb/pull/374 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	7f30395115	LabelNames() for Querier (#455 ) * LabelNames() for Querier Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Brian Brazil	41b54585d9	Use already open blocks while compacting. (#441 ) This roughly halves the RAM requirements of compaction. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Krasi Georgiev	3385571ddf	buffer-panic when reading a record after recPageTerm (#429 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	5a9ddeecef	fix lint errors (#439 ) unexported NewMemTombstones as this returns unexported memTombstones type which will not be shows in godoc. Added missing comments for exported methods. Removed unused RecordLogger,RecordReader interfaces. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Brian Brazil	910f3021b0	Use sampleBuf instead of maintaining lastValue. (#444 ) This cuts the size of memSize by 8B. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	10632217ce	Merge pull request #440 from prometheus/wal-reading Improve WAL reading	6 years ago
nilsocket	80981a6aac	FromMap(), sorts and returns instead of calling New() (#433 ) Signed-off-by: nilsocket <nilsocket@gmail.com>	6 years ago
Alin Sinpalean	171fc4ab5d	Limit the returned db.Querier to the requested time range (#351 ) Limit the returned `db.Querier` to the requested time range. Preallocate the `baseChunkSeries.lset` and `baseChunkSeries.chks` slices to the previous series' slice sizes to avoid unnecessary grow slice reallocations.	6 years ago
Krasi Georgiev	e4843938ba	add missing zero to tombstone magic number (#448 ) add missing zero to tombstone magic number constant. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	a9470dd8d5	few more comments to explain the WAL workflow (#430 ) More comments for the WAL package. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ganesh Vernekar	3a08a71d86	LabelNames() method to get all unique label names (#369 ) * LabelNames() method to get all unique label names Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Ganesh Vernekar	a95323c021	Add license headers to missing files (#447 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Brian Brazil	c7e7fd355e	Only send WAL read workers the samples they need. Calculating the modulus in each worker was a hotspot, and meant that you had more work to do the more cores you had. This cuts CPU usage (on my 8 core, 4 real core machine) by 33%, and walltime by 3% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	a64b0d51c4	Precalculate memSeries.head This is read far more than it changes. This cuts ~14% off walltme and ~27% off CPU for WAL reading. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	d8c8e4e6e4	Keep local cache of ids. With the various goroutines running, the locking in getByID is notable. This cuts cpu usage by ~25% and walltime by ~20%. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Brian Brazil	f0e79ec264	Actually reuse samples in loadWAL across records. This cuts walltime by 2.5X and CPU by 2X Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	6 years ago
Ye Ben	23a5f09085	fix a typo dont -> dont't (#438 ) Signed-off-by: yeya24 <ben.ye@daocloud.io>	6 years ago
Krasi Georgiev	ae91febcbb	Update the files format README.md (#437 )	6 years ago
Krasi Georgiev	d804a27062	refactor util funcs to allow re-usage. (#419 ) * refactor util funcs to allow reusage. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Ben	a8351dc9d0	Using filepath.Join() instead of strings with slashes (#428 ) fixes: https://github.com/prometheus/tsdb/issues/426 Using `filepath.Join()` instead of strings containing forward slash path delimiters (needed for non-*nix OSes), as suggested by @krasi-georgiev	6 years ago
Chris Marchbanks	f4afc7dff2	Add benchmark for querying a persisted block (#425 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
Krasi Georgiev	1dd9a6bd29	comments about the 120samples const and link to Gorilla papers. (#423 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	66b6b87cd4	fix windows tests (#421 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Thomas Jackson	b4132df5f7	Reduce allocations for queries on `HEAD` (#417 ) Some benchmarks for HEAD and allocate the correct slice size in LabelValues , we already know what it'll be This is ~15% time improvement, and ~25% allocation improvement: ``` benchmark old ns/op new ns/op delta BenchmarkHeadPostingForMatchers-4 74452 63514 -14.69% benchmark old allocs new allocs delta BenchmarkHeadPostingForMatchers-4 20 13 -35.00% benchmark old bytes new bytes delta BenchmarkHeadPostingForMatchers-4 5425 3137 -42.18% ``` Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>	6 years ago
Simon Pasquier	18af5763d8	Merge pull request #409 from simonpasquier/remove-prometheus-dep fileutil: remove dependency on prometheus/prometheus	6 years ago
Simon Pasquier	4dd740e4cc	fileutil: remove dependency on prometheus/prometheus Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Krasi Georgiev	cf2af2b371	add maintainers file (#404 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Bartek Płotka	047b1b1357	compact: Verify for chunks outside of compacted time range. Added unit test for populateBlocs. (#349 ) * compact: Verify for chunks outside of compacted time range. Unit test for populateBlocs. Signed-off-by: Bartek Plotka <bwplotka@gmail.com> Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	d7492b9350	more descriptive var names and some more logging. (#405 ) * more descriptive checkpoint var names and some more logging. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Camille Janicki	0ce41118ed	Add msg parameter to Equals function in testutil (#398 ) * Add msg parameter to Equals function in testutil Co-authored-by: Chris Marchbanks <csmarchbanks@gmail.com> Signed-off-by: Camille Janicki <camille.janicki@gmail.com>	6 years ago
Kautilya Tripathi	3fd6d2f920	more meaningful names for serializedStringTuples and stringTuples (#377 ) more meaningful names for serializedStringTuples and stringTuples structs Signed-off-by: knrt10 <tripathi.kautilya@gmail.com> Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Simon Pasquier	043e3bb5f9	Merge pull request #396 from codesome/new-metrics Add new metrics.	6 years ago
Ganesh Vernekar	2638b587f6	Merge remote-tracking branch 'upstream/master' into new-metrics Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Ganesh Vernekar	61d0868966	Fix TestCorrectNumTombstones (#399 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Ganesh Vernekar	61b000ee0e	Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Catalin Patulea	b4c7c80227	Reword chunk references, LSB usually means 'bits'. (#364 ) https://en.wikipedia.org/wiki/Bit_numbering#Least_significant_bit Signed-off-by: Catalin Patulea <catalinp@google.com>	6 years ago
Ganesh Vernekar	6e712963e2	Fix updating of NumTombstones in block.Delete(..) (#385 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Ganesh Vernekar	632dfb349e	Add new metrics. 1. 'prometheus_tsdb_wal_truncate_fail' for failed WAL truncation. 2. 'prometheus_tsdb_checkpoint_delete_fail' for failed old checkpoint delete. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Callum Styan	a971f52ac8	clean up after running repair tests (#372 ) implement a `CopyDirs` helper to walk all dirs and subdirs and copy all files within to a destination folder. Refactor `TestRepairBadIndexVersion` to use the `CopyDirs` so it works on a copy db and remove the copy after the test. This allows running the test more than once and doesn't confuse git. Signed-off-by: Callum Stytan <callumstyan@gmail.com>	6 years ago
Martin Chodur	dbd765a0e3	Refactor Makefile to use Makefile.common (#324 ) make use of the Makefile.common Signed-off-by: Martin Chodur <m.chodur@seznam.cz>	6 years ago
Krasi Georgiev	d05611c027	removed some unused code and moved mockSeriesSet in querier_test (#394 ) Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Krasi Georgiev	d38516b1c2	remove unused changes variable (#391 ) This was added in `55a9b5428a` and later not used after some refactoring in following PRs. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	6 years ago
Chris Marchbanks	2db59a71a6	Fix assert order being backwards in a few places (#388 ) Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago

1 2 3 4 5 ...

868 Commits (d50b9a5619b2efb7559ea1a7187df9fa5c209121) All Branches Search

868 Commits (d50b9a5619b2efb7559ea1a7187df9fa5c209121)

All Branches