prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	8f92c90897	Add TODOs and some minor tweaks Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	4998b9750f	chunkenc: Bugfix and naming tweaks Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	78ef9c6359	chunkenc: make xor reading more DRY Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	4a1b84f8b2	chunkenc: make xor writing more DRY Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Björn Rabenstein	3704c6c20a	Merge pull request #9533 from prometheus/beorn7/sparsehistogram tsdb: Complete chunk format documentation	3 years ago
beorn7	1a4e54cfbb	tsdb: Complete chunk format documentation This also tweaks and fixes a few things done previously. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	0876d57aea	chunkenc: Add test for chunk layout encoding And fix a bug exposed by it... Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	ad9b4c2b68	Fix typos Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	ed33aea392	Avoid redundant varint decoding in chunk appender construction Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	d31bb75dc4	Use VarbitUint rather than VarbitInt to encode len(spans) Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	3179215a59	Encode zero threshold first This guaranees that the zero threshold is byte-aligned. Not sure if that helps in any way, but at least it won't harm. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	c5522677bf	Improve encoding of zero threshold Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	7093b089f2	Use more varbit in histogram chunks This adds bit buckets for larger numbers to varbit encoding and also an unsigned version of varbit encoding. Then, varbit encoding is used for all the histogram chunk data instead of varint. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Björn Rabenstein	7309c20e7e	Merge pull request #9500 from codesome/resettests Add unit test for counter reset header	3 years ago
Ganesh Vernekar	dcaf568279	Metadata -> Layout renaming Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	4e206c7c77	Fix reviews Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	85e6686f84	Add unit test for counter reset header Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Björn Rabenstein	311673d62e	Save on slice allocations in histogramIterator (#9494 ) Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Björn Rabenstein	c450c01eb9	Remove obsolete TODOs about metadata (#9490 ) Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	7a8bb8222c	Style cleanup of all the changes in sparsehistogram so far A lot of this code was hacked together, literally during a hackathon. This commit intends not to change the code substantially, but just make the code obey the usual style practices. A (possibly incomplete) list of areas: * Generally address linter warnings. * The `pgk` directory is deprecated as per dev-summit. No new packages should be added to it. I moved the new `pkg/histogram` package to `model` anticipating what's proposed in #9478. * Make the naming of the Sparse Histogram more consistent. Including abbreviations, there were just too many names for it: SparseHistogram, Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in general. Only add "Sparse" if it is needed to avoid confusion with conventional Histograms (which is rare because the TSDB really has no notion of conventional Histograms). Use abbreviations only in local scope, and then really abbreviate (not just removing three out of seven letters like in "Histo"). This is in the spirit of https://github.com/golang/go/wiki/CodeReviewComments#variable-names * Several other minor name changes. * A lot of formatting of doc comments. For one, following https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences , but also layout question, anticipating how things will look like when rendered by `godoc` (even where `godoc` doesn't render them right now because they are for unexported types or not a doc comment at all but just a normal code comment - consistency is queen!). * Re-enabled `TestQueryLog` and `TestEndopints` (they pass now, leaving them disabled was presumably an oversight). * Bucket iterator for histogram.Histogram is now created with a method. * HistogramChunk.iterator now allows iterator recycling. (I think @dieterbe only commented it out because he was confused by the question in the comment.) * HistogramAppender.Append panics now because we decided to treat staleness marker differently. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Ganesh Vernekar	5d4dc7e413	Convert the header into an enum Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	175ef4ebcf	Add a NotCounterReset flag Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	a280b6c2da	Fix review comments Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	eb9931e961	Add info about counter resets in chunk meta Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	1dd22ed655	Support stale samples for sparse histograms (#9352 ) * Support stale samples for sparse histograms Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Don't cut a new chunk for every stale sample Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Update comments for HistoAppender.Appendable Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix review comments Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	c373200b75	Cut a new chunk on counter resets for any bucket Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	19e98e5469	Support storing the zero threshold in the histogram chunk (#9165 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	7026e6b4e4	Fix tests in histo_test.go (#9163 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8b70e87ab9	Merge remote-tracking branch 'upstream/main' into sparse-refactor Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
jinglina	1a430e5f89	remove redundant parentheses (#9134 ) Signed-off-by: jinglina <jinglinax@163.com>	3 years ago
Bryan Boreham	6788760efa	Reduce memory allocation in benchmarkIterator() (#5983 ) Previously it was allocating millions of chunks, all containing the same 250 samples. Above some ratio of CPU performance to available memory, the benchmark cannot run. Make 250 a const and just allocate one chunk which we iterate repeatedly till we reach the benchmark count. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	4fefd7520e	Skip the failing TestHistoChunkSameBuckets (#9089 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
beorn7	cb75747bce	Fix re-encoding Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	01957eee2b	Fix interjections even more Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	dc1c744169	Fix interjections at the end Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Oleg Zaytsev	40126a8494	Use binary literals for xor chunk encoding An opinionated cosmetic change, but since go 1.13 we have this fancy 0b.... literals so we don't need to write hex and comment the binary value. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	3 years ago
beorn7	deb02d59fb	Fix lint issues Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Dieter Plaetinck	dc6b068c67	bugfix: only bump numRead when all fields are successfully read Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	98f86d671a	cleanup comments Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	99ae04bb6f	add SHS chunk recoding and head cutting to head block (no tests yet) Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Ganesh Vernekar	4c01ff5194	Bunch of fixes for sparse histograms (#9043 ) * Do not panic on histoAppender.Append Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * M-map all chunks on shutdown Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Support negative schema for querying Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	6c13375ac8	sparsehistogram recoding upon detection that new buckets have appeared (#9030 ) * bucketIterator which returns all valid bucket indices for a []span Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * support for comparing []spans and generating interjections Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add license header Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * assert order fix Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * handle pathological 0-length span case more gracefully Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * stale todo Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * decode-recode histograms when new buckets appear Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * factor out recoding and also add it to the fallback case Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * make linter happy Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
beorn7	518b77c59d	Fix a few trivial style nits Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Ganesh Vernekar	f4d3af73f0	Query histograms from TSDB and unit test for append+query (#9022 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	4d27816ea5	Sparsehistogram: improve dod encoding, testing, encode chunk metadata (#9015 ) * factor out different varbit schemes and include Beorn's "optimum" for buckets Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * use more compact dod encoding scheme for SHS chunk columns Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * remove FB VB and xor dod encoding because we won't use it Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * HistoChunk metadata encoding Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add SparseHistogram.Copy() Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * histogram test: test appending a few histograms Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add license headers Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Ganesh Vernekar	04ad56d9b8	Append sparse histograms into the Head block (#9013 ) * Append sparse histograms into the Head block Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Add AtHistogram() to Iterator interface. Make HistoChunk conform to Chunk interface. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	58917d1b76	sparsehistogram: integer types and timestamp separation (#9014 ) * integer types and timestamp separation 1) unify types to int64. as suggested by beorn. we want to support counters going down (resets) even if we plan to create new chunks for now, in that case 2) histogram type doesn't know its own timestamp. include it separately in appending and iteration Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * correction: count and zeroCount to remain unsigned to make api more resilient and that's what we use in protobuf anyway Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * temp hack. Ganesh will fix Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	fd11a339a7	Sparsehistogram chunk implementation (#9009 ) * histogram chunk Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * xorAppender.AppendHistogram non-method Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * basic histogram chunk test Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
johncming	28ca0965f0	tsdb/chunkenc: fix typo of return error. (#7670 ) * tsdb/chunkenc: fix typo of return error. Signed-off-by: johncming <johncming@yahoo.com> * tsdb: fix typo of function in markdonw. Signed-off-by: johncming <johncming@yahoo.com>	4 years ago

1 2

67 Commits (f1065e44a4241097c4c2a53c44793e80d3475cf1)