prometheus

Commit Graph

Author	SHA1	Message	Date
Ganesh Vernekar	19e98e5469	Support storing the zero threshold in the histogram chunk (#9165 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	7026e6b4e4	Fix tests in histo_test.go (#9163 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8b70e87ab9	Merge remote-tracking branch 'upstream/main' into sparse-refactor Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
jinglina	1a430e5f89	remove redundant parentheses (#9134 ) Signed-off-by: jinglina <jinglinax@163.com>	3 years ago
Bryan Boreham	6788760efa	Reduce memory allocation in benchmarkIterator() (#5983 ) Previously it was allocating millions of chunks, all containing the same 250 samples. Above some ratio of CPU performance to available memory, the benchmark cannot run. Make 250 a const and just allocate one chunk which we iterate repeatedly till we reach the benchmark count. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	4fefd7520e	Skip the failing TestHistoChunkSameBuckets (#9089 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
beorn7	cb75747bce	Fix re-encoding Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	01957eee2b	Fix interjections even more Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	dc1c744169	Fix interjections at the end Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Oleg Zaytsev	40126a8494	Use binary literals for xor chunk encoding An opinionated cosmetic change, but since go 1.13 we have this fancy 0b.... literals so we don't need to write hex and comment the binary value. Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	3 years ago
beorn7	deb02d59fb	Fix lint issues Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Dieter Plaetinck	dc6b068c67	bugfix: only bump numRead when all fields are successfully read Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	98f86d671a	cleanup comments Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	99ae04bb6f	add SHS chunk recoding and head cutting to head block (no tests yet) Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Ganesh Vernekar	4c01ff5194	Bunch of fixes for sparse histograms (#9043 ) * Do not panic on histoAppender.Append Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * M-map all chunks on shutdown Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Support negative schema for querying Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	6c13375ac8	sparsehistogram recoding upon detection that new buckets have appeared (#9030 ) * bucketIterator which returns all valid bucket indices for a []span Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * support for comparing []spans and generating interjections Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add license header Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * assert order fix Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * handle pathological 0-length span case more gracefully Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * stale todo Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * decode-recode histograms when new buckets appear Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * factor out recoding and also add it to the fallback case Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * make linter happy Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
beorn7	518b77c59d	Fix a few trivial style nits Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Ganesh Vernekar	f4d3af73f0	Query histograms from TSDB and unit test for append+query (#9022 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	4d27816ea5	Sparsehistogram: improve dod encoding, testing, encode chunk metadata (#9015 ) * factor out different varbit schemes and include Beorn's "optimum" for buckets Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * use more compact dod encoding scheme for SHS chunk columns Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * remove FB VB and xor dod encoding because we won't use it Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * HistoChunk metadata encoding Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add SparseHistogram.Copy() Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * histogram test: test appending a few histograms Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * add license headers Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Ganesh Vernekar	04ad56d9b8	Append sparse histograms into the Head block (#9013 ) * Append sparse histograms into the Head block Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Add AtHistogram() to Iterator interface. Make HistoChunk conform to Chunk interface. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Dieter Plaetinck	58917d1b76	sparsehistogram: integer types and timestamp separation (#9014 ) * integer types and timestamp separation 1) unify types to int64. as suggested by beorn. we want to support counters going down (resets) even if we plan to create new chunks for now, in that case 2) histogram type doesn't know its own timestamp. include it separately in appending and iteration Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * correction: count and zeroCount to remain unsigned to make api more resilient and that's what we use in protobuf anyway Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * temp hack. Ganesh will fix Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Dieter Plaetinck	fd11a339a7	Sparsehistogram chunk implementation (#9009 ) * histogram chunk Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * xorAppender.AppendHistogram non-method Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * basic histogram chunk test Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
johncming	28ca0965f0	tsdb/chunkenc: fix typo of return error. (#7670 ) * tsdb/chunkenc: fix typo of return error. Signed-off-by: johncming <johncming@yahoo.com> * tsdb: fix typo of function in markdonw. Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Marco Pracucci	3b529ddbce	Cleanup bstream_test.go based on post-merge feedback received on #7390 (#7413 ) * Fixed bstream test license Signed-off-by: Marco Pracucci <marco@pracucci.com> * Simplified bstreamReader.loadNextBuffer() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed date in license Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Marco Pracucci	f42ed03dc5	Optimized bstream reader used by XORChunk iterator (#7390 ) * Optimized bstream reader Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed linter Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added license to new file Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed type cast Signed-off-by: Marco Pracucci <marco@pracucci.com> * Changed comments Signed-off-by: Marco Pracucci <marco@pracucci.com> * Improved comments and rolledback no-op changes Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed race condition Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Bartlomiej Plotka	d5c33877f9	storage: Added Chunks{Queryable/Querier/SeriesSet/Series/Iteratable. Added generic Merge{SeriesSet/Querier} implementation. (#7005 ) * storage: Added Chunks{Queryable/Querier/SeriesSet/Series/Iteratable. Added generic Merge{SeriesSet/Querier} implementation. ## Rationales: In many places (e.g. chunk Remote read, Thanos Receive fetching chunk from TSDB), we operate on encoded chunks not samples. This means that we unnecessary decode/encode, wasting CPU, time and memory. This PR adds chunk iterator interfaces and makes the merge code to be reused between both seriesSets I will make the use of it in following PR inside tsdb itself. For now fanout implements it and mergers. All merges now also allows passing series mergers. This opens doors for custom deduplications other than TSDB vertical ones (e.g. offline one we have in Thanos). ## Changes * Added Chunk versions of all iterating methods. It all starts in Querier/ChunkQuerier. The plan is that Storage will implement both chunked and samples. * Added Seek to chunks.Iterator interface for iterating over chunks. * NewMergeChunkQuerier was added; Both this and NewMergeQuerier are now using generigMergeQuerier to share the code. Generic code was added. * Improved tests. * Added some TODO for further simplifications in next PRs. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Moved s/Labeled/SeriesLabels as per Krasi suggestion. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Krasi's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Second iteration of Krasi comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Another round of comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Peter Štibraný	1d396b96dc	Specify that returned samples must be ordered by timestamp. (#6877 ) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Bartlomiej Plotka	59c9d6ef45	Addressed Brian's comments, moved metrics to main.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	5d84e5d895	Make chunkenc.Iterator.At behaviour unspecified without Next done. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	cfba92a133	Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	849faa407b	Minor fixes. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	2cf637fbf5	Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	34426766d8	Unify Iterator interfaces. All point to storage now. This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Marco Pracucci	699f3e8f4d	Added comments to the Chunk interface Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Marco Pracucci	0703dae7cc	Compact TSDB head chunks after being cut, to reduce inuse memory Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
yuxiaobo	47e51c8b2b	Correct spelling mistakes Signed-off-by: yuxiaobo <yuxiaobogo@163.com>	5 years ago
Bartek Plotka	f0863a604e	Removed extra tsdb/testutil after merge. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	5 years ago
Ganesh Vernekar	5ecef3542d	Cleanup after merging tsdb into prometheus Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ganesh Vernekar	7cf09b0395	Moving tsdb into its own subdirectory Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago

1 2 3

141 Commits (bafb4d68de6a876873f105babff6aeda4b4e3354)