prometheus

Commit Graph

Author	SHA1	Message	Date
Filip Petkovski	583f3e587c	Optimize histogram iterators (#13340 ) Optimize histogram iterators Histogram iterators allocate new objects in the AtHistogram and AtFloatHistogram methods, which makes calculating rates over long ranges expensive. In #13215 we allowed an existing object to be reused when converting an integer histogram to a float histogram. This commit follows the same idea and allows injecting an existing object in the AtHistogram and AtFloatHistogram methods. When the injected value is nil, iterators allocate new histograms, otherwise they populate and return the injected object. The commit also adds a CopyTo method to Histogram and FloatHistogram which is used in the BufferedIterator to overwrite items in the ring instead of making new copies. Note that a specialized HPoint pool is needed for all of this to work (`matrixSelectorHPool`). --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	10 months ago
Oleg Zaytsev	ed172a6667	Optimize label values with matchers by taking shortcuts (#13426 ) Don't calculate postings beforehand: we may not need them. If all matchers are for the requested label, we can just filter its values. Also, if there are no values at all, no need to run any kind of logic. Also add more labelValuesWithMatchers benchmarks Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	10 months ago
Matthieu MOREL	8f6cf3aabb	tsdb: use Go standard errors Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	12 months ago
Fiona Liao	ce126230e7	Fix chunks iterator bug when tombstone covers a whole chunk (#13209 ) When no samples are returned in a chunk because all the samples have been deleted, the chunk iterator then stops without iterating through any remaining chunks. Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com>	1 year ago
Fiona Liao	5bee0cfce2	Change `ChunkReader.Chunk()` to `ChunkOrIterable()` The ChunkReader interface's Chunk() has been changed to ChunkOrIterable(). This is a precursor to OOO native histogram support - with OOO native histograms, the chunks.Meta passed to Chunk() can result in multiple chunks being returned rather than just a single chunk (e.g. if oooMergedChunk has a counter reset in the middle). To support this, ChunkOrIterable() requires either a single chunk or an iterable to be returned. If an iterable is returned, the caller has the responsibility of converting the samples from the iterable into possibly multiple chunks. The OOOHeadChunkReader now returns an iterable rather than a chunk to prepare for the native histograms case. Also as a beneficial side effect, oooMergedChunk and boundedChunk has been simplified as they only need to implement the Iterable interface now, not the full Chunk interface. --------- Signed-off-by: Fiona Liao <fiona.y.liao@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>	1 year ago
Jeanette Tan	52eb303031	Refactor assigning MinTime in histogram chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	1 year ago
Jeanette Tan	27abf09e7f	Fix missing MinTime in histogram chunks Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	1 year ago
Goutham Veeramachaneni	86729d4d7b	Update exp package (#12650 )	1 year ago
Björn Rabenstein	f8dd8770ac	Merge pull request #12757 from bboreham/reuse-bufiter TSDB: re-use iterator when moving between series	1 year ago
Alan Protasio	959c98441b	Add context argument to tsdb.PostingsForMatchers Signed-off-by: Alan Protasio <alanprot@gmail.com>	1 year ago
zenador	69edd8709b	Add warnings (and annotations) to PromQL query results (#12152 ) Return annotations (warnings and infos) from PromQL queries This generalizes the warnings we have already used before (but only for problems with remote read) as "annotations". Annotations can be warnings or infos (the latter could be false positives). We do not treat them different in the API for now and return them all as "warnings". It would be easy to distinguish them and return infos separately, should that appear useful in the future. The new annotations are then used to create a lot of warnings or infos during PromQL evaluations. Partially these are things we have wanted for a long time (e.g. inform the user that they have applied `rate` to a metric that doesn't look like a counter), but the new native histograms have created even more needs for those annotations (e.g. if a query tries to aggregate float numbers with histograms). The annotations added here are not yet complete. A prominent example would be a warning about a range too short for a rate calculation. But such a warnings is more tricky to create with good fidelity and we will tackle it later. Another TODO is to take annotations into account when evaluating recording rules. --------- Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>	1 year ago
Arve Knudsen	156222cc50	Add context argument to LabelQuerier.LabelValues (#12665 ) Add context argument to LabelQuerier.LabelValues and LabelQuerier.SortedLabelValues. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Arve Knudsen	a964349e97	Add context argument to LabelQuerier.LabelNames (#12666 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Arve Knudsen	4451ba10b4	Add context argument to IndexReader.Postings (#12667 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Arve Knudsen	6daee89e5f	Add context argument to Querier.Select (#12660 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	1 year ago
Dimitar Dimitrov	b40865833d	PostingsForMatchers race with creating new series (#12558 ) Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>	1 year ago
Bryan Boreham	bdc7983956	TSDB: re-use iterator when moving between series Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	1 year ago
George Krajcsovits	6cd2d1621f	Hide histogram chunk append and reset header internals (#12352 ) tsdb: Hide histogram chunk append and reset header internals Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	1 year ago
Patrick Oyarzun	68e5937474	Apply relevant label matchers in LabelValues before fetching extra postings (#12274 ) * Apply matchers when fetching label values Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com> * Avoid extra copying of label values Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com> --------- Signed-off-by: Patrick Oyarzun <patrick.oyarzun@grafana.com>	1 year ago
Alan Protasio	73078bf738	Opmizing Group Regex (#12375 ) Signed-off-by: Alan Protasio <alanprot@gmail.com>	2 years ago
Alan Protasio	8c5d4b4add	Opmize MatchNotEqual (#12377 ) Signed-off-by: Alan Protasio <alanprot@gmail.com>	2 years ago
George Krajcsovits	92d6980360	Fix populateWithDelChunkSeriesIterator and gauge histograms (#12330 ) Use AppendableGauge to detect corrupt chunk with gauge histograms. Detect if first sample is a gauge but the chunk is not set up to contain gauge histograms. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>	2 years ago
Alan Protasio	c0f1abb574	MatchNotRegexp optimization Signed-off-by: Alan Protasio <alanprot@gmail.com>	2 years ago
Matthieu MOREL	bae9a21200	Merge branch 'main' into linter/nilerr Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
beorn7	5b53aa1108	style: Replace `else if` cascades with `switch` Wiser coders than myself have come to the conclusion that a `switch` statement is almost always superior to a statement that includes any `else if`. The exceptions that I have found in our codebase are just these two: * The `if else` is followed by an additional statement before the next condition (separated by a `;`). * The whole thing is within a `for` loop and `break` statements are used. In this case, using `switch` would require tagging the `for` loop, which probably tips the balance. Why are `switch` statements more readable? For one, fewer curly braces. But more importantly, the conditions all have the same alignment, so the whole thing follows the natural flow of going down a list of conditions. With `else if`, in contrast, all conditions but the first are "hidden" behind `} else if `, harder to spot and (for no good reason) presented differently from the first condition. I'm sure the aforemention wise coders can list even more reasons. In any case, I like it so much that I have found myself recommending it in code reviews. I would like to make it a habit in our code base, without making it a hard requirement that we would test on the CI. But for that, there has to be a role model, so this commit eliminates all `if else` occurrences, unless it is autogenerated code or fits one of the exceptions above. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Matthieu MOREL	fb3eb21230	enable gocritic, unconvert and unused linters Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Alex Le	1936868e9d	Allow populate block logic in compact to be overriden outside Prometheus (#11711 ) Signed-off-by: Alex Le <leqiyue@amazon.com> Signed-off-by: Alex Le <emoc1989@gmail.com>	2 years ago
Alan Protasio	6ddadd98b4	Optimization on `mergedStringIter` (#12132 ) Optimization on NewMergedStringIter Signed-off-by: Alan Protasio <alanprot@gmail.com>	2 years ago
Ganesh Vernekar	0c0c2af7f5	Do not re-encode head chunk in ChunkQuerier Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Oleg Zaytsev	de93a279a0	Shortcut postings for matchers when empty postings are selected (#11813 ) * Add more benchmark cases * Add shortcuts for empty postings Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2 years ago
Ganesh Vernekar	fd89d7892c	Merge pull request #11809 from bboreham/dont-sort-postings-values tsdb: sort values for Postings only when required	2 years ago
Bryan Boreham	e61348d9f3	tsdb/index: fast-track postings for label="" We need to special-case ""="" too, which is used in some tests to mean "everything". Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	cf92cd2688	tsdb: sort values for Postings only when required In the head and in v1 postings on disk, it makes no difference whether postings are sorted. Only for v2 does the code step through in order. So, move the sorting to where it is required, and thus skip it entirely in the head. Label values in on-disk blocks are already sorted, but `slices.Sort` is very fast on already-sorted data so we don't bother checking. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Marc Tudurí	9474610baf	Support FloatHistogram in TSDB (#11522 ) Extends Appender.AppendHistogram function to accept the FloatHistogram. TSDB supports appending, querying, WAL replay, for this new type of histogram. Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	1848623c77	tsdb: re-use iterator when stepping through chunks Saves memory allocations, hence reduces garbage-collection overheads. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	10b27dfb84	Simplify IndexReader.Series interface Instead of passing in a `ScratchBuilder` and `Labels`, just pass the builder and the caller can extract labels from it. In many cases the caller didn't use the Labels value anyway. Now in `Labels.ScratchBuilder` we need a slightly different API: one to assign what will be the result, instead of overwriting some other `Labels`. This is safer and easier to reason about. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	543c318ec2	Update package tsdb for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	d3d96ec887	tsdb/index: use ScratchBuilder to create Labels This necessitates a change to the `tsdb.IndexReader` interface: `index.Reader` is used from multiple goroutines concurrently, so we can't have state in it. We do retain a `ScratchBuilder` in `blockBaseSeriesSet` which is iterator-like. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	0853250695	Review feedback Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	f0866c0774	tsdb: optimise block series iterators Re-use previous memory if it is already of the correct type. Also turn two levels of function closure into a single object that holds the required data. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	3c7de69059	storage: allow re-use of iterators Patterned after `Chunk.Iterator()`: pass the old iterator in so it can be re-used to avoid allocating a new object. (This commit does not do any re-use; it is just changing all the method signatures so re-use is possible in later commits.) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	db99fc43e4	Merge pull request #11632 from bboreham/improve-bbss tsdb: improve blockBaseSeriesSet scan	2 years ago
Ganesh Vernekar	72a48321da	Merge pull request #11633 from pstibrany/populate-error Enhance "cannot populate chunk" error message to include source block ID	2 years ago
Julien Pivotto	0372e259ba	Merge pull request #11634 from prometheus/release-2.40 Merge release-2.40 branch into main	2 years ago
Peter Štibraný	af838ccf83	Include source block in error message when loading chunk fails. Signed-off-by: Peter Štibraný <pstibrany@gmail.com>	2 years ago
Bryan Boreham	1226922ff5	tsdb: improve blockBaseSeriesSet scan Inverting the test for chunks deleted by tombstones makes all three rejections consistent, and also avoids the case where a chunk is excluded but still causes `trimFront` or `trimBack` to be set. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	0c05f95e92	tsdb: use smaller allocation in blockBaseSeriesSet This reduces garbage, hence goes faster, when a short time range is required compared to the amount of chunks in the block. For example recording rules and alerts often look only at the last few minutes. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	ad79fb9f25	Do not error on empty chunk during iteration in populateWithDelChunkSeriesIterator Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Jesus Vazquez	e934d0f011	Merge 'main' into sparsehistogram Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>	2 years ago

1 2 3

109 Commits (88d07c43cb12299ee2f6f1279133d8057931f521)