prometheus

Commit Graph

Author	SHA1	Message	Date
Đurica Yuri Nikolić	c9b85afd93	Making the number of CPUs used for WAL replay configurable (#12066 ) Adds `WALReplayConcurrency` as an option on tsdb `Options` and `HeadOptions`. If it is not set or set <=0, then `GOMAXPROCS` is used, which matches the previous behaviour. Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>	2 years ago
ansalamdaniel	c1c444504e	Feat: metrics for head_chunks & wal folders (#12013 ) Signed-off-by: ansalamdaniel <ansalam.daniel@infracloud.io>	2 years ago
Rens Groothuijsen	d33eb3ab17	Automatically remove incorrect snapshot with index that is ahead of WAL (#11859 ) Signed-off-by: Rens Groothuijsen <l.groothuijsen@alumni.maastrichtuniversity.nl> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	f34b2cede3	Remove microbenchmarks These benchmarks are all testing things related to what Prometheus does, so perhaps have some historical interest, but we should not retain them in the main repo. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	66da1d51fd	Merge pull request #12003 from codesome/redundant-chunk-access Remove unnecessary chunk fetch in Head queries	2 years ago
Ganesh Vernekar	d504c950a2	Remove unnecessary chunk fetch in Head queries `safeChunk` is only obtained from the `headChunkReader.Chunk` call where the chunk is already fetched and stored with the `safeChunk`. So, when getting the iterator for the `safeChunk`, we don't need to get the chunk again. Also removed a couple of unnecessary fields from `safeChunk` as a part of this. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Vishal N	96ba6831ae	Observe delta in seconds prometheus_tsdb_sample_ooo_delta Signed-off-by: Vishal Nadagouda <vishalmn1996@gmail.com>	2 years ago
Justin Lei	af1d9e01c7	Refactor tsdbutil for tests/native histograms (#11948 ) * Add float histograms to ChunkFromSamplesGeneric Signed-off-by: Justin Lei <justin.lei@grafana.com> * Add GenerateSamples functions to tsdbutil Signed-off-by: Justin Lei <justin.lei@grafana.com> PR responses Signed-off-by: Justin Lei <justin.lei@grafana.com> --------- Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
George Krajcsovits	1f0cc09579	Export single ith test histogram generation functions (#11911 ) * Export single ith test histogram generation functions Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Do not set counter reset hint for non-gauge histograms individually Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Apply suggestions from code review Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com> --------- Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	8e8b718365	Merge pull request #11858 from fayzal-g/fix-chunks-metrics tsdb: when reading WAL, correctly update chunksRemoved and chunks metrics	2 years ago
beorn7	1cfc8f65a3	histograms: Return actually useful counter reset hints This is a bit more conservative than we could be. As long as a chunk isn't the first in a block, we can be pretty sure that the previous chunk won't disappear. However, the incremental gain of returning NotCounterReset in these cases is probably very small and might not be worth the code complications. Wwith this, we now also pay attention to an explicitly set counter reset during ingestion. While the case doesn't show up in practice yet, there could be scenarios where the metric source knows there was a counter reset even if it might not be visible from the values in the histogram. It is also useful for testing. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	57c18420ab	histograms: General readability tweaks - Adjust doc comments to go1.19 style. - Break down some overly long lines. - Minor doc comment tweaks and fixes. - Some renaming. Some rationales for the last point: I have renamed “interjections” into “inserts”, mostly because it is shorter, and the word shows up a lot by now (and the concept is cryptic enough to not obfuscate it even more with abbreviations). I have also tried to find more descriptive naming for the “compare spans” functions. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
fayzal-g	cfa4ea53cc	Correctly update chunksRemoved and chunks metrics Signed-off-by: fayzal-g <fayzal.ghantiwala@grafana.com>	2 years ago
Ganesh Vernekar	6e560fe19b	tsdb: Avoid unnecessary allocation from 11779 Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Mingjie Shao	78d3c4e823	tsdb: Fixed typo in Histogram Signed-off-by: Mingjie Shao <com.jerryshao@jerryshao.com>	2 years ago
Ganesh Vernekar	cb2be6e62f	Merge pull request #11779 from codesome/memseries-ooo tsdb: Only initialise out-of-order fields when required	2 years ago
Jesus Vazquez	136956cca4	Attempt to append ooo sample at the end first (#11615 ) This is an optimization on the existing append in OOOChunk. What we've been doing so far is find the place inside the out-of-order slice where the new sample should go in and then place it there and move any samples to the right if necessary. This is OK but requires a binary search every time the slice is bigger than 0. The optimization is opinionated and suggests that although out-of-order samples can be out-of-order amongst themselves they'll probably be in order thus we can probably optimistically append at the end and if not do the binary search. OOOChunks are capped to 30 samples by default so this is a small optimization but everything adds up, specially if you handle many active timeseries with out-of-order samples. Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Signed-off-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Marc Tudurí	721f33dbb0	histograms: Add remote-write support for Float Histograms (#11817 ) * adapt code.go and write_handler.go to support float histograms * adapt watcher.go to support float histograms * wip adapt queue_manager.go to support float histograms * address comments for metrics in queue_manager.go * set test cases for queue manager * use same counts for histograms and float histograms * refactor createHistograms tests * fix float histograms ref in watcher_test.go * address PR comments Signed-off-by: Marc Tuduri <marctc@protonmail.com>	2 years ago
Sebastian Rabenhorst	c057318578	agent: native histogram support (#11842 ) Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com>	2 years ago
Ganesh Vernekar	38fa151a7c	tsdb: Only initialise out-of-order fields when required Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
beorn7	6dcd03dbf3	tsdb: Add integer gauge histogram support This follows what #11783 has done for float gauge histograms. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Ganesh Vernekar	57bcbf1888	Merge pull request #11783 from codesome/gauge-histogram tsdb: Add gauge histogram support	2 years ago
Ganesh Vernekar	3c2ea91a83	tsdb: Test gauge float histograms Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	609b12d719	tsdb: Support gauge float histogram with recoding of chunk Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	8ad0d2d5d7	tsdb: Find union of two sets of histogram spans Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	d7f5129042	tsdb: Add logic to determine appendable gauge float histograms This is to check if a gauge histogram can be appended to the given chunk. If not, it tells what changes to make to the chunk and the histogram if possible. Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	a87e7e9e33	tsdb: Add counter reset hint to histograms and support in WAL Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Oleg Zaytsev	de93a279a0	Shortcut postings for matchers when empty postings are selected (#11813 ) * Add more benchmark cases * Add shortcuts for empty postings Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	2 years ago
Ganesh Vernekar	fd89d7892c	Merge pull request #11809 from bboreham/dont-sort-postings-values tsdb: sort values for Postings only when required	2 years ago
Ganesh Vernekar	c94a41c4b2	Merge pull request #11785 from Fish-pro/erroris Use errors.Is to check for a specific error	2 years ago
György Krajcsovits	97626c9583	Fix comment Comment was not updated when code changed from labels to builder in #11717 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>	2 years ago
Björn Rabenstein	c49a28bb97	Merge pull request #11782 from codesome/floatappendabletest tsdb: Improve TestFloatHistogramChunkAppendable and TestHistogramChunkAppendable	2 years ago
Bryan Boreham	e61348d9f3	tsdb/index: fast-track postings for label="" We need to special-case ""="" too, which is used in some tests to mean "everything". Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	cf92cd2688	tsdb: sort values for Postings only when required In the head and in v1 postings on disk, it makes no difference whether postings are sorted. Only for v2 does the code step through in order. So, move the sorting to where it is required, and thus skip it entirely in the head. Label values in on-disk blocks are already sorted, but `slices.Sort` is very fast on already-sorted data so we don't bother checking. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	7ed1ddb338	tsdb: Improve TestHistogramChunkAppendable and add new cases Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	fa0f04bbc6	Merge pull request #11805 from bboreham/fix-benchmark-intersect tsdb/index: fix BenchmarkIntersect to do work on each loop	2 years ago
Bryan Boreham	3da2c99ffd	tsdb/index: don't call ExpandPostings in a benchmark This allocates memory for all the returned values, which skews the result. We aren't trying to benchmark `ExpandPostings`, so just step through all the values without storing them to consume them. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	4931983ca9	tsdb/index: make BenchmarkIntersect do work on each loop Previously all the postings constructed were consumed on the first iteration, so subsequent iterations did no work. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Fish-pro	6ed71a229e	Use errors.Is to check for a specific error Signed-off-by: Fish-pro <zechun.chen@daocloud.io>	2 years ago
Ganesh Vernekar	b42802af9a	tsdb: Improve TestFloatHistogramChunkAppendable and add new cases Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	c155c0e312	tsdb: Test staleness handling of FloatHistogram Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	2820e327db	tsdb: Add staleness handling for FloatHistogram Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	e555469ba1	tsdb: Remove isHistogramSeries from memSeries Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Marc Tudurí	9474610baf	Support FloatHistogram in TSDB (#11522 ) Extends Appender.AppendHistogram function to accept the FloatHistogram. TSDB supports appending, querying, WAL replay, for this new type of histogram. Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	1848623c77	tsdb: re-use iterator when stepping through chunks Saves memory allocations, hence reduces garbage-collection overheads. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ccea61c7bf	Merge pull request #11717 from bboreham/labels-abstraction Add and use abstractions over labels.Labels	2 years ago
Ganesh Vernekar	6fd89a6fd2	Add chunk encoding for float histogram (#11716 ) Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Marc Tudurí <marctc@protonmail.com>	2 years ago
Bryan Boreham	10b27dfb84	Simplify IndexReader.Series interface Instead of passing in a `ScratchBuilder` and `Labels`, just pass the builder and the caller can extract labels from it. In many cases the caller didn't use the Labels value anyway. Now in `Labels.ScratchBuilder` we need a slightly different API: one to assign what will be the result, instead of overwriting some other `Labels`. This is safer and easier to reason about. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	4b6a4d1425	Update package tsdb tests for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ce2cfad0cb	Update package tsdb/record for new labels.Labels type Implement decoding via labels.ScratchBuilder, which we retain and re-use to reduce memory allocations. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago

1 2 3 4 5 ...

691 Commits (5f3a02f1e2648aef82f578d7d55aefc0c5dae45f)