prometheus

Commit Graph

Author	SHA1	Message	Date
Matthieu MOREL	bae9a21200	Merge branch 'main' into linter/nilerr Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Björn Rabenstein	69155c6ba1	Merge pull request #12252 from prometheus/beorn7/lint Lint clean-up	2 years ago
beorn7	5b53aa1108	style: Replace `else if` cascades with `switch` Wiser coders than myself have come to the conclusion that a `switch` statement is almost always superior to a statement that includes any `else if`. The exceptions that I have found in our codebase are just these two: * The `if else` is followed by an additional statement before the next condition (separated by a `;`). * The whole thing is within a `for` loop and `break` statements are used. In this case, using `switch` would require tagging the `for` loop, which probably tips the balance. Why are `switch` statements more readable? For one, fewer curly braces. But more importantly, the conditions all have the same alignment, so the whole thing follows the natural flow of going down a list of conditions. With `else if`, in contrast, all conditions but the first are "hidden" behind `} else if `, harder to spot and (for no good reason) presented differently from the first condition. I'm sure the aforemention wise coders can list even more reasons. In any case, I like it so much that I have found myself recommending it in code reviews. I would like to make it a habit in our code base, without making it a hard requirement that we would test on the CI. But for that, there has to be a role model, so this commit eliminates all `if else` occurrences, unless it is autogenerated code or fits one of the exceptions above. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	c3c7d44d84	lint: Adjust to the lint warnings raised by current versions of golint-ci We haven't updated golint-ci in our CI yet, but this commit prepares for that. There are a lot of new warnings, and it is mostly because the "revive" linter got updated. I agree with most of the new warnings, mostly around not naming unused function parameters (although it is justified in some cases for documentation purposes – while things like mocks are a good example where not naming the parameter is clearer). I'm pretty upset about the "empty block" warning to include `for` loops. It's such a common pattern to do something in the head of the `for` loop and then have an empty block. There is still an open issue about this: https://github.com/mgechev/revive/issues/810 I have disabled "revive" altogether in files where empty blocks are used excessively, and I have made the effort to add individual `// nolint:revive` where empty blocks are used just once or twice. It's borderline noisy, though, but let's go with it for now. I should mention that none of the "empty block" warnings for `for` loop bodies were legitimate. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
gotjosh	cf230bcd18	more wordsmithing Signed-off-by: gotjosh <josue.abreu@gmail.com>	2 years ago
gotjosh	28909a4636	more worthsmithing Signed-off-by: gotjosh <josue.abreu@gmail.com>	2 years ago
gotjosh	e2a2790b2c	add more docs Signed-off-by: gotjosh <josue.abreu@gmail.com>	2 years ago
gotjosh	96b6463f25	review comments Signed-off-by: gotjosh <josue.abreu@gmail.com>	2 years ago
Đurica Yuri Nikolić	b028112331	Making the number of CPU cores used for sorting postings lists editable (#12247 ) Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>	2 years ago
Julien Pivotto	bb217dded8	Merge pull request #12269 from yeya24/add-ctx-engine-interface Add ctx to QueryEngine interface	2 years ago
gotjosh	f3394bf7a1	Rules API: Allow filtering by rule name Introduces support for a new query parameter in the `/rules` API endpoint that allows filtering by rule names. If all the rules of a group are filtered, we skip the group entirely. Signed-off-by: gotjosh <josue.abreu@gmail.com>	2 years ago
Bryan Boreham	cdf42df698	Merge pull request #12267 from bboreham/faster-decodesize labels: small optimization to stringlabels	2 years ago
Ben Ye	fd3630b9a3	add ctx to QueryEngine interface Signed-off-by: Ben Ye <benye@amazon.com>	2 years ago
Bryan Boreham	1801cd4196	labels: small optimization to stringlabels Add a fast path for the common case that a string is less than 127 bytes long, to skip a shift and the loop. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Sebastian Rabenhorst	5d4ec08a1f	Fixed sampleRingIterator for mixed histograms Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com> Fixed sampleRingIterator for mixed histograms Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com> Fixed lint	2 years ago
ianwoolf	79e4bdee8e	add Close for ActiveQueryTracker to close the file. Signed-off-by: ianwoolf <btw515wolf2@gmail.com>	2 years ago
Ganesh Vernekar	7309ac2721	Merge pull request #12257 from alexqyle/block-populator-rename Rename PopulateBlockFunc to BlockPopulator	2 years ago
Justin Lei	c3e6b85631	Reverse test changes Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Justin Lei	052993414a	Add storage.tsdb.samples-per-chunk flag Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Matthieu MOREL	0de12cb2e7	Merge branch 'prometheus:main' into patch-1	2 years ago
Matthieu MOREL	fb3eb21230	enable gocritic, unconvert and unused linters Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Björn Rabenstein	6b8573a846	Merge pull request #11687 from prometheus/beorn7/histogram histograms: Optimize query performance	2 years ago
beorn7	717a3f8e25	storage: Manually expand `genericAdd` for specific types This commit is doing what I would have expected that Go generics do for me. However, the PromQL benchmarks show a significant runtime and allocation increase with `genericAdd`, so this replaces it with hand-coded non-generic versions of it. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	551de0346f	promql: Do not return nil slices to the pool Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	817a2396cb	Name float values as "floats", not as "values" In the past, every sample value was a float, so it was fine to call a variable holding such a float "value" or "sample". With native histograms, a sample might have a histogram value. And a histogram value is still a value. Calling a float value just "value" or "sample" or "V" is therefore misleading. Over the last few commits, I already renamed many variables, but this cleans up a few more places where the changes are more invasive. Note that we do not to attempt naming in the JSON APIs or in the protobufs. That would be quite a disruption. However, internally, we can call variables as we want, and we should go with the option of avoiding misunderstandings. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	462240bc78	storage: add specialized buffers to sampleRing This utilizes the fact that most sampleRings will only contain samples of one type. In this case, the generic interface is circumvented, and a bespoke buffer for the one actually occurring sample type is used. Should a sampleRing receive a sample of a different kind later, it will transparently switch to the generic behavior. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	630bcb494b	storage: Use separate sample types for histogram vs. float Previously, we had one “polymorphous” `sample` type in the `storage` package. This commit breaks it up into `fSample`, `hSample`, and `fhSample`, each still implementing the `tsdbutil.Sample` interface. This reduces allocations in `sampleRing.Add` but inflicts the penalty of the interface wrapper, which makes things worse in total. This commit therefore just demonstrates the step taken. The next commit will tackle the interface overhead problem. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
beorn7	c0879d64cf	promql: Separate `Point` into `FPoint` and `HPoint` In other words: Instead of having a “polymorphous” `Point` that can either contain a float value or a histogram value, use an `FPoint` for floats and an `HPoint` for histograms. This seemingly small change has a _lot_ of repercussions throughout the codebase. The idea here is to avoid the increase in size of `Point` arrays that happened after native histograms had been added. The higher-level data structures (`Sample`, `Series`, etc.) are still “polymorphous”. The same idea could be applied to them, but at each step the trade-offs needed to be evaluated. The idea with this change is to do the minimum necessary to get back to pre-histogram performance for functions that do not touch histograms. Here are comparisons for the `changes` function. The test data doesn't include histograms yet. Ideally, there would be no change in the benchmark result at all. First runtime v2.39 compared to directly prior to this commit: ``` name old time/op new time/op delta RangeQuery/expr=changes(a_one[1d]),steps=1-16 391µs ± 2% 542µs ± 1% +38.58% (p=0.000 n=9+8) RangeQuery/expr=changes(a_one[1d]),steps=10-16 452µs ± 2% 617µs ± 2% +36.48% (p=0.000 n=10+10) RangeQuery/expr=changes(a_one[1d]),steps=100-16 1.12ms ± 1% 1.36ms ± 2% +21.58% (p=0.000 n=8+10) RangeQuery/expr=changes(a_one[1d]),steps=1000-16 7.83ms ± 1% 8.94ms ± 1% +14.21% (p=0.000 n=10+10) RangeQuery/expr=changes(a_ten[1d]),steps=1-16 2.98ms ± 0% 3.30ms ± 1% +10.67% (p=0.000 n=9+10) RangeQuery/expr=changes(a_ten[1d]),steps=10-16 3.66ms ± 1% 4.10ms ± 1% +11.82% (p=0.000 n=10+10) RangeQuery/expr=changes(a_ten[1d]),steps=100-16 10.5ms ± 0% 11.8ms ± 1% +12.50% (p=0.000 n=8+10) RangeQuery/expr=changes(a_ten[1d]),steps=1000-16 77.6ms ± 1% 87.4ms ± 1% +12.63% (p=0.000 n=9+9) RangeQuery/expr=changes(a_hundred[1d]),steps=1-16 30.4ms ± 2% 32.8ms ± 1% +8.01% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=10-16 37.1ms ± 2% 40.6ms ± 2% +9.64% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=100-16 105ms ± 1% 117ms ± 1% +11.69% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16 783ms ± 3% 876ms ± 1% +11.83% (p=0.000 n=9+10) ``` And then runtime v2.39 compared to after this commit: ``` name old time/op new time/op delta RangeQuery/expr=changes(a_one[1d]),steps=1-16 391µs ± 2% 547µs ± 1% +39.84% (p=0.000 n=9+8) RangeQuery/expr=changes(a_one[1d]),steps=10-16 452µs ± 2% 616µs ± 2% +36.15% (p=0.000 n=10+10) RangeQuery/expr=changes(a_one[1d]),steps=100-16 1.12ms ± 1% 1.26ms ± 1% +12.20% (p=0.000 n=8+10) RangeQuery/expr=changes(a_one[1d]),steps=1000-16 7.83ms ± 1% 7.95ms ± 1% +1.59% (p=0.000 n=10+8) RangeQuery/expr=changes(a_ten[1d]),steps=1-16 2.98ms ± 0% 3.38ms ± 2% +13.49% (p=0.000 n=9+10) RangeQuery/expr=changes(a_ten[1d]),steps=10-16 3.66ms ± 1% 4.02ms ± 1% +9.80% (p=0.000 n=10+9) RangeQuery/expr=changes(a_ten[1d]),steps=100-16 10.5ms ± 0% 10.8ms ± 1% +3.08% (p=0.000 n=8+10) RangeQuery/expr=changes(a_ten[1d]),steps=1000-16 77.6ms ± 1% 78.1ms ± 1% +0.58% (p=0.035 n=9+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1-16 30.4ms ± 2% 33.5ms ± 4% +10.18% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=10-16 37.1ms ± 2% 40.0ms ± 1% +7.98% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=100-16 105ms ± 1% 107ms ± 1% +1.92% (p=0.000 n=10+10) RangeQuery/expr=changes(a_hundred[1d]),steps=1000-16 783ms ± 3% 775ms ± 1% -1.02% (p=0.019 n=9+9) ``` In summary, the runtime doesn't really improve with this change for queries with just a few steps. For queries with many steps, this commit essentially reinstates the old performance. This is good because the many-step queries are the one that matter most (longest absolute runtime). In terms of allocations, though, this commit doesn't make a dent at all (numbers not shown). The reason is that most of the allocations happen in the sampleRingIterator (in the storage package), which has to be addressed in a separate commit. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Bartlomiej Plotka	136b48855a	Merge pull request #12259 from bboreham/labels-overwrite labels: add ScratchBuilder.Overwrite for slice implementation	2 years ago
Bryan Boreham	10cc60af01	labels: add ScratchBuilder.Overwrite for slice implementation This is a method used by some downstream projects; it was created to optimize the implementation in `labels_string.go` but we should have one for both implementations so the same code works with either. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Alex Le	01d0dda4fc	Rename PopulateBlockFunc to BlockPopulator Signed-off-by: Alex Le <leqiyue@amazon.com>	2 years ago
Łukasz Mierzwa	b6573353c1	Add query_samples_total metric query_samples_total is a counter that tracks the total number of samples loaded by all queries. The goal with this metric is to be able to see the amount of 'work' done by Prometheus to service queries. At the moment we have metrics with the number of queries, plus more detailed metrics showing how much time each step of a query takes. While those metrics do help they don't show us the whole picture. Queries that do load more samples are (in general) more expensive than queries that do load fewer samples. This means that looking only at the number of queries doesn't tell us how much 'work' Prometheus received. Adding a counter that tracks the total number of samples loaded allows us to see if there was a spike in the cost of queries, not just the number of them. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	2 years ago
Björn Rabenstein	8ed90b567b	Merge pull request #12234 from aknuds1/chore/improve-histogram-comments tsdb: Improve a couple of histogram documentation comments	2 years ago
Ben Ye	fb67d368a2	use consistent error for instant and range query 400 Signed-off-by: Ben Ye <benye@amazon.com>	2 years ago
Björn Rabenstein	6e0a46900b	Merge pull request #12192 from leizor/leizor/prometheus/issues/11204 Add support for native histograms to concreteSeriesIterator	2 years ago
Matthieu MOREL	d5b33e1419	yamllint: don’t check keys and add file extension to config file Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>	2 years ago
Arve Knudsen	cca7178a12	tsdb: Improve a couple of histogram documentation comments Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	2 years ago
Justin Lei	f90013a5a0	Update storage/remote/codec.go Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Justin Lei <97976793+leizor@users.noreply.github.com>	2 years ago
Justin Lei	83f43982c9	Add support for native histograms to concreteSeriesIterator Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Justin Lei	73ff91d182	Test fixes Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Justin Lei	c770ba8047	Add comment linking to PR Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
Justin Lei	79db04eb12	Adjust samplesPerChunk from 120 to 220 Signed-off-by: Justin Lei <justin.lei@grafana.com>	2 years ago
znley	0c75f76193	fix: RLIM_INFINITY type is uint64 on loong64 Signed-off-by: znley <shanjiantao@loongson.cn>	2 years ago
François Gouteroux	8472596fd0	fix: apply suggested changes Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>	2 years ago
Xiaochao Dong (@damnever)	2b7202c4cc	Validate the metric names and labels in the remote write handler Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>	2 years ago
François Gouteroux	034eb2b3f2	promtool: read from stdin if no filenames are provided in check rules Signed-off-by: François Gouteroux <francois.gouteroux@gmail.com>	2 years ago
Julien Pivotto	8dba9163f1	Merge pull request #12216 from prometheus/dependabot/go_modules/github.com/Azure/go-autorest/autorest/adal-0.9.23 build(deps): bump github.com/Azure/go-autorest/autorest/adal from 0.9.22 to 0.9.23	2 years ago
Julien Pivotto	391473141d	Check health & ready: move to flags (#12223 ) This makes it more consistent with other command like import rules. We don't have stricts rules and uniformity accross promtool unfortunately, but I think it's better to only have the http config on relevant check commands to avoid thinking Prometheus can e.g. check the config over the wire. Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Soon-Ping	6cecb87941	Generalized rule group iteration evaluation hook (#11885 ) Signed-off-by: Soon-Ping Phang <soonping@amazon.com>	2 years ago
Chris Marchbanks	044b3ecd14	Merge pull request #12203 from bboreham/raise-max-samples-per-send remote-write: raise default samples per send to 2,000	2 years ago

... 5 6 7 8 9 ...

11441 Commits (ad4f514e66efce3fa7e09e916925e3126bdd4eec) All Branches Search

11441 Commits (ad4f514e66efce3fa7e09e916925e3126bdd4eec)

All Branches