prometheus

Commit Graph

Author	SHA1	Message	Date
Łukasz Mierzwa	277f04f0c4	Stop compactions if there's a block to write (#13754 ) * Stop compactions if there's a block to write db.Compact() checks if there's a block to write with HEAD chunks before calling db.compactBlocks(). This is to ensure that if we need to write a block then it happens ASAP, otherwise memory usage might keep growing. But what can also happen is that we don't need to write any block, we start db.compactBlocks(), compaction takes hours, and in the meantime HEAD needs to write out chunks to a block. This can be especially problematic if, for example, you run Thanos sidecar that's uploading block, which requires that compactions are disabled. Then you disable Thanos sidecar and re-enable compactions. When db.compactBlocks() is finally called it might have a huge number of blocks to compact, which might take a very long time, during which HEAD cannot write out chunks to a new block. In such case memory usage will keep growing until either: - compactions are finally finished and HEAD can write a block - we run out of memory and Prometheus gets OOM-killed This change adds a check for pending HEAD block writes inside db.compactBlocks(), so that we bail out early if there are still compactions to run, but we also need to write a new block. Also add a test for compactBlocks. --------- Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com> Signed-off-by: Lukasz Mierzwa <lukasz@cloudflare.com>	8 months ago
Bryan Boreham	fc567a1df8	Merge pull request #13889 from komisan19/refactor/utilize_standard_functions_max/min refactor: utilize standard functions max/min in promtool and tsdb	8 months ago
Arthur Silva Sens	b4a973753c	Merge pull request #13897 from dashpole/unregister_scrape_metrics	8 months ago
David Ashpole	c755fa9935	support unregistering scrape manager metrics Signed-off-by: David Ashpole <dashpole@google.com>	8 months ago
Bryan Boreham	2278d2377c	Merge pull request #13744 from bboreham/wip-aggr-index [ENHANCEMENT] PromQL: Re-structure aggregations for clarity and performance	8 months ago
Bryan Boreham	12961c6a37	promql: refactor: eliminate one 'else' Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	0ac927515b	promql: move group-seen into group struct Save allocating an auxilliary array. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	7499d90913	promql: remove pointer to aggregation groups Just allocate in one slice. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	cfbeb6681b	promql: re-use one heap for topk and bottomk Slightly ugly casting saves memory. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	5e3914a27c	promql: remove histogramMean from groupedAggregation Re-use histogramValue since we don't need them separately. Tidy up initialization. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	2cf3c9de8f	promql: store labels per-group only for count_values This saves memory in other kinds of aggregation. We don't need `orderedResult` in `aggregationCountValues`; the ordering is not guaranteed. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	185290a0d2	promql: pull checking of q and k out of loop Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	4584f67e17	promql: inline nextSample function Move Sample out of loop to reduce allocations, otherwise it escapes to the heap. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	526ce4ee7a	promql: simplify data collection in aggregations We don't need a Sample, just the float and histogram values. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	2f03acbafc	promql: refactor: split topk/bottomk from sum/avg/etc They aggregate results in different ways. topk/bottomk don't consider histograms so can simplify data collection. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	74eed67ef6	promql: refactor: pull fetching input data out of rangeEvalAgg This is a cleaner split of responsibilities. We now check the sample count after calling rangeEvalAgg. Changed re-use of samples to use `Clone` and `defer`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	602eb69edf	promql: refactor: extract function nextSample With sub-function nextValues which we shall use shortly. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	eb41e770b7	promql: refactor: extract function addToSeries Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	53a3138eeb	promql aggregations: pre-generate mapping from inputs to outputs So we don't have to re-create it on every time step. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	cb6c4b3092	promql: simplify k/q parameter to topk/bottomk/quantile Pass it as a float64 not as interface{}. Make k a simple int, since that is the parameter to make(). Pull invalid quantile warning out of the loop. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	b3bda7df4b	promql: aggregations: skip copying input to a Vector We can work directly from the inputMatrix on each timestep. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	c9b6c4c55a	promql: aggregations: output directly to matrix for instant queries Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	3851b74db1	promql: aggregations: skip result vector in range queries Adjust test to match the lower count, since samples in the vector are no longer counted. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	59548b8a0b	promql: refactor: move collection of results into aggregation() We don't need to check for duplicates as aggregation cannot generate them. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	bd9bdccb22	promql: refactor: simplify internal data structures Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	5f10d17cef	promql: refactor: split out aggregations over range The new function `rangeEvalAgg` is mostly a copy of `rangeEval`, but without `initSeries` which we don't need and inlining the callback to `aggregation()`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	e5f667537c	promql: refactor: initialize aggregation before storing in map This seems more consistent to me. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	29244fb841	promql: refactor: extract count_values implementation The existing aggregation function is very long and covers very different cases. `aggregationCountValues` is just for `count_values`, which differs from other aggregations in that it outputs as many series per group as there are values in the input. Remove the top-level switch on string parameter type; use the same `Op` check there as elswehere. Pull checking parameters out to caller, where it is only executed once. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Bryan Boreham	8e04ab6dd4	promql: refactor: extract generateGroupingLabels function Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
David Ashpole	bbfc72b4e2	support unregistering discovery manager metrics (#13896 ) Signed-off-by: David Ashpole <dashpole@google.com>	8 months ago
Julien	8b72ed77f8	Merge pull request #13869 from prometheus/dependabot/go_modules/go-opentelemetry-io-1393210b43 build(deps): bump the go-opentelemetry-io group with 3 updates	8 months ago
Julien	8eb9228af8	Merge pull request #13864 from yeya24/expose-compactor-metrics Expose compactor metrics	8 months ago
Julien	48c8ec19bc	Merge pull request #13882 from prometheus/update-featureflag-docs Update documentation about existing feature-flags	8 months ago
dandrucz	38b75bc0d7	Linode IPv6 Range support, Optional region filtering, Missing fields in Documentation fixed (#13774 ) * Add support for discovering Linode IPv6 ranges associated with linodes. * Add optional but recommended region filtering (faster queries, more relevant information). * Added missing fields in configuration.md, updated linode test cases. * Convert to TableDrivenTests as per tjhop request. Signed-off-by: David Andruczyk <dandrucz@akamai.com>	8 months ago
Jonathan Halterman	113938aeb8	Log out of order when writing a block (#13888 ) Signed-off-by: Jonathan Halterman <jonathan@grafana.com>	8 months ago
Bryan Boreham	8799581b24	Merge pull request #13554 from machine424/k8s-failures discovery(k8s): add metric prometheus_sd_kubernetes_failures_total	8 months ago
komisan19	0249e080b4	refactor: utilize standard functions max/min Signed-off-by: komisan19 <18901496+komisan19@users.noreply.github.com>	8 months ago
Bryan Boreham	31491eb37c	Merge pull request #13885 from bboreham/readable-test [TESTS] Truncate some long test names, for readability	8 months ago
Bryan Boreham	7c28521451	[TESTS] Truncate some long test names, for readability The strings produced by these tests can run to thousands of characters, which makes test logs difficult to read. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	8 months ago
Charles Korn	cd72ebb05f	promql: include more details in error message when creating test query fails or an unexpected series is returned (#13847 ) * promql: include more details in error message when creating test query fails Signed-off-by: Charles Korn <charles.korn@grafana.com> * Include more details when an unexpected metric is returned Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com>	8 months ago
Nicolas Takashi	8125634086	[refactor] moving mergedOOOChunks Iterator (#13881 ) Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com>	8 months ago
Arthur Silva Sens	db64d2dcdc	Update documentation about existing feature-flags Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>	8 months ago
Julius Volz	9b7de47787	Remove unused Dmn field on EvalNodeHelper (#13877 ) https://github.com/prometheus/prometheus/pull/13446 removed the last usage of this field, but didn't remove the field. Signed-off-by: Julius Volz <julius.volz@gmail.com>	8 months ago
dependabot[bot]	2e6c1c35a4	build(deps): bump the go-opentelemetry-io group with 3 updates Bumps the go-opentelemetry-io group with 3 updates: [go.opentelemetry.io/collector/featuregate](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/pdata](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector). Updates `go.opentelemetry.io/collector/featuregate` from 1.3.0 to 1.4.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.3.0...pdata/v1.4.0) Updates `go.opentelemetry.io/collector/pdata` from 1.3.0 to 1.4.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.3.0...pdata/v1.4.0) Updates `go.opentelemetry.io/collector/semconv` from 0.96.0 to 0.97.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.96.0...v0.97.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/collector/featuregate dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/pdata dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/semconv dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io ... Signed-off-by: dependabot[bot] <support@github.com>	8 months ago
Augustin Husson	9e2c335bab	Merge pull request #13855 from prometheus/merge-2.51-into-main Merge 2.51.1 into main	8 months ago
carehabit	a672662073	all: fix some typos (#13863 ) Signed-off-by: carehabit <shenyuting@outlook.com>	8 months ago
Bryan Boreham	5e2caa32e7	Merge pull request #13856 from bboreham/release-ci-needs CI: Publish step should require all Go tests to pass	8 months ago
Ben Ye	ded35ef20d	expose compactor metrics Signed-off-by: Ben Ye <benye@amazon.com>	8 months ago
Arthur Silva Sens	b51bbdd7ad	Merge pull request #13862 from nicolastakashi/refactor/moving-mergedOOOChunks-struct [refactor] moving mergedOOOChunks to ooo_head_read	8 months ago
Nicolas Takashi	79d6750364	Merge branch 'prometheus:main' into refactor/moving-mergedOOOChunks-struct	8 months ago

1 2 3 4 5 ...

12643 Commits (277f04f0c44219bb4158b021c7038015410c9745) All Branches Search

12643 Commits (277f04f0c44219bb4158b021c7038015410c9745)

All Branches