Commit Graph

14695 Commits (ee18650636ded348348f648dc3edd90633dd0480)

Author SHA1 Message Date
Bryan Boreham 8ea2c6adce Prepare release 2.55.0-rc.1
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-10-16 13:56:24 +01:00
Arve Knudsen de16f5e387
[FEATURE] PromQL: Add experimental info function MVP (#14495)
The `info` function is an experiment to improve UX
around including labels from info metrics.
`info` has to be enabled via the feature flag `--enable-feature=promql-experimental-functions`.

This MVP of info simplifies the implementation by assuming:
* Only support for the target_info metric
* That target_info's identifying labels are job and instance

Also:
* Encode info samples' original timestamp as sample value
* Deduce info series select hints from top-most VectorSelector

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Ying WANG <ying.wang@grafana.com>
Co-authored-by: Augustin Husson <augustin.husson@amadeus.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
2024-10-16 13:52:11 +01:00
akunszt 08a7162502
discovery: aws/ec2 unit tests (#14364)
* discovery: add aws/ec2 unit tests 

* discovery: initial skeleton for aws/ec2 unit tests

This is a - very likely - not too useful unit test for the AWS SD. It is
commited so other people can check the basic logic and the
implementation.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix linter complains about ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: add basic unit test for aws

This tests only the basic labelling, not including the VPC related
information.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix linter complains about ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: other linter fixes in aws/ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: implement remaining tests for aws/ec2

The coverage is not 100% but I think it is a good starting point if
someone wants to improve that.

Currently it covers all the AWS API calls.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make linter happy in aws/ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make utility funtcions private

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discover: no global variable in the aws/ec2 test

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: common body for some tests in ec2

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: try to make golangci-lint happy

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make every non-test function private

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: test for errors first in TestRefresh

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: move refresh tests into the function

This way people can find both the test cases and the execution of the
test at the same place.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix copyright date

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: remove misleading comment

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: rename test for easier identification

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: use static values for the test cases

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discover: try to make the linter happy

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: drop redundant data from ec2 and use common ptr functions

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: use Error instead of Equal

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: merge refreshAZIDs tests into one

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

---------

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
2024-10-16 14:36:37 +02:00
Bryan Boreham 249969d7ff
Merge pull request #15168 from aknuds1/arve/refactor-range-eval
[Release 2.55] Refactor evaluator.rangeEval by splitting out gatherVector method
2024-10-16 13:19:36 +01:00
Arve Knudsen e05e97cdd7 evaluator.rangeEval: Split out gatherVector method
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-16 14:01:03 +02:00
Bryan Boreham 04ec65b74a
Merge pull request #15166 from aknuds1/arve/upgrade-enterprise-certificate-proxy
[Release 2.55] [BUILD] Upgrade github.com/googleapis/enterprise-certificate-proxy to v0.3.4
2024-10-16 12:52:25 +01:00
George Krajcsovits 55419275ab
Merge pull request #15159 from prometheus/nhcb-scrape-optimize1
convertnhcb: use CutSuffix instead of regex replace for histogram name
2024-10-16 10:25:18 +02:00
machine424 b1c356beea
fix(discovery): Handle cache.DeletedFinalStateUnknown in node informers' DeleteFunc
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-16 10:20:37 +02:00
Joshua Hesketh 2294712fe8 Update CHANGELOG
Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
2024-10-16 15:11:01 +11:00
Joshua Hesketh 31d19381f6 Clamp functions should ignore native histograms
As per the documentation, native histograms are skipped. This is in line
with other `simpleFunc`'s.

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
2024-10-16 15:10:54 +11:00
machine424 ab2475c426
test(tsdb): add a reproducer for https://github.com/prometheus/prometheus/issues/14422
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-15 20:39:25 +02:00
Arve Knudsen 90cc7e5723 Upgrade github.com/googleapis/enterprise-certificate-proxy to v0.3.4
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-15 16:37:58 +02:00
Bryan Boreham e7f23c049f
Merge pull request #15163 from prometheus/backport-14900-2-55
[Release 2.55] [REFACTOR] promql.Engine: Refactor vector selector evaluation into a method
2024-10-15 15:33:32 +01:00
Arve Knudsen f7b396a1dc promql.Engine: Refactor vector selector evaluation into a method (#14900)
New method is named `evalVectorSelector`.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-15 14:57:54 +01:00
Bryan Boreham e26a3dba16
Merge pull request #15162 from prometheus/cp-15210-2-55
[Release 2.55] [BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
2024-10-15 14:52:07 +01:00
Bryan Boreham 105c692f77 [BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
Because we are reimplementing the `IndexReader` to fetch in-order and
out-of-order chunks together, we must reproduce the behaviour of
`Head.indexRange()`, which floors the minimum time queried at `head.MinTime()`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-10-15 14:23:05 +01:00
Neeraj Gartia d4b1f9eb33
Corrects the behaviour of binary opperators between histogram and float (#14726)
promql: corrects binary operators functioning for mixed sample with histogram and float

For invalid pairings of sample types, an annotation is added now.

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-10-15 14:44:36 +02:00
Bryan Boreham 1e1f6ab9df
Merge pull request #15120 from bboreham/floor-ino-mint
[BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
2024-10-15 10:27:38 +01:00
Arthur Silva Sens 911c3efb96
Merge pull request #14976 from prometheus/prep-3.0-beta.1
Prepare 3.0.0-beta.1
2024-10-14 12:44:41 -03:00
György Krajcsovits 78de9bd10f convertnhcb: use CutSuffix instead of regex replace for histogram name
This is much quicker.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 16:50:05 +02:00
György Krajcsovits d930648afb Add doc string for NHCBParser
Very high level since we'll do a lot of optimizations so not
worth going into details at this time.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 14:35:11 +02:00
György Krajcsovits c13585665d Merge branch 'main' into nhcb-scrape-impl
# Conflicts:
#	promql/promqltest/test.go
#	util/convertnhcb/convertnhcb.go
2024-10-14 14:26:11 +02:00
George Krajcsovits 522149a2ae
model: move classic NHCB conversion into its own file (#15156)
* model: move classic to NHCB conversion into its own file

In preparation for #14978.

Author: Jeanette Tan <jeanette.tan@grafana.com>  2024-07-03 11:56:48
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Better naming from review comment

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Add doc strings.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-10-14 14:23:11 +02:00
Manik Rana c3b8582cf4
[PERF] textparse: optimize OM `p.isCreatedSeries()` (#15150)
* refac: remove p.Metric usage

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* perf: use bytes instead of strings

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <Manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-14 10:18:15 +01:00
George Krajcsovits a1700aab3a
Apply suggestions from code review
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-14 11:13:58 +02:00
György Krajcsovits 0a40a09da5 Use const instead of -53
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 11:09:03 +02:00
George Krajcsovits b8867f8ead
Merge pull request #15142 from krajorama/fix-appendhistogram-race
bugfix: data race in head.Appender.AppendHistogram and Commit
2024-10-14 08:13:39 +02:00
M Viswanath Sai 16bba78f15
discovery: Improve Azure test coverage to 50% (#14586)
* azure sd: separate refresh and refreshAzure
* azure sd: create a client with mocked servers for tests
* add test for refresh function

---------

Signed-off-by: mviswanathsai <mviswanath.sai.met21@itbhu.ac.in>
2024-10-13 10:24:51 +02:00
dependabot[bot] 7ce9cfa6e4
Bump the k8s-io group with 3 updates (#15030)
Bumps the k8s-io group with 3 updates: [k8s.io/api](https://github.com/kubernetes/api), [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) and [k8s.io/client-go](https://github.com/kubernetes/client-go).


Updates `k8s.io/api` from 0.31.0 to 0.31.1
- [Commits](https://github.com/kubernetes/api/compare/v0.31.0...v0.31.1)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.31.1
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.31.1)

Updates `k8s.io/client-go` from 0.31.0 to 0.31.1
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.31.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
- dependency-name: k8s.io/apimachinery
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-12 10:40:23 +02:00
dependabot[bot] 1e3953ab13
Bump github.com/docker/docker (#15033)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 27.2.0+incompatible to 27.3.1+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v27.2.0...v27.3.1)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-12 10:38:57 +02:00
Oleg Zaytsev 50ef0dc954
Fix `MemPostings.Add` and `MemPostings.Get` data race (#15141)
* Tests for Mempostings.{Add,Get} data race
* Fix MemPostings.{Add,Get} data race

We can't modify the postings list that are held in MemPostings as they
might already be in use by some readers.

* Modify BenchmarkHeadStripeSeriesCreate to have common labels

If there are no common labels on the series, we don't excercise the
ordering part of MemSeries, as we're just creating slices of one element
for each label value.

---------

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-10-11 15:21:15 +02:00
Julien e6f9e2dde3
Merge pull request #15119 from roidelapluie/dochttp
docs: extract HTTP client option documentation in their own sections
2024-10-11 13:10:34 +02:00
Ayoub Mrini d9a54284f5
Merge pull request #14987 from machine424/yafix
fix(notifier): avoid dropping known alertmanagers after each ApplyConfig
2024-10-10 21:14:35 +02:00
György Krajcsovits bb70370d72 TSDB head: fix race between AppendHistogram and Commit
Move writing memSeries lastHistogramValue and lastFloatHistogramValue
after series creation under lock.

The resulting code isn't totally correct in the sense that we're setting
these values before Commit() , so they might be overwritten/rolled back
later.

Also Append of stale sample checks the values without lock, so there's
still a potential race.

The correct solution would be to set these only in Commit() which we
actually do, but then Commit() would also need to process samples in
order and not floats first, then histograms, then float histograms - which
leads to not knowing what stale marker to write for histograms.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-10 16:59:15 +02:00
György Krajcsovits 631fadc4ca Unit test for data race in head.Appender.AppendHistogram
Two Appenders race when creating a series with a native histogram
as the memSeries will be common and the lastHistogram field is written
without lock.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-10 14:10:07 +02:00
Manik Rana 032ca9ef96
[PERF] textparse: further optimzations for OM `CreatedTimestamps` (#15097)
* feat: Added more tests; some changes/optimizations when pair-programming with Bartek.

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: imports

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: gofumpt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: use an efficient replacement to p.Metric

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: reduce mem allocs + comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: gofumpt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: use single quotes

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: rename

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: rename to seriesHash

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: switch condition order

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: switch condition order

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* feat: stronger checking

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: fmt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: pass pointer of buf into seriesHash()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-10 12:01:13 +01:00
Björn Rabenstein 8b545bab2f
Merge pull request #15026 from huochexizhan/main
fix: fix slice init length
2024-10-10 12:55:28 +02:00
Björn Rabenstein 720a2599a6
Merge pull request #15135 from prometheus/beorn7/docs
docs: Update chunk layot for NHCB
2024-10-10 12:00:51 +02:00
huochexizhan ff16fa1827 fix: fix slice init length
Signed-off-by: huochexizhan <huochexizhan@outlook.com>
2024-10-10 16:14:40 +08:00
György Krajcsovits 9b5d7287bb Use labels hash to determine change in metric like CT
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 15:16:46 +02:00
György Krajcsovits 14f92319d9 Add basic benchmark cases for NHCB over OM
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 15:16:27 +02:00
beorn7 12c39d5421 docs: Some nitpicking in chunks.md
- `float histogram` → `floathistogram`, as it is used in the code.
- Actual link encodings to the code (to find the actual numerical values).
- `<bytes>` → `<data>` for consistency.

Signed-off-by: beorn7 <beorn@grafana.com>
2024-10-09 14:32:12 +02:00
beorn7 a4cb52ff15 docs: Update chunk layot for NHCB
Signed-off-by: beorn7 <beorn@grafana.com>
2024-10-09 14:19:20 +02:00
Björn Rabenstein 02d0de9987
Merge pull request #14997 from fionaliao/fl/update-format-docs
Update chunk format docs with native histograms and OOO
2024-10-09 13:29:01 +02:00
György Krajcsovits 530e9514b7 Fix case of keeping classic series
Make it more obvious that the code in if had side effect.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 13:08:54 +02:00
Björn Rabenstein 5e987e7c7d
Merge pull request #15092 from jan--f/clarify-storage-contract
storage: require selectors to always return matching results
2024-10-09 13:07:59 +02:00
György Krajcsovits 8dfa733596 Fix labels handling with dedupelabels tag
Use scratch builder.
Use hash compare instead of compare by label.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 12:29:59 +02:00
György Krajcsovits 6b15791ec6 Follow the follow-up
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 11:56:54 +02:00
György Krajcsovits 4e911a3a05 Follow-up merge from main
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 11:54:49 +02:00
György Krajcsovits a2805e7934 Merge branch 'main' into nhcb-scrape-impl 2024-10-09 11:53:26 +02:00