Commit Graph

868 Commits (1bd16db101933f70308beb5b85e7fa3968a5228c)

Author SHA1 Message Date
Bryan Boreham dd1d707e1c
Merge pull request #15371 from bboreham/main-naming
[REFACTOR] Small improvement to top-level naming
2024-11-24 17:55:10 +00:00
Ben Ye 40c3ba9fd4
fix promtool analyze block shows metric name with 0 cardinality (#15438)
Signed-off-by: Ben Ye <benye@amazon.com>
2024-11-24 17:30:20 +01:00
Bryan Boreham 3c22b35a03
Merge pull request #15339 from colega/export-go-sync-mutex-wait-total-seconds-total-metric
Export 'go_sync_mutex_wait_total_seconds_total' metric
2024-11-20 13:34:37 +00:00
Björn Rabenstein 384c5951ef
Merge pull request #14489 from harry671003/implement_metadata_limit
storage: Implement limit in mergeGenericQuerier
2024-11-19 17:32:16 +01:00
Ben Kochie 869addfec8
Enable auto-gomaxprocs by default (#15378)
Enable the `auto-gomaxprocs` feature flag by default.
* Add command line flag `--no-auto-gomaxprocs` to disable.

Signed-off-by: SuperQ <superq@gmail.com>
2024-11-12 13:27:01 +01:00
Bryan Boreham f0bd5fdcc6
Merge pull request #15226 from aniketnk/i15185
Parallelize tests in cmd/promtool/
2024-11-12 11:40:30 +00:00
Julien e9179cb240
Merge pull request #15011 from roidelapluie/fixreload
Fix auto reload when a config file with a syntax error is reverted
2024-11-12 11:55:21 +01:00
Ben Kochie 0f0deb77a2
Enable auto-gomemlimit by default (#15372)
Enable the `auto-gomemlimit` feature flag by default.
* Add command line flag `--no-auto-gomemlimit` to disable.

Signed-off-by: SuperQ <superq@gmail.com>
2024-11-11 18:26:38 +01:00
Bryan Boreham de35a40ed5 [REFACTOR] Small improvement to top-level naming
Previously, top-level initialization (including WAL reading) had this at
the top of the stack:

    github.com/oklog/run.(*Group).Run.func1:38

I found this confusing, and thought it had something to do with logging.

Introducing another local function should make this clearer.
Also make the error message user-centric not code-centric.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-11-11 10:55:05 +00:00
Ben Ye 140f4aa9ae
feat: Allow customizing TSDB postings decoder (#13567)
* allow customizing TSDB postings decoder

---------

Signed-off-by: Ben Ye <benye@amazon.com>
2024-11-11 07:59:24 +01:00
Aniket Kaulavkar 650abaccce Parallelize tests in cmd/promtool/
Signed-off-by: Aniket Kaulavkar <aniket.kaulavkar@gmail.com>
2024-11-08 21:54:08 +05:30
🌲 Harry 🌊 John 🏔 f9bc50b247 storage: Implement limit in mergeGenericQuerier
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2024-11-07 09:08:23 -08:00
Arthur Silva Sens 96c2fe04e7
Merge pull request #15328 from prometheus/deadcode
chore: Remove dead code
2024-11-06 13:20:53 -03:00
Matthieu MOREL af1a19fc78 enable errorf rule from perfsprint linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-11-06 16:50:36 +01:00
Oleg Zaytsev 50e83e8f98 Export 'go_sync_mutex_wait_total_seconds_total' metric
This metric can be useful to debug mutex contention.

Ref: https://github.com/golang/go/issues/49881
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-11-05 15:25:55 +01:00
Julien 9d9e86b333 Fix auto reload when a config file with a syntax error is reverted
When we had a syntax error but restored the old file, we did not
re-trigger the config reload, so the config reload metric was showing
that config reload was unsucessful.

I made magic to handle logs in cmd/prometheus.

For now it is a separate file so we can backport this easily.

I will generalize the helper in another PR.

Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-11-05 12:43:34 +01:00
Arthur Silva Sens 88818c9cb3
chore: Remove dead code
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-11-04 13:00:22 -03:00
Alban Hurtaud 4b56af7eb8
Add hidden flag for the delayed compaction random time window (#14919)
* Add hidden flag for the delayed compaction random time window

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>

* Update cmd/prometheus/main.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com>

* Update cmd/prometheus/main.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com>

* Update tsdb/db.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com>

* Fix flag name according to review - add test for delay

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>

* Fix afer main rebase

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>

* Implement review comments

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>

* Update generatedelaytest to try with limit values

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>

---------

Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>
Signed-off-by: Alban Hurtaud <alban.hurtaud@amadeus.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
2024-11-04 08:26:26 +01:00
Vanshika cccbe72514
TSDB: Fix some edge cases when OOO is enabled (#14710)
Fix some edge cases when OOO is enabled

Signed-off-by: Vanshikav123 <vanshikav928@gmail.com>
Signed-off-by: Vanshika <102902652+Vanshikav123@users.noreply.github.com>
Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>
Co-authored-by: Jesus Vazquez <jesusvzpg@gmail.com>
2024-10-23 17:34:28 +02:00
George Krajcsovits 1b4e7f74e6
feat(tools): add debug printouts to rules unit testing (#15196)
* promtool: Add debug flag for rule tests

This makes it print out the tsdb state (both input_series and rules that
are run) at the end of a test, making reasoning about tests much easier.

Signed-off-by: David Leadbeater <dgl@dgl.cx>

* Reuse generated test name from junit testing

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: David Leadbeater <dgl@dgl.cx>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: David Leadbeater <dgl@dgl.cx>
2024-10-22 15:24:36 +02:00
Bryan Boreham 70e2d23027
Merge pull request #11474 from clwluvw/group-label
[FEATURE] rules: add labels at group level
2024-10-21 14:47:12 +01:00
machine424 cf128a0472
test(cmd/prometheus): speed up test execution by t.Parallel() when possible
turn some loops into subtests to make use of t.Parallel()

requires Go 1.22 to make use of https://go.dev/blog/loopvar-preview

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-18 17:52:59 +02:00
Alan Protasio c78d5b94af
Disallowing configure AM with the v1 api (#13883)
* Stop supporting Alertmanager v1

* Disallowing configure AM with the v1 api

Signed-off-by: alanprot <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Addressing coments

Signed-off-by: alanprot <alanprot@gmail.com>

* Update notifier/notifier.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

---------

Signed-off-by: alanprot <alanprot@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
2024-10-18 15:23:14 +02:00
Yi 2cabd1b707
config: remove expand-external-labels flag in release 3.0 (#14657)
remove expand-external-labels feature flag

and enabled env arg expansion for external labels by default.

Signed-off-by: jyz0309 <45495947@qq.com>
2024-10-17 10:25:05 +02:00
TJ Hoplock 6ebfbd2d54 chore!: adopt log/slog, remove go-kit/log
For: #14355

This commit updates Prometheus to adopt stdlib's log/slog package in
favor of go-kit/log. As part of converting to use slog, several other
related changes are required to get prometheus working, including:
- removed unused logging util func `RateLimit()`
- forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger
- move some of the json file logging functionality to use prom/common package functionality
- refactored some of the new json file logging for scraping
- changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers
- updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition
- added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type

Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
2024-10-07 15:58:50 -04:00
Julien 9d8f93ea92
Merge pull request #15113 from yeya24/add-missing-allow-overlapping-compaction-flag
Add missing flag storage.tsdb.allow-overlapping-compaction
2024-10-07 11:17:24 +02:00
Matthieu MOREL ab64966e9d
fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" (#15094)
* fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()"

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-06 16:35:29 +00:00
Ben Ye 6c3d11629b add missing flag storage.tsdb.allow-overlapping-compaction
Signed-off-by: Ben Ye <benye@amazon.com>
2024-10-05 17:05:42 -07:00
Julien 297667469a
Merge pull request #15090 from roidelapluie/querylogrule
Fix flakiness of QueryLogTest
2024-10-04 14:57:24 +02:00
Julien 9d275c23de cmd/prometheus: Fix flakiness of QueryLogTest
Now we check that a rule execution has taken place.

This also reduces the time to run the rules tests from 45s to 25s.

Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-10-04 11:23:57 +02:00
Julien 21e0f83b68 Move notifications in utils
Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-10-04 10:11:56 +02:00
Julien b6158e8956 Notify web UI when starting up and shutting down
Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-10-03 10:27:04 +02:00
Jesus Vazquez 77d3b3aff3
OTLP: Remove experimental word form OTLP receiver (#14894)
The OTLP receiver can now considered stable. We've had it for longer
than a year in main and has received constant improvements.

Signed-off-by: Jesus Vazquez <jesusvzpg@gmail.com>
2024-10-01 14:36:52 +02:00
Julien f9bbad1148 Limit the number of SSE Subscribers to 16 by default
Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-09-27 15:51:51 +02:00
Julien 7aa4721373
Merge pull request #14946 from roidelapluie/notifications
Add notifications to the Web UI
2024-09-27 15:50:43 +02:00
Julien 6cde0096e2 Add notifications to the web UI when configuration reload fails.
This commit introduces a new `/api/v1/notifications/live` endpoint that
utilizes Server-Sent Events (SSE) to stream notifications to the web UI.
This is used to display alerts such as when a configuration reload
has failed.

I opted for SSE over WebSockets because SSE is simpler to implement and
more robust for our use case. Since we only need one-way communication
from the server to the client, SSE fits perfectly without the overhead
of establishing and maintaining a two-way WebSocket connection.

When the SSE connection fails, we go back to a classic
/api/v1/notifications API endpoint.

This commit also contains the required UI changes for the new Mantine UI.

Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-09-27 15:28:38 +02:00
Bryan Boreham 410fcce6f0
Remove unnecessary pprof import (#14988)
The pattern of `import _ "net/http/pprof"` adds handlers to the default
http handler, but Prometheus does not use that. There are explicit
handlers in `web/web.go`.

So, we can remove this line with no impact to behaviour.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-09-27 07:45:49 +01:00
Björn Rabenstein f74722841b
Merge pull request #14160 from alex-kattathra-johnson/issue-13959
Remove no-default-scrape-port featureFlag
2024-09-26 18:45:56 +02:00
Arthur Silva Sens 6bd9b1a7cc
Histogram CT Zero ingestion
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-09-26 11:29:22 -03:00
Alex Johnson be0f10054e Remove no-default-scrape-port featureFlag
Signed-off-by: Alex Johnson <alex.kattathra.johnson@gmail.com>
2024-09-25 10:13:19 -05:00
Julien 919648cafc
Merge pull request #14947 from roidelapluie/reloadinvalidyaml
fix(autoreload): Reload invalid yaml files
2024-09-23 10:03:23 +02:00
Björn Rabenstein 5b9148e552
Merge pull request #14820 from charleskorn/promqltest-native-histogram-format
promqltest: use test expression format for histograms in assertion failure messages and include reset hint in the test expression
2024-09-20 16:47:08 +02:00
Julien 146b22d196 fix(autoreload): Reload invalid yaml files
When a YAML file is invalid, trigger auto-reload anyway so that user is
aware that the configuration file is incorrect.

Failing to do so does not change the reload status in metrics and api.

Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-09-20 13:38:05 +02:00
Julius Volz b8d1336d42
Merge pull request #14912 from roidelapluie/notready
mantine UI: Distinguish between Not Ready and Stopping
2024-09-17 19:40:13 +02:00
Julien ac5377873f mantine UI: Distinguish between Not Ready and Stopping
Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-09-17 16:02:16 +02:00
Carrie Edwards 14e3c05ce8
tsdb: Add support for ingestion of out-of-order native histogram samples (#14546)
Add support for ingesting OOO native histograms

* Add flag for enabling and disabling OOO native histogram ingestion
* Update OOO querying tests to include native histogram samples
* Add OOO head tests
* Add test for OOO native histogram counter reset headers

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored by: Carrie Edwards <edwrdscarrie@gmail.com>
Co-authored by: Jeanette Tan <jeanette.tan@grafana.com>
Co-authored by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored by: Fiona Liao <fiona.liao@grafana.com>
2024-09-17 11:19:06 +02:00
Augustin Husson 6febfbb3be
put back feature flag 'delayed-compaction' and 'old-ui' (#14909)
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
2024-09-16 15:46:28 +02:00
Nathan Baulch 50cd453c8f
chore: Fix typos (#14868)
* Fix typos

---------

Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-10 22:32:03 +02:00
Jan Fajerski fa318711f4 Merge branch 'main' into 3.0-main-sync-24-09-09
Conflicts:
	cmd/prometheus/main.go
	docs/command-line/prometheus.md
	docs/feature_flags.md
	web/ui/build_ui.sh
	web/web.go
    Resolved by dropping the UTF-8 feature flag and adding the
    `auto-reload-config` feature flag.
    For the new web ui pick all changes from `main`.
2024-09-09 15:44:22 +02:00
Julius Volz ad21ef3d76 Merge branch 'main' into mantine-ui
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2024-09-06 21:26:00 +02:00