prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	5d4db805ac	Merge branch 'main' into sparsehistogram	3 years ago
beorn7	4c28d9fac7	Move to histogram.Histogram pointers This is to avoid copying the many fields of a histogram.Histogram all the time. This also fixes a bunch of formerly broken tests. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Robert Fratto	72a9f7fee9	Share TSDB locker code with agent (#9623 ) * share tsdb db locker code with agent Closes #9616 Signed-off-by: Robert Fratto <robertfratto@gmail.com> * add flag to disable lockfile for agent Signed-off-by: Robert Fratto <robertfratto@gmail.com> * use agentOnlySetting instead of PreAction Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb: address review feedback 1. Rename Locker to DirLocker 2. Move DirLocker to tsdb/tsdbutil 3. Name metric using fmt.Sprintf 4. Refine error checking in DirLocker test Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb: create test utilities to assert expected DirLocker behavior Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb/tsdbutil: fix lint errors Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb/agent: fix windows test failure Use new DB variable instead of overriding the old one. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 years ago
Mateusz Gozdek	c08bb86be0	cmd/prometheus: use random listen port in TestStartupInterrupt test So it can be run in parallel safely. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	7bd7573891	cmd/prometheus/main_unix_test.go: fix unix test styling * Formatting of error message is missing a space after ':'. * t.Fatalf should be used instead of t.Errorf+return. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Robert Fratto	4a83e6f453	Remove agent mode warnings when loading configs (#9622 ) PR #9618 introduced failing to load the config file when agent mode is configured to run with unspported settings. This made the block that logs a warning on their configuration no-op, which is now removed. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 years ago
Mateusz Gozdek	fa1b14e146	cmd/prometheus: randomize test port and isolate test data directory Between the tests. This enables parallelizing those tests, which should cut the test execution time. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
beorn7	c954cd9d1d	Move packages out of deprecated pkg directory This creates a new `model` directory and moves all data-model related packages over there: exemplar labels relabel rulefmt textparse timestamp value All the others are more or less utilities and have been moved to `util`: gate logging modetimevfs pool runtime Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Dieter Plaetinck	cda025b5b5	TSDB: demistify SeriesRefs and ChunkRefs (#9536 ) * TSDB: demistify seriesRefs and ChunkRefs The TSDB package contains many types of series and chunk references, all shrouded in uint types. Often the same uint value may actually mean one of different types, in non-obvious ways. This PR aims to clarify the code and help navigating to relevant docs, usage, etc much quicker. Concretely: * Use appropriately named types and document their semantics and relations. * Make multiplexing and demuxing of types explicit (on the boundaries between concrete implementations and generic interfaces). * Casting between different types should be free. None of the changes should have any impact on how the code runs. TODO: Implement BlockSeriesRef where appropriate (for a future PR) Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * feedback Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * agent: demistify seriesRefs and ChunkRefs Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Bartlomiej Plotka	789274bf9c	cmd: Fixed storage flag regression introduced in #9660 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	3 years ago
Sunil Thaha	4bdaea7663	fix: storage.tsdb.path randomly initialised to data-agent/ (#9660 ) Using the same variable for storage.tsdb.path and storage.agent.path as below in main.go causes cfg.localStoragePath to be data/ or data-agent/ at random. a.Flag("storage.tsdb.path", "Base path for metrics storage."). PreAction(serverOnlySetting()). Default("data/").StringVar(&cfg.localStoragePath) a.Flag("storage.agent.path", "Base path for metrics storage."). PreAction(agentOnlySetting()). Default("data-agent/").StringVar(&cfg.localStoragePath) This patch fixes it by using a different variable for storage.agent.path Signed-off-by: Sunil Thaha sthaha@redhat.com Signed-off-by: Sunil Thaha <sthaha@redhat.com>	3 years ago
Bartlomiej Plotka	e68ccc7708	Fix misleading agent-only/server-only check messages. (#9650 ) * Fix misleading agent-only/server-only check messages. Issue: ``` [root@host01 ~]# docker run -it --net=host --rm -v /root/editor/prom-agent-batcopter.yaml:/etc/prometheus/prometheus.yaml -v /root/prom-batcopter-data:/prometheus -u root --name prom-agent-batcopter quay.io/prometheus/prometheus:main --enable-feature=agent --config.file=/etc/prometheus/prometheus.yaml --storage.tsdb.path=/prometheus --web.listen-address=:9091 ts=2021-11-02T16:00:59.789Z caller=main.go:205 level=info msg="Experimental agent mode enabled." The following flag(s) can not be used in agent mode: ["--enable-feature"] ``` Problem was that PreAction gives us all parsed flag. Context does not give us any info on what flag clause it was defined. Also added info for flag help about being server or agent only. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * gofumpt. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	3 years ago
Mateusz Gozdek	c3beca72e2	cmd/prometheus: wait for Prometheus to shutdown in tests So temporary data directory can be successfully removed, as on Windows, directory cannot be in used while removal. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	b7bdf6fab2	Fix imports formatting According to `2829908806 (r58457095)`. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	1a6c2283a3	Format Go source files using 'gofumpt -w -s -extra' Part of #9557 Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Julien Pivotto	807f46a1ed	Gate agent behind a feature flag, valide mode flags (#9620 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Darshan Chaudhary	a7e554b158	add check service-discovery command (#8970 ) Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
Hu Shuai	4b799c361a	Fix in typo in cmd/prometheus/main.go (#9632 ) Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>	3 years ago
Arthur Silva Sens	be2599c853	config: Make remote-write required for Agent mode (#9618 ) * config: Make remote-write required for Agent mode Signed-off-by: ArthurSens <arthursens2005@gmail.com>	3 years ago
Robert Fratto	bc72a718c4	Initial draft of prometheus-agent (#8785 ) * Initial draft of prometheus-agent This commit introduces a new binary, prometheus-agent, based on the Grafana Agent code. It runs a WAL-only version of prometheus without the TSDB, alerting, or rule evaluations. It is intended to be used to remote_write to Prometheus or another remote_write receiver. By default, prometheus-agent will listen on port 9095 to not collide with the prometheus default of 9090. Truncation of the WAL cooperates on a best-effort case with Remote Write. Every time the WAL is truncated, the minimum timestamp of data to truncate is determined by the lowest sent timestamp of all samples across all remote_write endpoints. This gives loose guarantees that data from the WAL will not try to be removed until the maximum sample lifetime passes or remote_write starts functionining. Signed-off-by: Robert Fratto <robertfratto@gmail.com> * add tests for Prometheus agent (#22) * add tests for Prometheus agent * add tests for Prometheus agent * rearranged tests as per the review comments * update tests for Agent * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * incremental changes to prometheus agent Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * Commit feedback from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Port over some comments from grafana/agent Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Rename agent.Storage to agent.DB for tsdb consistency Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Consolidate agentMode ifs in cmd/prometheus/main.go Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Document PreAction usage requirements better for agent mode flags Signed-off-by: Robert Fratto <robertfratto@gmail.com> * remove unnecessary defaultListenAddr Signed-off-by: Robert Fratto <robertfratto@gmail.com> * `go fmt ./tsdb/agent` and fix lint errors Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: SriKrishna Paparaju <paparaju@gmail.com>	3 years ago
David Leadbeater	c91c2bbea5	promtool: Show more human readable got/exp output (#8064 ) Avoid using %#v, nothing needs to parse this, so escaping " and so on leads to hard to read output. Add new lines, number and indentation to each alert series output. Signed-off-by: David Leadbeater <dgl@dgl.cx>	3 years ago
DrAuYueng	69e309d202	Expose TargetsFromGroup/AlertmanagerFromGroup func and reuse this for (#9343 ) static/file sd config check in promtool Signed-off-by: DrAuYueng <ouyang1204@gmail.com>	3 years ago
Julien Pivotto	73255e15f6	Address golint failures from revive Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Will Tran	97b0738895	add --max-block-duration in promtool create-blocks-from rules (#9511 ) * support maxBlockDuration for promtool tsdb create-blocks-from rules Fixes #9465 Signed-off-by: Will Tran <will@autonomic.ai> * don't hardcode 2h as the default block size in rules test Signed-off-by: Will Tran <will@autonomic.ai>	3 years ago
Furkan Türkal	9d0058a09e	Bind port 0 in main_test (#9558 ) Fixes #9499 Signed-off-by: Furkan <furkan.turkal@trendyol.com>	3 years ago
Julien Pivotto	432005826d	Add a feature flag to enable the new discovery manager (#9537 ) * Add a feature flag to enable the new manager This PR creates a copy of the legacy manager and uses it by default. It is a companion PR to #9349. With this PR, users can enable the new discovery manager and provide us with any feedback / side effects that the new behaviour might have. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
beorn7	a9008f5423	Merge branch 'main' into sparsehistogram	3 years ago
jessicagreben	60d0990886	add more explicit label values Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
jessicagreben	3da87d2f39	add unit test to check label rule labels override Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
Julien Pivotto	f8372bc6b9	backfill: Apply rule labels after query labels Fix #9419 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
beorn7	7a8bb8222c	Style cleanup of all the changes in sparsehistogram so far A lot of this code was hacked together, literally during a hackathon. This commit intends not to change the code substantially, but just make the code obey the usual style practices. A (possibly incomplete) list of areas: * Generally address linter warnings. * The `pgk` directory is deprecated as per dev-summit. No new packages should be added to it. I moved the new `pkg/histogram` package to `model` anticipating what's proposed in #9478. * Make the naming of the Sparse Histogram more consistent. Including abbreviations, there were just too many names for it: SparseHistogram, Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in general. Only add "Sparse" if it is needed to avoid confusion with conventional Histograms (which is rare because the TSDB really has no notion of conventional Histograms). Use abbreviations only in local scope, and then really abbreviate (not just removing three out of seven letters like in "Histo"). This is in the spirit of https://github.com/golang/go/wiki/CodeReviewComments#variable-names * Several other minor name changes. * A lot of formatting of doc comments. For one, following https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences , but also layout question, anticipating how things will look like when rendered by `godoc` (even where `godoc` doesn't render them right now because they are for unexported types or not a doc comment at all but just a normal code comment - consistency is queen!). * Re-enabled `TestQueryLog` and `TestEndopints` (they pass now, leaving them disabled was presumably an oversight). * Bucket iterator for histogram.Histogram is now created with a method. * HistogramChunk.iterator now allows iterator recycling. (I think @dieterbe only commented it out because he was confused by the question in the comment.) * HistogramAppender.Append panics now because we decided to treat staleness marker differently. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	fd5ea4e0b5	Merge branch 'main' into sparsehistogram	3 years ago
Julien Pivotto	bd217c58a7	Backfill: Do not query after --end (#9340 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Julien Pivotto	1ea774f184	Merge pull request #9339 from roidelapluie/remove-double-align backfill: Do not align the start of the group since we align every rule.	3 years ago
Julien Pivotto	2bde71ec5f	Merge pull request #9338 from prometheus/release-2.30 merge back release 2.30	3 years ago
Julien Pivotto	691ce066fb	backfill: Do not align the start of the group since we align every rule. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
jessicagreben	b0a21f9eab	rm overlap, add label builder to fix name bug Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
Julien Pivotto	0111aa987e	Merge pull request #9312 from fpetkovski/promtool-analyze-compaction promtool: add extended flag for tsdb analysis	3 years ago
Julien Pivotto	48a101be1b	Allow to tune the scrape tolerance (#9283 ) * Allow to tune the scrape tolerance In most of the classic monitoring use cases, a few milliseconds difference can be omitted. In Prometheus, a few millisecond difference can however make a big difference. Currently, Prometheus will ignore up to 2 ms difference in the alignments. It turns out that for users who can afford a 10ms difference, there is a lot of resources and disk space to win, as shown in this graph, which shows the bytes / samples over a production Prometheus server. You can clearly see the switch from 2ms to 10ms tolerance. This pull request enables the adjustment of the scrape timestamp alignment tolerance. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix golint Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
fpetkovski	449f874679	promtool: add extended flag for tsdb analysis The compaction analysis which runs under promtool tsdb analyze can be an intensive process which slows down the entire command. This commit adds an --extended flag to tsdb analyze which can be toggled for running long running tasks, such as compaction analysis. Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>	3 years ago
Julien Pivotto	ad642a85c0	Merge pull request #9304 from LeviHarrison/backfill-fix-date Rules backfill: fix new rule importer message	3 years ago
Julien Pivotto	bd24e2fb92	Merge pull request #9303 from LeviHarrison/backfill-return-1 Rules backfill: return 1 if unsuccessful	3 years ago
Levi Harrison	ded95ff434	Fix new rule importer message Signed-off-by: Levi Harrison <git@leviharrison.dev>	3 years ago
Levi Harrison	34e1b47968	Fixed error handling Signed-off-by: Levi Harrison <git@leviharrison.dev>	3 years ago
Holger Hans Peter Freyther	5edec40d60	promtool: Speed up checking for duplicate rules Trade space for speed. Convert all rules into our temporary struct, sort and then iterate. This is a significant when having many rules. Signed-off-by: Holger Hans Peter Freyther <holger@moiji-mobile.com>	3 years ago
Holger Hans Peter Freyther	3a309c1ae5	promtool: Add simple benchmark checkDuplicates benchmark Add a simple benchmark with a large number of rules. Signed-off-by: Holger Hans Peter Freyther <holger@moiji-mobile.com>	3 years ago
Holger Hans Peter Freyther	794937b3d6	promtool: Add testcase for detecting duplicates Introduce a basic test for checking for duplicate rules. Signed-off-by: Holger Hans Peter Freyther <holger@moiji-mobile.com>	3 years ago
SuperQ	31f4108758	Add scrape_timeout_seconds metric Add a new built-in metric `scrape_timeout_seconds` to allow monitoring of the ratio of scrape duration to the scrape timeout. Hide behind a feature flag to avoid additional cardinality by default. Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
SuperQ	e167a45c65	Add new Go build tags. Add new go:build comments based on 1.17 formatting[0]. [0]: https://golang.org/doc/go1.17#gofmt Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
Julien Pivotto	cab96a06ef	Merge release 2.29 in main (#9196 ) * PromQL: Fix start and end keywords masking label and metric names This commit fixes an issue with the "at modifier" that introduced two new keywords: `start` and `end`. In grouping options and in metric names, these keywords took precedence over metric or label names, so that those metrics and labels could no longer be referenced. Signed-off-by: Clayton Peters <clayton.peters@man.com> * Add in additional tests for metrics and/or labels called start/end. Signed-off-by: Clayton Peters <clayton.peters@man.com> * : Cut 2.29.0-rc.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> VERSION: bump to 2.29.0-rc.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> * Remove experimental wording on size-based retention Followup of #9004 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix PR reference in changelog Signed-off-by: George Brighton <george@gebn.co.uk> * Describe EC2 availability zone IDs at most once per refresh (#9142) Signed-off-by: George Brighton <george@gebn.co.uk> * Describe EC2 availability zones at most once per SD load Closes #9142. Signed-off-by: George Brighton <george@gebn.co.uk> * Incorporate feedback Signed-off-by: George Brighton <george@gebn.co.uk> * Integrate feedback Signed-off-by: George Brighton <george@gebn.co.uk> * Add a compatibility note for macOS users. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * : Cut v2.29.0-rc.1 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> Fix `kuma_sd` targetgroup reporting (#9157) * Bundle all xDS targets into a single group Signed-off-by: austin ce <austin.cawley@gmail.com> * : cut v2.29.0-rc.2 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> Rename links Signed-off-by: Levi Harrison <git@leviharrison.dev> * bump codemirror-promql to 0.17.0 Signed-off-by: Augustin Husson <husson.augustin@gmail.com> * : cut v2.29.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> tsdb: align atomically accessed int64 (#9192) This prevents a panic in 32-bit archs: https://pkg.go.dev/sync/atomic#pkg-note-BUG Fixed #9190 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Release 2.29.1 (#9193) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> Co-authored-by: Clayton Peters <clayton.peters@man.com> Co-authored-by: Frederic Branczyk <fbranczyk@gmail.com> Co-authored-by: George Brighton <george@gebn.co.uk> Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com> Co-authored-by: Levi Harrison <git@leviharrison.dev> Co-authored-by: Augustin Husson <husson.augustin@gmail.com>	3 years ago

1 2 3 4 5 ...

551 Commits (26c0a433f53777dcc8faa62b66f5ec4a1bee477a)