prometheus

Commit Graph

Author	SHA1	Message	Date
Levi Harrison	fa9bc5184a	Update and fix interface (#11131 ) Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Levi Harrison	d61459d826	`no-default-scrape-port` feature flag (#9523 ) * Add `no-default-scrape-port` flag Signed-off-by: Levi Harrison <git@leviharrison.dev>	2 years ago
Paschalis Tsilias	d1122e0743	Introduce TSDB changes for appending metadata to the WAL (#10972 ) * Append metadata to the WAL Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Remove extra whitespace; Reword some docstrings and comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use RLock() for hasNewMetadata check Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use single byte for metric type in RefMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Update proposed WAL format for single-byte type metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Implementa MetadataAppender interface for the Agent Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Address first round of review comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Amend description of metadata in wal.md Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Correct key used to retrieve metadata from cache When we're setting metadata entries in the scrapeCace, we're using the p.Help(), p.Unit(), p.Type() helpers, which retrieve the series name and use it as the cache key. When checking for cache entries though, we used p.Series() as the key, which included the metric name _with_ its labels. That meant that we were never actually hitting the cache. We're fixing this by utiling the __name__ internal label for correctly getting the cache entries after they've been set by setHelp(), setType() or setUnit(). Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Put feature behind a feature flag Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix AppendMetadata docstring Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reorder WAL format document Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Change error message of AppendMetadata; Fix access of s.meta in AppendMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reuse temporary buffer in Metadata encoder Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Only keep latest metadata for each refID during checkpointing Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix test that's referencing decoding metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Avoid creating metadata block if no new metadata are present Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add tests for corrupt metadata block and relevant record type Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix CR comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Extract logic about changing metadata in an anonymous function Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Implement new proposed WAL format and amend relevant tests Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use 'const' for metadata field names Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Apply metadata to head memSeries in Commit, not in AppendMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add docstring and rename extracted helper in scrape.go Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add tests for tsdb-related cases Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linter issues vol1 Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linter issues vol2 Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix Windows test by closing WAL reader files Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Use switch instead of two if statements in metadata decoding Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix review comments around TestMetadata* tests Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Add code for replaying WAL; test correctness of in-memory data after a replay Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Remove scrape-loop related code from PR Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Address first round of comments Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Simplify tests by sorting slices before comparison Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix test to use separate transactions Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Empty out buffer and record slices after encoding latest metadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix linting issue Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Update calculation for DroppedMetadata metric Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Rename MetadataAppender interface and AppendMetadata method to MetadataUpdater/UpdateMetadata Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Reuse buffer when encoding latest metadata for each series Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Fix review comments; Check all returned error values using two helpers Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Simplify use of helpers Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com> * Satisfy linter Signed-off-by: Paschalis Tsilias <paschalist0@gmail.com>	2 years ago
Julien Pivotto	7a2d24b76a	Fix flakiness in windows tests (#10983 ) Our windows CI is too slow, process takes lots of time to start. Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Julien Pivotto	13bd4fd3c8	Fix promtool check config not erroring properly on failures (#10952 ) Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
lixin18	735a07444a	Update main_unix_test.go (#10917 ) so->,so Signed-off-by: lixin18 <68135097+lixin963@users.noreply.github.com>	2 years ago
David Leadbeater	355b8bcf0b	Add --lint-fatal option (#10815 ) This keeps the previous behaviour of printing details about duplicate rules but doesn't exit with a fatal exit code unless turned on. Signed-off-by: David Leadbeater <dgl@dgl.cx>	3 years ago
Ben Kochie	9570924511	Merge pull request #9638 from prometheus/superq/agentMode Add agent mode identifier	3 years ago
Matthieu MOREL	36eee11434	refactor (package cmd): move from github.com/pkg/errors to 'errors' and 'fmt' packages (#10733 ) Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com> Co-authored-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>	3 years ago
Łukasz Mierzwa	44e5f220c0	Move prometheus_ready metric to web package (#10729 ) This moves prometheus_ready to the web package and links it with the ready variable that decides if HTTP requests should return 200 or 503. This is a follow up change from #10682 Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Łukasz Mierzwa	070e409dba	Add prometheus_ready metric (#10682 ) When Prometheus starts it can take a long time before WAL is replayed and it can do anything useful. While it's starting it exposes metrics and other Prometheus servers can scrape it. We do have alerts that fire if any Prometheus server is not ingesting samples and so far we've been dealing with instances that are starting for a long time by adding a check on Prometheus process uptime. Relying on uptime isn't ideal because the time needed to start depends on the number of metrics scraped, and so on the amount of data in WAL. To help write better alerts it would be great if Prometheus exposed a metric that tells us it's fully started, that way any alert that suppose to notify us about any runtime issue can filter out starting instances. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Ben Ye	af5ea213f7	promtool: support matchers when querying label values (#10727 ) * promtool: support matchers when querying label values Signed-off-by: Ben Ye <ben.ye@bytedance.com> * address review comment Signed-off-by: Ben Ye <ben.ye@bytedance.com>	3 years ago
Łukasz Mierzwa	d3c9c4f574	Stop rule manager before TSDB is stopped (#10680 ) During shutdown TSDB is stopped before rule manager is stopped. Since TSDB shutdown can take a long time (minutes or 10s of minutes) it keeps rule manager running while parts of Prometheus are already stopped (most notebly scrape manager). This can cause false positive alerts to fire, mostly those that rely on absent() calls since new sample appends will stop while alert queries are still evaluated. Stop rules before stopping TSDB and scrape manager to avoid this problem. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Alban Hurtaud	41630b8e88	Add hidden flag to configure discovery loop interval (#10634 ) * Add hidden flag to configure discovery loop interval Signed-off-by: Alban HURTAUD <alban.hurtaud@amadeus.com>	3 years ago
Matthieu MOREL	e2ede285a2	refactor: move from io/ioutil to io and os packages (#10528 ) * refactor: move from io/ioutil to io and os packages * use fs.DirEntry instead of os.FileInfo after os.ReadDir Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>	3 years ago
Filip Petkovski	1c1b174a8e	Add a --lint flag to the promtool check rules and check config commands (#10435 ) * Add a --lint flag to the promtool check rules and check config commands Checking rules with promtool emits warnings in the case of duplicate rules. These warnings do not result in a non-zero exit code and are difficult to spot in CI environments. Additionally, checking for duplicates is closer to a lint check rather than a syntax check. This commit adds a --lint flag to commands which include checking rules. The flag can be used to enable or disable certain linting options and cause the execution to return a non-zero exit code in case those options are not met. Signed-off-by: fpetkovski <filip.petkovsky@gmail.com> * Exit with status 3 on lint error Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>	3 years ago
Julien Pivotto	390956d317	Log gomaxprocs messages (#10506 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
TomasKohout	c0fd228bad	Add dependency on go.uber.org/automaxprocs (#10498 ) * add dependency on go.uber.org/automaxprocs Signed-off-by: Tomas Kohout <tomas.kohout1995@gmail.com> Co-authored-by: Peter Bourgon <peterbourgon@users.noreply.github.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>	3 years ago
Julien Pivotto	f9d8e5245a	Plugins support (#10495 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Wilbert Guo	83a2e52bc2	Add SyncForState Implementation for Ruler HA (#10070 ) * continuously syncing activeAt for alerts Signed-off-by: Yijie Qin <qinyijie@amazon.com> Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> * add import Signed-off-by: Yijie Qin <qinyijie@amazon.com> Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> * Refactor SyncForState and add unit tests Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> * Format code Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> * Add hook for syncForState Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Fix go lint Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Refactor syncForState override implementation Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Add syncForState override func as argument to Update() Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Fix go formatting Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Fix circleci test errors Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> Remove overrideFunc as argument to run() Signed-off-by: Wilbert Guo <wilbeguo@amazon.com> * remove the syncForState Signed-off-by: Yijie Qin <qinyijie@amazon.com> * use the override function to decide if need to replace the activeAt or not Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix test case Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix format Signed-off-by: Yijie Qin <qinyijie@amazon.com> * Trigger build Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fixing comments Signed-off-by: Yijie Qin <qinyijie@amazon.com> * return the result of map of alerts instead of single one Signed-off-by: Yijie Qin <qinyijie@amazon.com> * upper case the QueryforStateSeries Signed-off-by: Yijie Qin <qinyijie@amazon.com> * use a more generic rule group post process function type Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix indentation Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix gofmt Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix lint Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fixing naming Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fix comments Signed-off-by: Yijie Qin <qinyijie@amazon.com> * add the lastEvalTimestamp as parameter Signed-off-by: Yijie Qin <qinyijie@amazon.com> * fmt Signed-off-by: Yijie Qin <qinyijie@amazon.com> * change funcType to func Signed-off-by: Yijie Qin <qinyijie@amazon.com> Co-authored-by: Yijie Qin <qinyijie@amazon.com> Co-authored-by: Yijie Qin <63399121+qinxx108@users.noreply.github.com>	3 years ago
Alan Protasio	606ef33d91	Track and report Samples Queried per query We always track total samples queried and add those to the standard set of stats queries can report. We also allow optionally tracking per-step samples queried. This must be enabled both at the engine and query level to be tracked and rendered. The engine flag is exposed via a Prometheus feature flag, while the query flag is set when stats=all. Co-authored-by: Alan Protasio <approtas@amazon.com> Co-authored-by: Andrew Bloomgarden <blmgrdn@amazon.com> Co-authored-by: Harkishen Singh <harkishensingh@hotmail.com> Signed-off-by: Andrew Bloomgarden <blmgrdn@amazon.com>	3 years ago
Mauro Stettler	b025390cb4	Disable chunk write queue by default, allow user to configure the exact size (#10425 ) * Disable chunk write queue by default Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * update flag description Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>	3 years ago
ian woolf	025528a5d6	cmd: use os.MkdirTemp instead of ioutil.TempDir (#10320 ) Signed-off-by: ianwoolf <btw515wolf2@gmail.com>	3 years ago
Łukasz Mierzwa	a4317bf0ec	Run gofumpt on all files (#10392 ) * Run gofumpt on all files Getting golangci-lint errors when building on my laptop, possibly because I have newer version of gofumpt then what it was formatted with. Run gofumpt -w -extra on all files as it will be needed in the future anyway. * Update golangci-lint to v1.44.2 v1.44.0 upgraded gofumpt so bumping version in CI will help keep formatting correct for everyone * Address golangci-lint error Getting 'error-strings: error strings should not be capitalized or end with punctuation or a newline' from revive here. Drop new line. Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
SuperQ	b297520666	Add agent mode identifier Identify in the logs and liveness endpoints if the server is running in Agent mode or not. Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
Tobias Klausmann	b998636893	Improve error logging for missing config and QL dir (#10260 ) * Improve error logging for missing config and QL dir Currently, when Prometheus can't open its config file or the query logging dir under the data dir, it only logs what it has been given default or commandline/config. Depending on the environment this can be less than helpful, since the working directory may be unclear to the user. I have specifically kept the existing error messages as intact as possible to a) still log the parameter as given and b) cause as little disruption for log-parsers/-analyzers as possible. So in case of the config file or the data dir being non-absolute paths, I use os.GetWd to find the working dir and assemble an absolute path for error logging purposes. If GetWd fails, we just log "unknown", as recovering from an error there would be very complex measure, likely not worth the code/effort. Example errors: ``` $ ./prometheus ts=2022-02-06T16:00:53.034Z caller=main.go:445 level=error msg="Error loading config (--config.file=prometheus.yml)" fullpath=/home/klausman/src/prometheus/prometheus.yml err="open prometheus.yml: no such file or directory" $ touch prometheus.yml $ ./prometheus [...] ts=2022-02-06T16:01:00.992Z caller=query_logger.go:99 level=error component=activeQueryTracker msg="Error opening query log file" file=data/queries.active fullpath=/home/klausman/src/prometheus/data/queries.active err="open data/queries.active: permission denied" panic: Unable to create mmap-ed active query log [...] $ ``` Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de> * Replace our own logic with just using filepath.Abs() Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de> * Further simplification Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de> * Review edits Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de> * Review edits Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de> * Review edits Signed-off-by: Tobias Klausmann <klausman@schwarzvogel.de>	3 years ago
Julien Pivotto	9a2e93228e	Switch to grafana/regexp everywhere (#10268 ) Let's have a consistent library for regexp. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Matej Gera	2c61d29b2a	Tracing: Migrate to OpenTelemetry library (#9724 ) Signed-off-by: Matej Gera <matejgera@gmail.com>	3 years ago
Rodrigo Queiro	70c1446a64	Clarify units of --storage.tsdb.retention.size (#10154 ) The flag uses Base2Bytes: `129ed4ec8b/cmd/prometheus/main.go (L1476)` Signed-off-by: Rodrigo Queiro <rodrigoq@google.com>	3 years ago
beorn7	b39f2739e5	PromQL: Always enable negative offset and @ modifier This follows the line of argument that the invariant of not looking ahead of the query time was merely emerging behavior and not a documented stable feature. Any query that looks ahead of the query time was simply invalid before the introduction of the negative offset and the @ modifier. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
beorn7	61509fc840	PromQL: Promote negative offset and @ modifer to stable Following the argument that breaking the invariant that PromQL does not look ahead of the evaluation time implies a breaking change, we still need to keep the feature flag around, but at least we can communicate that the feature is considered stable, and that the feature flags will be ignored from v3 on. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Björn Rabenstein	0f4a1e6eac	Merge pull request #10119 from prometheus/beorn7/remote API: Promote remote-write-receiver to stable	3 years ago
chenlujjj	2ce94ac196	Add '--weight' flag to 'promtool check metrics' command (#10045 )	3 years ago
beorn7	8fdfa52976	API: Promote remote-write-receiver to stable Since `/api/v1/write` is a mutating endpoint, we should still activate the remote-write-receiver explicitly. But we should do it in the same way as the other mutating endpoints, i.e. via a flag `--web.enable-remote-write-receiver`. This commit marks the feature flag as deprecated, i.e. it still works but logs a warning on startup. This enables users to seamlessly migrate. With the next minor release, we can start ignoring the feature flag (but still warn a user that is trying to use it). Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
David Leadbeater	a961062c37	Disable time based retention in tests (#8818 ) Fixes #7699. Signed-off-by: David Leadbeater <dgl@dgl.cx>	3 years ago
Jessica G	174a1147d5	Merge pull request #9861 from JessicaGreben/minor-prom-improvements Add exit code constants in promtool	3 years ago
jessicagreben	4b03fa3100	replace config exit code with failure exit code Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
jessicagreben	59f7ef06d0	update exit code for sd Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
Nicholas Blott	c92673fb14	Remove check against cfg so interval/ timeout are always set (#10023 ) Signed-off-by: Nicholas Blott <blottn@tcd.ie>	3 years ago
Julien Pivotto	db1551bd21	Merge pull request #10016 from prometheus/release-2.32 Merge back release 2.32	3 years ago
Ben Ye	d9bbe7f3dd	mention agent mode in enable-feature flag help description (#9939 ) Signed-off-by: Ben Ye <ben.ye@bytedance.com>	3 years ago
zzehring	42628899b5	promtool: Add `--syntax-only` flag for `check config` This commit adds a `--syntax-only` flag for `promtool check config`. When passing in this flag, promtool will omit various file existence checks that would cause the check to fail (e.g. the check would not fail if `rule_files` files don't exist at their respective paths). This functionality will allow CI systems to check the syntax of configs without worrying about referenced files. Fixes: #5222 Signed-off-by: zzehring <zack.zehring@grafana.com>	3 years ago
jessicagreben	99bb56fc46	add errcodes from sd file Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
Filip Petkovski	5849521e90	promtool: Fix credentials file check (#9883 ) The promtool check config command still uses the bearer_token_file field which is deprecated in favour of authorization.credentials_file. This commit modifies the command to use the new field insted. Fixes #9874 Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>	3 years ago
jessicagreben	764f2d03a5	add const for exit codes Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	3 years ago
Matheus Alcantara	d9a8c453a0	cmd: use t.TempDir instead of ioutil.TempDir on tests (#9852 )	3 years ago
Robert Fratto	72a9f7fee9	Share TSDB locker code with agent (#9623 ) * share tsdb db locker code with agent Closes #9616 Signed-off-by: Robert Fratto <robertfratto@gmail.com> * add flag to disable lockfile for agent Signed-off-by: Robert Fratto <robertfratto@gmail.com> * use agentOnlySetting instead of PreAction Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb: address review feedback 1. Rename Locker to DirLocker 2. Move DirLocker to tsdb/tsdbutil 3. Name metric using fmt.Sprintf 4. Refine error checking in DirLocker test Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb: create test utilities to assert expected DirLocker behavior Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb/tsdbutil: fix lint errors Signed-off-by: Robert Fratto <robertfratto@gmail.com> * tsdb/agent: fix windows test failure Use new DB variable instead of overriding the old one. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 years ago
Mateusz Gozdek	c08bb86be0	cmd/prometheus: use random listen port in TestStartupInterrupt test So it can be run in parallel safely. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	7bd7573891	cmd/prometheus/main_unix_test.go: fix unix test styling * Formatting of error message is missing a space after ':'. * t.Fatalf should be used instead of t.Errorf+return. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Robert Fratto	4a83e6f453	Remove agent mode warnings when loading configs (#9622 ) PR #9618 introduced failing to load the config file when agent mode is configured to run with unspported settings. This made the block that logs a warning on their configuration no-op, which is now removed. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 years ago

1 2 3 4 5 ...

587 Commits (0fa8469f76107dc12e949164fc174a4098aedb19)