prometheus

Commit Graph

Author	SHA1	Message	Date
Ganesh Vernekar	9199fcb8d1	'@ <timestamp>' modifier (#8121 ) This commit adds `@ <timestamp>` modifier as per this design doc: https://docs.google.com/document/d/1uSbD3T2beM-iX4-Hp7V074bzBRiRNlqUdcWP6JTDQSs/edit. An example query: ``` rate(process_cpu_seconds_total[1m]) and topk(7, rate(process_cpu_seconds_total[1h] @ 1234)) ``` which ranks based on last 1h rate and w.r.t. unix timestamp 1234 but actually plots the 1m rate. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	ac2626757c	Update exporter-toolkit to 0.5.0 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Guangwen Feng	2df1a482da	Fix misspelled word in comment (#8348 ) Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	4 years ago
Julien Pivotto	bc9f9ee3aa	Backfilling: fast-path for non-consecutive blocks (#8324 ) * Backfilling: optimize for non-consecutive blocks When you have missing data for > 2 hours, you spend a lot of time re-reading the complete file. It is not optimal. This introduces a fastpath for this scenario. Next, we do parse the metric even when we know we will not use it, based on its timestamp. This only computes the metric when we know its timestamp is right. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	003d6451fc	Promtool: add web config validation Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	5b4f46a348	Add TLS and basic authentication Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ben Kochie	5055dfbbe4	Listen on web early in startup Avoid starting up components like the TSDB if we can't bind to the web listening port. Signed-off-by: Ben Kochie <superq@gmail.com>	4 years ago
beorn7	6bfa33308e	promtool: Print block meta-data slightly more nicely I initially thought I could somehow rescue the current column layout by recycling the tabwriter, but flushing completely blanks it. However, by setting a minimum width of 13, we get a slightly broader DURATION column but otherwise nice formatting, unless numbers get really big, but that's OK, I guess. Before: ``` BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE 01ETN0KGNP5WWK9T5QMQGBG9F1 2020-11-19 07:39:17 +0000 UTC 2020-11-19 07:44:17 +0000 UTC 5m0.001s 8 2 2 624B 01ETN0KGQSFF0AB2QDZVQG3CWC 2020-11-19 10:25:57 +0000 UTC 2020-11-19 10:30:57 +0000 UTC 5m0.001s 8 2 2 622B 01ETN0KGSW8KYP3YPG4X20P60Z 2020-11-19 13:12:37 +0000 UTC 2020-11-19 13:17:37 +0000 UTC 5m0.001s 8 2 2 625B ``` After: ``` BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE 01ETN0R72SXN9A1FG732P7KFFN 2020-11-19 07:39:17 +0000 UTC 2020-11-19 07:44:17 +0000 UTC 5m0.001s 8 2 2 624B 01ETN0R74Y9AG1A1MKN4MZK7WM 2020-11-19 10:25:57 +0000 UTC 2020-11-19 10:30:57 +0000 UTC 5m0.001s 8 2 2 622B 01ETN0R76KXZ5VQECMDNES49J6 2020-11-19 13:12:37 +0000 UTC 2020-11-19 13:17:37 +0000 UTC 5m0.001s 8 2 2 625B ``` After without the `-r` flag: ``` BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE 01ETN0RFFJ42274NWR1GH0RTV6 1605771557000 1605771857001 5m0.001s 8 2 2 624 01ETN0RFJ1MZCHHS2SBZS8XC27 1605781557000 1605781857001 5m0.001s 8 2 2 622 01ETN0RFM98N3V4KD2DZXFGHGN 1605791557000 1605791857001 5m0.001s 8 2 2 625 ``` Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
beorn7	651b57b9ab	Merge branch 'backfillhr' of git://github.com/roidelapluie/prometheus into review	4 years ago
yeya24	cedd2dbec9	create output directory before backfilling Signed-off-by: yeya24 <yb532204897@gmail.com>	4 years ago
Julien Pivotto	53480c168d	Backfill: print created blocks only, add human-readable option Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
AdaephonBen	dca6954b0a	promtool: Add URL scheme when not provided (#7956 ) Signed-off-by: AdaephonBen <ma18btech11011@iith.ac.in>	4 years ago
lzhfromustc	27a6e1e174	test: add buffer to channel to avoid goroutine leak (#8274 ) Signed-off-by: lzhfromustc <lzhfromustc@gmail.com>	4 years ago
Julien Pivotto	7957731339	Inline defer Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	82b5f1d8b1	Backfill: Use mmap to reuse parser code Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
jessicagreben	e32e4fcc53	fix unit test Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	cec3515fa3	fix linter Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	2e9946e4d7	add test Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	ac06d0a657	merge master/resolve conflict Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	ee85c22adb	flush samples to disk every 5k samples Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
Atibhi Agrawal	b317b6ab9c	Backfill from OpenMetrics format (#8084 ) * get parser working Signed-off-by: aSquare14 <atibhi.a@gmail.com> * import file created Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Find min and max ts Signed-off-by: aSquare14 <atibhi.a@gmail.com> * make two passes over file and write to tsdb Signed-off-by: aSquare14 <atibhi.a@gmail.com> * print error messages Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix Max and Min initializer Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Start with unit tests Signed-off-by: aSquare14 <atibhi.a@gmail.com> * reset file read Signed-off-by: aSquare14 <atibhi.a@gmail.com> * align blocks to two hour range Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Add cleanup test Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove .ds_store Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add license to import_test Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix Circle CI error Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Refactor code Move backfill from tsdb to promtool directory Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix gitignore Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Remove panic Rename ContenType Signed-off-by: aSquare14 <atibhi.a@gmail.com> * adjust mint Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix return statement Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix go modules Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Added unit test for backfill Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix CI error Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix file handling Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Close DB Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Close directory Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Error Handling Signed-off-by: aSquare14 <atibhi.a@gmail.com> * inline err Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix command line flags Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add spaces before func fix pointers Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Add defer'd calls Signed-off-by: aSquare14 <atibhi.a@gmail.com> * move openmetrics.go content to backfill Signed-off-by: aSquare14 <atibhi.a@gmail.com> * changed args to flags Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add tests for wrong OM files Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Added additional tests Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Add comment to warn of func reuse Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Make input required in main.go Signed-off-by: aSquare14 <atibhi.a@gmail.com> * defer blockwriter close Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix defer Signed-off-by: aSquare14 <atibhi.a@gmail.com> * defer Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Remove contentType Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove defer from backfilltest Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix defer remove in backfill_test Signed-off-by: aSquare14 <atibhi.a@gmail.com> * changes to fix CI errors Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix go.mod Signed-off-by: aSquare14 <atibhi.a@gmail.com> * change package name Signed-off-by: aSquare14 <atibhi.a@gmail.com> * assert->require Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove todo Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix format Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix todo Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix createblock Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix tests Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix defer Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix return Signed-off-by: aSquare14 <atibhi.a@gmail.com> * check err for anon func Signed-off-by: aSquare14 <atibhi.a@gmail.com> * change comments Signed-off-by: aSquare14 <atibhi.a@gmail.com> * update comment Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix for the Flush Bug Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix formatting, comments, names Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Print Blocks Signed-off-by: aSquare14 <atibhi.a@gmail.com> * cleanup Signed-off-by: aSquare14 <atibhi.a@gmail.com> * refactor test to take care of multiple samples Signed-off-by: aSquare14 <atibhi.a@gmail.com> * refactor tests Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove om Signed-off-by: aSquare14 <atibhi.a@gmail.com> * I dont know what I fixed Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix tests Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fix tests, add test description, print blocks Signed-off-by: aSquare14 <atibhi.a@gmail.com> * commit after 5000 samples Signed-off-by: aSquare14 <atibhi.a@gmail.com> * reviews part 1 Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Series Count Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix CI Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove extra func Signed-off-by: aSquare14 <atibhi.a@gmail.com> * make timestamp into sec Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Reviews 2 Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Add Todo Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Fixes Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fixes reviews Signed-off-by: aSquare14 <atibhi.a@gmail.com> * =0 Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove backfill.om Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add global err var, remove stuff Signed-off-by: aSquare14 <atibhi.a@gmail.com> * change var name Signed-off-by: aSquare14 <atibhi.a@gmail.com> * sampleLimit pass as parameter Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Add test when number of samples greater than batch size Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Change name of batchsize Signed-off-by: aSquare14 <atibhi.a@gmail.com> * revert export Signed-off-by: aSquare14 <atibhi.a@gmail.com> * nits Signed-off-by: aSquare14 <atibhi.a@gmail.com> * remove Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add comment, remove newline,consistent err Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Print Blocks Signed-off-by: aSquare14 <atibhi.a@gmail.com> * Modify comments Signed-off-by: aSquare14 <atibhi.a@gmail.com> * db.Querier Signed-off-by: aSquare14 <atibhi.a@gmail.com> * add sanity check , get maxt and mint Signed-off-by: aSquare14 <atibhi.a@gmail.com> * ci error Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix Signed-off-by: aSquare14 <atibhi.a@gmail.com> * comment change Signed-off-by: aSquare14 <atibhi.a@gmail.com> * nits Signed-off-by: aSquare14 <atibhi.a@gmail.com> * NoError Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix Signed-off-by: aSquare14 <atibhi.a@gmail.com> * fix Signed-off-by: aSquare14 <atibhi.a@gmail.com>	4 years ago
jessicagreben	5dd3577424	change name of promtool subcommand to create-blocks-from Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	19dee0a569	add name and labels to metric, eval all rules for each block Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
gotjosh	4eca4dffb8	Allow metric metadata to be propagated via Remote Write. (#6815 ) * Introduce a metadata watcher Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage. Signed-off-by: gotjosh <josue@grafana.com> * Additional fixes after rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Rework samples/metadata metrics. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use more descriptive variable names in MetadataWatcher collect. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix issues caused during rebasing. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix missing metric add and unneeded config code. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix metrics and docs Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Replace assert with require Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Bring back max_samples_per_send metric Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
jessicagreben	75654715d3	fix panics Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	61c9a89120	use milliseconds for blocksize Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	6980bcf671	unexport backfiller Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	3ed6457dd4	use blockwriter, rm multiwriter code Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	3d8826a3d4	MultiError: Refactored MultiError for more concise and safe usage. (#8066 ) * MultiError: Refactored MultiError for more concise and safe usage. * Less lines * Goland IDE was marking every usage of old MultiError "potential nil" error * It was easy to forgot using Err() when error was returned, now it's safely assured on compile time. NOTE: Potentially I would rename package to merrors. (: In different PR. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed review comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix after rebase. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Julien Pivotto	1282d1b39c	Refactor test assertions (#8110 ) * Refactor test assertions This pull request gets rid of assert.True where possible to use fine-grained assertions. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
David Leadbeater	e7e60623ff	promtool: Calculate mint and maxt per test (#8096 ) * promtool: Calculate mint and maxt per test Previously a single test that used a later eval time would make all other tests in the file share the [mint, maxt] and potentially evaluate far more samples than needed. Fixes: #8019 Signed-off-by: David Leadbeater <dgl@dgl.cx>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
jessicagreben	36ac0b68f1	merge master, fix conflicts	4 years ago
Björn Rabenstein	71577e45eb	Merge pull request #8044 from prometheus/beorn7/metrics Instrumentation: Report valid configs in the respective metrics from the beginning	4 years ago
Arthur Silva Sens	4f45e201cc	Promtool tsdb list now prints block sizes (#7993 ) * promtool tsdb list now prints blocks' size Signed-off-by: arthursens <arthursens2005@gmail.com>	4 years ago
beorn7	0f3c1bf6cf	Report valid configs in the respective metrics from the beginning In #7399, an early validity check of the config was introduced to prevent the scenario where an invalid config is only detected after a possibly very long startup procedure. However, the respective success metrics are not updated after the initial validation so that the success metrics suggest an invalid config. If the startup procedure, like replaying the WAL, really takes very long, alerts about invalid config will trigger. This commit sets the succes metrics after initial validation. They will be set again after the "real" config (re-)load, but that shouldn't be a problem. The metric now truthfully represents whenever the config was successfully loaded, no matter if the result was then thrown away (because it was just for validation) or actually used. Signed-off-by: beorn7 <beorn@grafana.com>	4 years ago
David Leadbeater	5393ec22cb	promtool: Don't end alert tests early, in some failure situations If an alert test had a failing test, then any other alert test interval specified after that point would result in the test exiting early. This made debugging some tests more difficult than needed. Now only exit early for evaluation failures. Signed-off-by: David Leadbeater <dgl@dgl.cx>	4 years ago
Frederic Branczyk	da3ea43242	Merge pull request #7976 from roidelapluie/tolerance Introduce timestamp tolerance in scrapes	4 years ago
Julien Pivotto	be5ba1a62d	Fix wordings Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	4617d16b4b	Specify the removal Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	e2a2bf3c06	Add context Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	627ff84599	Adjust flag Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	6b618ecf02	Better description Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	536dfb6234	Add an experimental, hidden flag Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Frederic Branczyk	6be3ebdfe7	Merge pull request #8015 from simonpasquier/bump-k8s-deps Bump k8s dependencies + support k8s.io/klog/v2	4 years ago
Julien Pivotto	946819e16e	cmd/prometheus: Issue a warning on 32 bit archs (#8012 ) * cmd/prometheus: Issue a warning on 32 bit archs Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Simon Pasquier	9bb3555fe4	cmd/prometheus: support k8s.io/klog/v2 Signed-off-by: Simon Pasquier <spasquie@redhat.com>	4 years ago
David Leadbeater	77c784ac93	Ensure alert rules are marked as restored in unit tests (#7661 ) This makes sure the ALERTS timeseries is created when unit testing alerting rules. Signed-off-by: David Leadbeater <dgl@dgl.cx>	4 years ago
jessicagreben	2e526cf2a7	add output dir parameter Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
jessicagreben	dfa510086b	add alignment, mv rule importer to promtool dir, add queryRange Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
Julien Pivotto	442b3364d7	Promtool: add evaluation time to instant query (#7829 ) * Promtool: add evaluation time to instant query Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Apply suggestion Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Andy Bursavich	4e6a94a27d	Invert service discovery dependencies (#7701 ) This also fixes a bug in query_log_file, which now is relative to the config file like all other paths. Signed-off-by: Andy Bursavich <abursavich@gmail.com>	4 years ago
Harold Dost	21a753c4e2	Make file permissions set to allow for wider umask options. (#7782 ) 0644 -> 0666 on all non vendored code. Fixes #7717 Signed-off-by: Harold Dost <harolddost@gmail.com>	4 years ago
Julien Pivotto	d661f84748	Log duration of reloads Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Annanay	9bba8a6eae	Merge branch 'master' into appender-context Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Julien Pivotto	01e3bfcd1a	Add warnings about NFS (#7691 ) * Add warnings about NFS Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Javier Palomo Almena	b58a613443	Replace sync/atomic with uber-go/atomic (#7683 ) * storage: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * cmd: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * scripts: Verify that we are not using restricted packages It checks that we are not directly importing 'sync/atomic'. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Reorganise imports in blocks Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier/test: Apply PR suggestions Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * storage/remote: avoid storing references on newEntry Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Revert "scripts: Verify that we are not using restricted packages" This reverts commit `278d32748e`. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Group imports accordingly Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
jessicagreben	7504b5ce7c	add rule importer with tsdb block writer Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>	4 years ago
Annanay	7f98a744e5	Add context to Appender interface Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
chinhnc	e05c19da5d	Display block duration in promtool list blocks command (#7653 ) * Update tsdb.go Added DURATION column to `tsdb list` command Signed-off-by: soup <chicknsoupuds@gmail.com> * Use time.Duration instead of hardcoded hour Signed-off-by: soup <chicknsoupuds@gmail.com>	4 years ago
Ben Ye	50c261502e	add tsdb cmds into promtool (#6088 ) Signed-off-by: yeya24 <yb532204897@gmail.com> update tsdb cli in makefile and promu Signed-off-by: yeya24 <yb532204897@gmail.com> remove building tsdb bin Signed-off-by: yeya24 <yb532204897@gmail.com> remove useless func Signed-off-by: yeya24 <yb532204897@gmail.com> refactor analyzeBlock Signed-off-by: yeya24 <yb532204897@gmail.com> Fix Makefile Signed-off-by: Simon Pasquier <spasquie@redhat.com>	4 years ago
Bartlomiej Plotka	a0df8a383a	promql: Removed global and add ability to have better interval for subqueries if not specified (#7628 ) * promql: Removed global and add ability to have better interval for subqueries if not specified ## Changes * Refactored tests for better hints testing * Added various TODO in places to enhance. * Moved DefaultEvalInterval global to opts with func(rangeMillis int64) int64 function instead Motivation: At Thanos we would love to have better control over the subqueries step/interval. This is important to choose proper resolution. I think having proper step also does not harm for Prometheus and remote read users. Especially on stateless querier we do not know evaluation interval and in fact putting global can be wrong to assume for Prometheus even. I think ideally we could try to have at least 3 samples within the range, the same way Prometheus UI and Grafana assumes. Anyway this interfaces allows to decide on promQL user basis. Open question: Is taking parent interval a smart move? Motivation for removing global: I spent 1h fighting with: === RUN TestEvaluations TestEvaluations: promql_test.go:31: unexpected error: error evaluating query "absent_over_time(rate(nonexistant[5m])[5m:])" (line 687): unexpected error: runtime error: integer divide by zero --- FAIL: TestEvaluations (0.32s) FAIL At the end I found that this fails on most of the versions including this master if you run this test alone. If run together with many other tests it passes. This is due to SetDefaultEvaluationInterval(1 * time.Minute) in test that is ran before TestEvaluations. Thanks to globals (: Let's fix it by dropping this global. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added issue links for TODOs. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Removed irrelevant changes. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Julien Pivotto	b83cbacbdd	Rule manager: remove blocking channel in mail (#7631 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ben Ye	e6ea798c32	promtool range query should exit when fail to parse time (#7505 ) Signed-off-by: yeya24 <yb532204897@gmail.com>	4 years ago
yeya24	797e48c1a3	support time range in promtool query labels Updated prometheus/client_golang and json-iterator/go Signed-off-by: yeya24 <yb532204897@gmail.com>	4 years ago
Frederic Branczyk	d17d88935c	rules: Use narrower interface for rule manager loading of for state (#7472 ) To load ALERT_FOR_STATE only `storage.Queryable` interface is required, so this patch uses this narrower interface for to perform this. Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Bartlomiej Plotka	b788986717	storage: Adjusted fully storage layer support for chunk iterators: Remote read client, readyStorage, fanout. (#7059 ) * Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334 * Added ChunkQueryable implementation to fanout and readyStorage. * Added more comments. * Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for both chunks and series, for both querying and compacting (!) + made sure duplicates are merged. * Added ErrChunkSeriesSet * Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion. * Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one. * Improved tests. * Split remote client into Write (old storage) and read. * Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved all config based options to sampleAndChunkQueryableClient to aboid boilerplate. In next commit: Changes for TSDB. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Harkishen Singh	70b0a34616	Exit early on invalid config file (#7399 ) * Reload config file at start Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * relocated config checking Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * change log lever Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * add helpful comment Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Ben Kochie	8d3c2f6829	Enable WAL compression by default (#7410 ) Enable the `--storage.tsdb.wal-compression` flag by defualt. Signed-off-by: Ben Kochie <superq@gmail.com>	5 years ago
Jordan Neufeld	268b4c29e1	Support extended durations in promtool unit tests (Fixes #6285 ) (#6297 ) * Fixed evaluation_time duration parsing in promtool unit tests (Fixes #6285) Signed-off-by: Jordan Neufeld <jordan@neufeldtech.com>	5 years ago
Arthur Silva Sens	7727b9012e	Correction of misleading help text(#5142 ) (#7231 ) * Correction of misleading help text(#5142) Signed-off-by: arthursens <arthursens2005@gmail.com>	5 years ago
Julien Pivotto	9e265aba10	Merge pull request #7225 from prometheus/release-2.18 [Merge without Squash] Merge release-2.18 back to master for 2.18.1 fixes.	5 years ago
Hongcai Ren	c7e82274c6	replace github.com/prometheus/prometheus/testutil/promlint by github.com/prometheus/client_golang/prometheus/testutil/promlint from our codebase (#7209 ) Signed-off-by: RainbowMango <renhongcai@huawei.com>	5 years ago
Julien Pivotto	645b71e9ef	Fix snapshots (#7217 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Ganesh Vernekar	d4b9fe801f	M-map full chunks of Head from disk (#6679 ) When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory Prom startup now happens in these stages - Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks. - Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series. If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss. [Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks. [The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files. In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file. Prombench results _WAL Replay_ 1h Wal reply time 30% less wal reply time - 4m31 vs 3m36 2h Wal reply time 20% less wal reply time - 8m16 vs 7m _Memory During WAL Replay_ High Churn: 10-15% less RAM - 32gb vs 28gb 20% less RAM after compaction 34gb vs 27gb No Churn: 20-30% less RAM - 23gb vs 18gb 40% less RAM after compaction 32.5gb vs 20gb Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ben Ye	1e4e37144d	Fixed wrongly handled not ready TSDB on web and API. (#7182 ) * fix federate endpoint panic Signed-off-by: yeya24 <yb532204897@gmail.com> * Fixed all cases of not ready TSDB being wrongly handled. * Fixed issue for federation. * Ensured this will never happen again thanks to interfaces * Fixes same issue for stats. * Added tests for readiness. * Fixed bug in stats. It was: status.MaxTime = db.Head().MaxTime() status.MinTime = db.Head().MaxTime() Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Vasily Sliouniaev	0393b188c9	Add Jaeger (#7148 ) * Trace remote read Signed-off-by: vas <vasily.sliouniaev@jet.com> * Use jaeger Signed-off-by: vas <vasily.sliouniaev@jet.com>	5 years ago
Marek Slabicki	8224ddec23	Capitalizing first letter of all log lines (#7043 ) Signed-off-by: Marek Slabicki <thaniri@gmail.com>	5 years ago
Brian Brazil	7646cbca32	Use .UTC everywhere we use time.Unix (#7066 ) time.Unix attaches the local timezone, which can then leak out (e.g. in the alert json). While this is harmless, we should be consistent. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	5 years ago
Ben Kochie	269e7c8091	Fix golint issues. Signed-off-by: Ben Kochie <superq@gmail.com>	5 years ago
johncming	bbacd2dd09	remove needless break. (#7008 ) Signed-off-by: johncming <johncming@yahoo.com>	5 years ago
李国忠	52025bd7a9	[comments] change word ‘wheter’ to ‘whether’ (#6912 ) * [comments] change word ‘wheter’ to ‘whether’ Signed-off-by: fuling <fuling.lgz@alibaba-inc.com> * [comments] change word ‘wheter’ to ‘whether’ Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	5 years ago
Tobias Guggenmos	4835bbf376	Merge branch 'master' into split_parser	5 years ago
Bartlomiej Plotka	48ead578a0	Moved tsdbconfig to main. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	a20bebf7eb	Moved readyStorage to main. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	8a775bc468	Moved unit agnostic options to separate pkg. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	59c9d6ef45	Addressed Brian's comments, moved metrics to main.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	cfba92a133	Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	34426766d8	Unify Iterator interfaces. All point to storage now. This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Tobias Guggenmos	454ba12676	Fix build errors in promtool Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Björn Rabenstein	af04cb22c8	Merge pull request #6821 from prometheus/release-2.16 Release 2.16	5 years ago
Julien Pivotto	ff0003e072	Make lookbackDelta a option of QueryEngine (#6746 ) * Make lookbackDelta a option of QueryEngine Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * julius' suggestion Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * remove trivial getter Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Assume lookback delta is always > 0 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * add debug log Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * don't expose loopback delta Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Specify that lookack delta is also used in federation Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix federation test While we have added some logic to the promql engine to keep it backwards compatible and have a 5 minute loopback by default, the web/ package is likely to really be internal to Prometheus and we should not add the same kind of heuritstics here. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * loopback delta: Fix debug log Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	d799078c88	also test start and end Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	881dde505a	promql: fix promql query log step unit Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	3c4c01eae2	Fix race in Query Log Test (#6727 ) A data race can happen if we run t.Log after the test t is done -- which in this case is highly possible because of the use of subtests and the fact that we call t.Log in a goroutine. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	9adad8ad30	Remove MaxConcurrent from the PromQL engine opts (#6712 ) Since we use ActiveQueryTracker to check for concurrency in `d992c36b3a` it does not make sense to keep the MaxConcurrent value as an option of the PromQL engine. This pull request removes it from the PromQL engine options, sets the max concurrent metric to -1 if there is no active query tracker, and use the value of the active query tracker otherwise. It removes dead code and also will inform people who import the promql package that we made that change, as it breaks the EngineOpts struct. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	5f27ac3583	Refactor query log fields (#6694 ) * Refactor query log fields Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	2b2eb79e8b	Add windows tests for query logger (#6653 ) * Add windows tests * Do not rely on time.Time in timer Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	0eb34299da	End-to-end Query Log test (#6600 ) * End-to-end Query Log test Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago

1 2 3 4 5 ...

483 Commits (7cb55d57328c60e4a69e741c4953b97e41bf0be3)