prometheus

Commit Graph

Author	SHA1	Message	Date
yeya24	797e48c1a3	support time range in promtool query labels Updated prometheus/client_golang and json-iterator/go Signed-off-by: yeya24 <yb532204897@gmail.com>	4 years ago
Frederic Branczyk	d17d88935c	rules: Use narrower interface for rule manager loading of for state (#7472 ) To load ALERT_FOR_STATE only `storage.Queryable` interface is required, so this patch uses this narrower interface for to perform this. Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Bartlomiej Plotka	b788986717	storage: Adjusted fully storage layer support for chunk iterators: Remote read client, readyStorage, fanout. (#7059 ) * Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334 * Added ChunkQueryable implementation to fanout and readyStorage. * Added more comments. * Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for both chunks and series, for both querying and compacting (!) + made sure duplicates are merged. * Added ErrChunkSeriesSet * Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion. * Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one. * Improved tests. * Split remote client into Write (old storage) and read. * Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved all config based options to sampleAndChunkQueryableClient to aboid boilerplate. In next commit: Changes for TSDB. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Harkishen Singh	70b0a34616	Exit early on invalid config file (#7399 ) * Reload config file at start Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * relocated config checking Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * change log lever Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * add helpful comment Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	5 years ago
Ben Kochie	8d3c2f6829	Enable WAL compression by default (#7410 ) Enable the `--storage.tsdb.wal-compression` flag by defualt. Signed-off-by: Ben Kochie <superq@gmail.com>	5 years ago
Jordan Neufeld	268b4c29e1	Support extended durations in promtool unit tests (Fixes #6285 ) (#6297 ) * Fixed evaluation_time duration parsing in promtool unit tests (Fixes #6285) Signed-off-by: Jordan Neufeld <jordan@neufeldtech.com>	5 years ago
Arthur Silva Sens	7727b9012e	Correction of misleading help text(#5142 ) (#7231 ) * Correction of misleading help text(#5142) Signed-off-by: arthursens <arthursens2005@gmail.com>	5 years ago
Julien Pivotto	9e265aba10	Merge pull request #7225 from prometheus/release-2.18 [Merge without Squash] Merge release-2.18 back to master for 2.18.1 fixes.	5 years ago
Hongcai Ren	c7e82274c6	replace github.com/prometheus/prometheus/testutil/promlint by github.com/prometheus/client_golang/prometheus/testutil/promlint from our codebase (#7209 ) Signed-off-by: RainbowMango <renhongcai@huawei.com>	5 years ago
Julien Pivotto	645b71e9ef	Fix snapshots (#7217 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Ganesh Vernekar	d4b9fe801f	M-map full chunks of Head from disk (#6679 ) When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory Prom startup now happens in these stages - Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks. - Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series. If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss. [Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks. [The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files. In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file. Prombench results _WAL Replay_ 1h Wal reply time 30% less wal reply time - 4m31 vs 3m36 2h Wal reply time 20% less wal reply time - 8m16 vs 7m _Memory During WAL Replay_ High Churn: 10-15% less RAM - 32gb vs 28gb 20% less RAM after compaction 34gb vs 27gb No Churn: 20-30% less RAM - 23gb vs 18gb 40% less RAM after compaction 32.5gb vs 20gb Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ben Ye	1e4e37144d	Fixed wrongly handled not ready TSDB on web and API. (#7182 ) * fix federate endpoint panic Signed-off-by: yeya24 <yb532204897@gmail.com> * Fixed all cases of not ready TSDB being wrongly handled. * Fixed issue for federation. * Ensured this will never happen again thanks to interfaces * Fixes same issue for stats. * Added tests for readiness. * Fixed bug in stats. It was: status.MaxTime = db.Head().MaxTime() status.MinTime = db.Head().MaxTime() Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Vasily Sliouniaev	0393b188c9	Add Jaeger (#7148 ) * Trace remote read Signed-off-by: vas <vasily.sliouniaev@jet.com> * Use jaeger Signed-off-by: vas <vasily.sliouniaev@jet.com>	5 years ago
Marek Slabicki	8224ddec23	Capitalizing first letter of all log lines (#7043 ) Signed-off-by: Marek Slabicki <thaniri@gmail.com>	5 years ago
Brian Brazil	7646cbca32	Use .UTC everywhere we use time.Unix (#7066 ) time.Unix attaches the local timezone, which can then leak out (e.g. in the alert json). While this is harmless, we should be consistent. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	5 years ago
Ben Kochie	269e7c8091	Fix golint issues. Signed-off-by: Ben Kochie <superq@gmail.com>	5 years ago
johncming	bbacd2dd09	remove needless break. (#7008 ) Signed-off-by: johncming <johncming@yahoo.com>	5 years ago
李国忠	52025bd7a9	[comments] change word ‘wheter’ to ‘whether’ (#6912 ) * [comments] change word ‘wheter’ to ‘whether’ Signed-off-by: fuling <fuling.lgz@alibaba-inc.com> * [comments] change word ‘wheter’ to ‘whether’ Signed-off-by: fuling <fuling.lgz@alibaba-inc.com>	5 years ago
Tobias Guggenmos	4835bbf376	Merge branch 'master' into split_parser	5 years ago
Bartlomiej Plotka	48ead578a0	Moved tsdbconfig to main. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	a20bebf7eb	Moved readyStorage to main. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	8a775bc468	Moved unit agnostic options to separate pkg. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	59c9d6ef45	Addressed Brian's comments, moved metrics to main.go Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	cfba92a133	Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	34426766d8	Unify Iterator interfaces. All point to storage now. This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Tobias Guggenmos	454ba12676	Fix build errors in promtool Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Björn Rabenstein	af04cb22c8	Merge pull request #6821 from prometheus/release-2.16 Release 2.16	5 years ago
Julien Pivotto	ff0003e072	Make lookbackDelta a option of QueryEngine (#6746 ) * Make lookbackDelta a option of QueryEngine Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * julius' suggestion Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * remove trivial getter Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Assume lookback delta is always > 0 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * add debug log Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * don't expose loopback delta Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Specify that lookack delta is also used in federation Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix federation test While we have added some logic to the promql engine to keep it backwards compatible and have a 5 minute loopback by default, the web/ package is likely to really be internal to Prometheus and we should not add the same kind of heuritstics here. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * loopback delta: Fix debug log Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	d799078c88	also test start and end Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	881dde505a	promql: fix promql query log step unit Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	3c4c01eae2	Fix race in Query Log Test (#6727 ) A data race can happen if we run t.Log after the test t is done -- which in this case is highly possible because of the use of subtests and the fact that we call t.Log in a goroutine. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	9adad8ad30	Remove MaxConcurrent from the PromQL engine opts (#6712 ) Since we use ActiveQueryTracker to check for concurrency in `d992c36b3a` it does not make sense to keep the MaxConcurrent value as an option of the PromQL engine. This pull request removes it from the PromQL engine options, sets the max concurrent metric to -1 if there is no active query tracker, and use the value of the active query tracker otherwise. It removes dead code and also will inform people who import the promql package that we made that change, as it breaks the EngineOpts struct. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	5f27ac3583	Refactor query log fields (#6694 ) * Refactor query log fields Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	2b2eb79e8b	Add windows tests for query logger (#6653 ) * Add windows tests * Do not rely on time.Time in timer Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	0eb34299da	End-to-end Query Log test (#6600 ) * End-to-end Query Log test Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	1a58d2657d	Removed compilation step inside main_test (#6658 ) Inspired by https://github.com/prometheus/prometheus/pull/6347 and https://github.com/prometheus/prometheus/pull/6347#issuecomment-570151979 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Harkishen Singh	84e6459c4d	Adds support for line-column numbers for invalid rules, promtool (#6533 ) Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>	5 years ago
Julien Pivotto	3885562587	Query Logging styling (#6594 ) - Fix Json vs JSON in activequerylogger - Fix SetQueryLogger always returns nil Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	9d9bc524e5	Add query log (#6520 ) * Add query log, make stats logged in JSON like in the API Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Simon Pasquier	cccd542891	*: avoid missed Alertmanager targets (#6455 ) This change makes sure that nearly-identical Alertmanager configurations aren't merged together. The config's identifier was the MD5 hash of the configuration serialized to JSON but because `relabel.Regexp` has no public field and doesn't implement the JSON.Marshaler interface, it was always serialized to "{}". In practice, the identifier can be based on the index of the configuration in the list. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	5 years ago
Brooks Swinnerton	0ea3a2218d	Add time units to storage.tsdb.retention.size flag (#6365 ) * Add time units to storage.tsdb.retention.size flag In an effort to reduce confusion with the `m` option of the `ParseDuration()` function, this commit adds the available time units to the `storage.tsdb.retention.time` flag to help showcase that there is no option for months (which could be assumed to be `m`). If someone were looking to set the retention to six months, they may mistakenly do so with `6m`, which would reduce their retention to six minutes. Signed-off-by: Brooks Swinnerton <bswinnerton@gmail.com>	5 years ago
johncming	ad4bc5701e	remove unwanted break (#6338 ) Signed-off-by: johncming <johncming@yahoo.com>	5 years ago
akerele abraham	9d39fdad0c	unittest: check for rule files existence (#6075 ) Signed-off-by: akerele abraham <abrahamakerele38@gmail.com>	5 years ago
Chris Marchbanks	1d1f64b4bc	Fix Promtool showing false duplicate rule warnings (#6270 ) Alert rules do not use the Record field, so any alerts with the same labels and different names would be counted as being duplicates. Promtool will now consider either field when finding duplicates. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	5 years ago
Simon Pasquier	ddff1480a7	cmd/promtool: improve output for PromQL tests (#6052 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	5 years ago
Harkishen Singh	e097c70e6d	add checks for metrics and display duplicate fields (#6026 ) Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	5 years ago
Simon Pasquier	06066a3619	*: improve error messages when parsing bad rules (#5965 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	5 years ago
Sayan Chowdhury	cb66e325d8	Show the warnings during label query (#5924 ) This patch loops through the warnings while querying the label and spits the output to stderr Fixes #5885 Signed-off-by: Sayan Chowdhury <sayan.chowdhury2012@gmail.com>	5 years ago
Bartek Płotka	48b2c9c8ea	remote-read: streamed chunked server side; Extended protobuf; Added chunked, checksumed reader (#5703 ) Part of: https://github.com/prometheus/prometheus/issues/4517 and https://github.com/improbable-eng/thanos/issues/488 Changes: * Extended protobuf for chunked remote read and negotation. * Added checksumed, chunked Writer/Reader. * Added Server side implementation for chunked streamed remote-read. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	5 years ago
Bartek Płotka	5cb32d67f9	Merge pull request #5893 from prometheus/unify-tsdbutil Removed extra tsdb/testutil after merge.	5 years ago

1 2 3 4 5 ...

368 Commits (983ebb4a513302315a8117932ab832815f85e3d2)