Commit Graph

8194 Commits (b90c7a55da04231643895b192d15d499adbc480f)

Author SHA1 Message Date
Julien Pivotto 7eedcc708e promql/parser: Cleanup generatedParserResult accross reuse
Reusing the same generatedParserResult ends up in strange panics:
See #7131 and #7127.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-17 22:06:42 +02:00
Julian Taylor e2c06a8898 register federation failure metrics (#7081)
Closes gh-7080

Signed-off-by: Julian Taylor <juliantaylor108@gmail.com>
2020-04-17 22:06:16 +02:00
Julien Pivotto a2fcdeb1ef Defer finalizer (#7129)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-17 22:05:38 +02:00
Julien Pivotto ed1852ab95
TSDB: Isolation: avoid creating appenderId's without appender (#7135)
Prior to this commit we could have situations where we are creating an
appenderId but never creating an appender to go with it, therefore
blocking the low watermak.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-17 20:51:03 +02:00
beorn7 f9f423ec0a Ensure queries are closed in API calls
Signed-off-by: beorn7 <beorn@grafana.com>
2020-04-17 20:32:36 +02:00
Chris Marchbanks cd12f0873c
Merge pull request #7073 from csmarchbanks/fix-md5-remote-write
Fix remote write not updating when relabel configs or secrets change
2020-04-16 16:36:25 -06:00
Julien Pivotto 209d4bb8a1
Defer finalizer (#7129)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-16 20:16:16 +02:00
Frederic Branczyk 1d6532e9e5
Merge pull request #7132 from roidelapluie/clpql
promql/parser: Cleanup generatedParserResult accross reuse
2020-04-16 15:03:42 +02:00
gotjosh 24af5049bb
API: Allow `TargetRetriever` to receive a Context (#7125)
Fixes #7103

Signed-off-by: gotjosh <josue@grafana.com>
2020-04-16 09:30:47 +01:00
Julien Pivotto 1f6f8e60ee promql/parser: Cleanup generatedParserResult accross reuse
Reusing the same generatedParserResult ends up in strange panics:
See #7131 and #7127.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-16 01:51:08 +02:00
Chris Marchbanks 5ab6b043c1
Always update lastSendTimestamp after a request (#7122)
If the server is returning non-recoverable errors, such as if we are
trying to push samples that are too old, remote write will never
reshard. Non-recoverable errors should be treated the same as success
for the purpose of resharding, just as we do with sample rates and
durations.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-04-15 09:03:28 -06:00
Ben Kochie 59d01913b1
Merge pull request #7114 from prometheus/superq/refactor_sync_makefiles
Refactor makefile script
2020-04-15 13:50:49 +02:00
ZouYu 2b7437d60e
Fix some warnings: 'redundant type from array, slice, or map composite literal' (#7109)
Signed-off-by: ZouYu <zouy.fnst@cn.fujitsu.com>
2020-04-15 11:17:41 +01:00
Callum Styan 386aea7774
Add missing remote write/read config name to docs. (#7105)
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-04-14 09:27:33 -07:00
Chris Marchbanks d88a2b0261 Handle secret changes in remote write ApplyConfig
Remake the http client whenever ApplyConfig is called. This allows
secrets to be updated without needing to restart an otherwise unchanged
queue.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-04-13 23:14:15 +00:00
Simon Pasquier 317e73de79 Hash YAML instead of JSON
But it doesn't work either because of secret fields.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2020-04-13 22:32:37 +00:00
Simon Pasquier 8cc84660fa storage/remote: add tests for config changes
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2020-04-13 22:32:37 +00:00
Julien Pivotto 3df7a0b734
switch to go 1.14 (#7100)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-13 14:55:35 +02:00
Ben Kochie 84342fb184
Refactor makefile script
* Fix shellcheck issues
* Fix some shell style inconsistencies.
* Split out some functions.

Signed-off-by: Ben Kochie <superq@gmail.com>
2020-04-11 13:56:51 +02:00
Marek Slabicki 8224ddec23
Capitalizing first letter of all log lines (#7043)
Signed-off-by: Marek Slabicki <thaniri@gmail.com>
2020-04-11 09:22:18 +01:00
Julien Pivotto b3cf6ef332
synchronize makefile.common for community (#7108)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-04-10 15:34:34 +02:00
Hu Shuai 002b391dfc
Add unit test for Has in pkg/labels/labels.go (#7039)
This PR is about adding a unit test for Has in pkg/labels/labels.go.

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2020-04-09 08:49:09 -07:00
ZouYu f494426f73
fix warning redundant type from array, slice, or map composite literal (#7106)
Signed-off-by: ZouYu <zouy.fnst@cn.fujitsu.com>
2020-04-09 11:29:19 +01:00
Frederic Branczyk 94f577580d
Merge pull request #7104 from cstyan/update-mixin-remote
Update mixin dashboards and alerts for new remote write label names.
2020-04-09 07:46:00 +02:00
Ben Ye 00730bfee7
add rule_group label to rule evaluation metrics (#7094)
Signed-off-by: yeya24 <yb532204897@gmail.com>
2020-04-08 23:21:37 +02:00
Callum Styan 5400e71b91 Update mixin dashboards and alerts for new remote write label names.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-04-08 12:56:00 -07:00
ga 9a21fdcd1b
[storage] clean imports (#7099)
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>
2020-04-07 22:05:39 +01:00
Ben Ye ecda6013ed
Use only local tsdb for federation (#7096)
Signed-off-by: yeya24 <yb532204897@gmail.com>
2020-04-07 16:42:42 +01:00
Frederic Branczyk 34c7f6e3c4
Merge pull request #7035 from tariq1890/rm_deprec
remove deprecated methods from the MetricProvider interface
2020-04-07 14:15:51 +02:00
Muhammad Falak R Wani 2d1a80aa82
rules: manager: clarify doc string for NewGroupMetrics (#7084)
* rules: manager: clarify doc string for NewGroupMetrics

Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
2020-04-07 14:06:01 +02:00
Brian Brazil cd73b3d33e
Reduce how much old WAL we keep around. (#7098)
Previously we were keeping up to around 6 hours of WAL around by
removing 1/3 every hours. This was excessive, so switch to removing 2/3
which will up to around 3 hours of WAL around.

This will roughly halve the size of the WAL and halve startup time for
those who are I/O bound. This may increase the checkpoint size for
those with certain churn patterns, but by much less than we're saving
from the segments.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-04-07 15:55:57 +05:30
Carlos de Paula 79a732203d Add image build for ppc64le architecture
Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-04-06 18:03:58 -03:00
Tariq Ibrahim 0730d6eb74
remove deprecated methods from the MetricProvider interface
Signed-off-by: Tariq Ibrahim <tariq181290@gmail.com>
2020-04-06 09:23:58 -07:00
Brad Walker 3348930df5
Replace fileutil.ReadDir with ioutil.ReadDir (#7029) (#7033)
* tsdb: Replace fileutil.ReadDir with ioutil.ReadDir (#7029)

Signed-off-by: Brad Walker <brad@bradmwalker.com>

* tsdb: Remove fileutil.ReadDir (#7029)

Signed-off-by: Brad Walker <brad@bradmwalker.com>
2020-04-06 19:04:20 +05:30
Julian Taylor 05442b31c8
register federation failure metrics (#7081)
Closes gh-7080

Signed-off-by: Julian Taylor <juliantaylor108@gmail.com>
2020-04-06 09:05:01 +01:00
Chris Marchbanks 62bd77bf93
Fix react tests (#7077)
https://github.com/facebook/create-react-app/issues/8689 is causing our
tests to fail in the CI pipeline. As the comments suggest, downgrading
to react-scripts 3.4.0 fixes the problem.

In addition, fix a test warning due to a missing id field.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-04-01 16:37:52 +02:00
Frederic Hemberger fe47c9c86e
[Docs] consul_sd_config: Add default value for `allow_stale` (#7075)
Ref: https://github.com/prometheus/prometheus/blob/master/discovery/consul/consul.go#L97
Signed-off-by: Frederic Hemberger <mail@frederic-hemberger.de>
2020-03-31 18:55:25 +01:00
MengZeLee a7982ffc0f
Fix typo (#7068)
Fix typo.

Signed-off-by: MengZn <adnt587@gmail.com>
2020-03-30 13:18:34 +05:30
Brian Brazil 7646cbca32
Use .UTC everywhere we use time.Unix (#7066)
time.Unix attaches the local timezone, which can then
leak out (e.g. in the alert json). While this is harmless,
we should be consistent.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-03-29 17:35:39 +01:00
Deepjyoti Mondal c38ca2ca95
Fix #6999 : Add architecture meta label for EC2 (#7000)
This PR adds architecture meta labels for EC2 instances

Signed-off-by: Deepjyoti Mondal <djmdeveloper060796@gmail.com>
2020-03-28 20:41:37 +00:00
Julien Pivotto 0c4ec8d9dd
Merge pull request #6911 from mjtrangoni/remove-buildnametocertificate
scrape/target_test.go: remove deprecated function BuildNameToCertificate()
2020-03-27 17:00:19 +01:00
Julien Pivotto 9057decce2
Merge pull request #7060 from prometheus/release-2.17
Release 2.17
2020-03-27 15:57:07 +01:00
Julien Pivotto ae041f97cf
Release 2.17.1 (#7055)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-26 17:14:12 +01:00
Julien Pivotto 635fff9ea4
Merge pull request #7051 from roidelapluie/revertopt
Revert head posting optimization
2020-03-26 14:54:53 +01:00
Bartlomiej Plotka 104a1313d4
testutil: Enriched Equals with diff on error. (#7053)
## Example:

```

            exp: []chunks.Meta{chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e160), MinTime:1, MaxTime:2}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e180), MinTime:3, MaxTime:3}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e1a0), MinTime:5, MaxTime:5}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e1c0), MinTime:24, MaxTime:6}}

            got: []chunks.Meta{chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e280), MinTime:1, MaxTime:2}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e2a0), MinTime:3, MaxTime:3}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e220), MinTime:5, MaxTime:5}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e240), MinTime:6, MaxTime:6}}

```

Now:

```

            exp: []chunks.Meta{chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e740), MinTime:1, MaxTime:2}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e760), MinTime:3, MaxTime:3}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e780), MinTime:5, MaxTime:5}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e7a0), MinTime:24, MaxTime:6}}

            got: []chunks.Meta{chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e800), MinTime:1, MaxTime:2}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e820), MinTime:3, MaxTime:3}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e860), MinTime:5, MaxTime:5}, chunks.Meta{Ref:0x0, Chunk:(*chunkenc.XORChunk)(0xc00000e880), MinTime:6, MaxTime:6}}

            Diff:
            --- Expected
            +++ Actual
            @@ -50,3 +50,3 @@
               }),
            -  MinTime: (int64) 24,
            +  MinTime: (int64) 6,
               MaxTime: (int64) 6


```
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-03-26 11:48:23 +02:00
Callum Styan c453def8c5
Separate scrape add error checking out into it's own function. (#6930)
* Separate scrape add error checking out into it's own function.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* pass sampleLimitError to checkAddError instead of returning an error

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Return bool, error from checkAddError so we can properly handle
ErrNotFound for AddFast. This should in theory never happen, but the
previous code path handled this case. Adds a test for this, which master
passes and the previous commit fails.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address comment changes.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Move sampleAdded inside the loop iteration within append, since that's
the only block the variable is used in.

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2020-03-25 19:31:48 -07:00
Julien Pivotto ceef10cee4 Reset comment
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-26 00:17:56 +01:00
Julien Pivotto 73228b1b68 Those links should not be reverted
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-25 20:37:26 +01:00
Julien Pivotto 653f343547 Revert head posting optimization
This reverts commit 52630ad0c7.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-03-25 20:19:33 +01:00
Bartlomiej Plotka b1fcfcf9c4
release: Volunteering for 2.18 release. (#7046)
👋 

I am happy to do next release. This is because we spent a lot time recent weeks in query optimizations on Prometheus, Thanos, and Cortex, so 2.18 release might have significant changes like:

* Chunk Iterators finally ❤️ 
* @pstibrany and @pracucci major optimizations to postings
* Potentially https://github.com/prometheus/prometheus/issues/6878 with @mkabischev help. 

I know I was a release Shephard just in December, but given that I have full context for those I feel like I can help in releasing 2.18 a lot.
2020-03-25 18:43:42 +00:00