Commit Graph

471 Commits (9d22f030cff641d4c2fe8bce330b9eaf4b5ba119)

Author SHA1 Message Date
bekbulatov ac702f66eb Resolve merge conflicts 2016-10-18 14:14:24 +01:00
Fabian Reinartz 228bfc1bb5 Merge pull request #2040 from prometheus/kubernetes
Add K8S v2 pod discovery
2016-10-17 20:09:22 +02:00
Fabian Reinartz ce45040e47 kubernetes: fix missing port labels
This commit fixes endpoint port labeling, adjusts tests accordingly
and enhances test delta printing
2016-10-17 11:05:13 +02:00
Frederic Branczyk 8f576a8510 retrieval: add kubernetes endpoint discovery tests 2016-10-17 10:32:10 +02:00
Frederic Branczyk 08fa4eaa92 retrieval: add kubernetes pod discovery tests 2016-10-17 10:32:10 +02:00
Frederic Branczyk 3762e39ce5 retrieval: add kubernetes service discovery tests 2016-10-17 10:32:10 +02:00
Frederic Branczyk 397072a482 retrieval: add kubernetes node discovery tests 2016-10-17 10:32:10 +02:00
Frederic Branczyk cc46058802 retrieval: kubernetes nodes are not namespaced 2016-10-17 10:32:10 +02:00
Frederic Branczyk a318d9ad27 retrieval: fix pod label and annotation prefixes 2016-10-17 10:32:10 +02:00
Fabian Reinartz b24602f713 kubernetes: merge back into single configuration 2016-10-17 10:32:10 +02:00
Fabian Reinartz a9cfb66b28 kubernetes: add node discovery 2016-10-17 10:32:10 +02:00
Fabian Reinartz d896a654f9 kubernetes: Add discovery of services 2016-10-17 10:32:10 +02:00
Fabian Reinartz 6d269ed870 kubernetes: infer pod information in endpoints discovery 2016-10-17 10:32:10 +02:00
Fabian Reinartz 7c439a9060 kubernetes: use and vendor 1.5 client 2016-10-17 10:32:10 +02:00
Fabian Reinartz de22524e57 kubernetes: add KubernetesV2 endpoints 2016-10-17 10:32:10 +02:00
Fabian Reinartz 2331701b50 kubernetes: Add K8S v2 pod discovery
This adds plumbing for a parallel version of the new K8S SD
and adds pod discovery as the first role.
2016-10-17 10:32:10 +02:00
Dominik Schulz bfa7099616 Report GCE instance metdata (#2084)
* Report GCE instance metdata

* Fix spelling acording to code review guidelines

* Address review comments
2016-10-17 09:45:43 +02:00
Dominik Schulz c73aa82589 Add GCE Instance Status 2016-10-08 08:40:12 +02:00
bekbulatov 01b53c1180 Add tls support 2016-10-07 13:40:22 +01:00
Roman Vynar db63a4bd2a
Do not fail Consul discovery on Prometheus startup when Consul is down. 2016-09-26 22:20:56 +03:00
Dominik Schulz f6fbcf9aa2 Expose ec2_instance_state 2016-09-22 15:01:23 +02:00
Tom Wilkie 4520e12440 Add HTTP Basic Auth & TLS support to the generic write path. (#1957)
* Add config, HTTP Basic Auth and TLS support to the generic write path.

- Move generic write path configuration to the config file
- Factor out config.TLSConfig -> tlf.Config translation
- Support TLSConfig for generic remote storage
- Rename Run to Start, and make it non-blocking.
- Dedupe code in httputil for TLS config.
- Make remote queue metrics global.
2016-09-19 22:47:51 +02:00
Matt Bostock 4fc619b605 Scrape: Remove JSON from Accept request header
JSON is no longer supported as an exposition format [1] [2] [3]. Remove
it from the `Accept` header added to requests when scraping targets.

[1]: https://github.com/prometheus/prometheus/blob/master/CHANGELOG.md#100--2016-07-18
[2]: https://prometheus.io/docs/instrumenting/exposition_formats/#historical-versions
[3]: https://docs.google.com/document/d/1ZjyKiKxZV83VI9ZKAXRGKaUKK2BIWCT7oiGBKDBpjEY/edit?usp=sharing
2016-09-17 10:28:03 +01:00
Ingo Gottwald 3b546d061f Add support for GCE discovery 2016-09-16 08:55:33 +02:00
Tobias Schmidt 29ced0090f Fix common english misspellings 2016-09-14 23:23:28 -04:00
Tobias Schmidt 27074863b4 Print url.URLs correctly in tests 2016-09-14 23:15:18 -04:00
Tobias Schmidt 8f3b62bfe4 Simplify struct initialization 2016-09-14 23:13:27 -04:00
Dan Milstein 0cb6b9962e Fix broken test which relied on DNS resolution #1962
Switched to testing by way of the static_configs rather than
dns_sd_config parameter.  Verified that the revised test both passes
without network access, and also still catches the bug it's supposed to
cover.
2016-09-08 16:59:46 -04:00
Fabian Reinartz fec3b54cfc Merge pull request #1946 from prometheus/ipv6
Fix IPv6 scraping
2016-09-06 17:18:28 +02:00
Fabian Reinartz a15237a0b8 retrieval: correctly handle IPv6 addresses
This updates all service discoveries to correctly
build the __address__ label for IPv6 addresses.
2016-09-06 15:06:49 +02:00
Fabian Reinartz 17cdd4f966 retrieval: fix IPv6 port default, add tests
This fixes port defaulting for IPv6 addresses and restructures
and test the construction of target label sets.
2016-09-06 15:06:48 +02:00
Fabian Reinartz 0322c59dc3 retrieval: export NewHTTPClient 2016-09-05 16:44:40 +02:00
Dan Milstein b9fb9742ed Move test helper function into scope of test func 2016-08-29 16:08:40 -04:00
Dan Milstein 79216011cb Add basic test for TargetManager.targetSet
Verify that if the configs change, target groups are cleaned on
TargetManager.reload (rather than having old ones linger around, even if
they are no longer present in the configs).

This covers the bug fixed in #1907 -- I verified that by checking out
source from before that commit.

This is a start on #1906
2016-08-26 14:30:26 -04:00
Björn Rabenstein 4b8f963847 Merge pull request #1915 from prometheus/release-1.0
Forward-merge the bug fix from release-1.0
2016-08-24 13:04:45 +02:00
beorn7 e2b3626e0c retrieval: Clean up target group map on config reload
Also, remove unused `providers` field in targetSet.

If the config file changes, we recreate all providers (by calling
`providersFromConfig`) and retrieve all targets anew from the newly
created providers. From that perspective, it cannot harm to clean up
the target group map in the targetSet. Not doing so (as it was the
case so far) keeps stale targets around. This mattered if an existing
key in the target group map was not overwritten in the initial fetch
of all targets from the providers. Examples where that mattered:

```
scrape_configs:
- job_name: "foo"
  static_configs:
  - targets: ["foo:9090"]
  - targets: ["bar:9090"]
```
updated to:
```
scrape_configs:
- job_name: "foo"
  static_configs:
  - targets: ["foo:9090"]
```

`bar:9090` would still be monitored. (The static provider just
enumerates the target groups. If the number of target groups
decreases, the old ones stay around.

```
scrape_configs:
- job_name: "foo"
  dns_sd_configs:
  - names:
    - "srv.name.one.example.org"
```
updated to:
```
scrape_configs:
- job_name: "foo"
  dns_sd_configs:
  - names:
    - "srv.name.two.example.org"
```

Now both SRV records are still monitored. The SRV name is part of the
key in the target group map, thus the new one is just added and the
old ane stays around.

Obviously, this should have tests, and should have tests before, not
only for this case. This is the quick fix. I have created
https://github.com/prometheus/prometheus/issues/1906 to track test
creation.

Fixes https://github.com/prometheus/prometheus/issues/1610 .
2016-08-22 19:25:33 +02:00
Anders Daljord Morken 95cadd0702 Run scrape loop with interval 1 instead of 0
0 is considered an invalid interval by time.NewTicker() and will cause a
panic if control reaches that point. Given the vagaries of timekeeping,
this may occasionally happen and make this test unstable.
2016-08-18 09:39:11 +02:00
Anders Daljord Morken 8633ac180e Strip stray whitespace from bearer token file
Apart from not trying to send a newline in a HTTP header,
this also allows Prometheus to build and pass tests with Go 1.7,
which features stricter checking of HTTP headers.
2016-08-17 15:36:18 +02:00
Frederic Branczyk 7714b9c781 move relabeling functionality to its own package
also remove the returned error as it was always nil
2016-08-09 14:19:20 +02:00
Jimmi Dyson 6c8080607f
Kubernetes SD: Add node name and host IP to pod discovery 2016-07-20 12:00:54 +01:00
Dmitry Vorobev 273e457da4 web: return status code and error message for config resource 2016-07-15 10:15:24 +02:00
beorn7 064b57858e Consistently use the `Seconds()` method for conversion of durations
This also fixes one remaining case of recording integral numbers
of seconds only for a metric, i.e. this will probably fix #1796.
2016-07-07 15:24:35 +02:00
Fabian Reinartz 4591a2623b discovery/kubernetes: filter pod/container, service/endpoint
This change distinguishes and filters by pod/container and
service/endpoint in the respective sub-SDs.
2016-07-05 14:24:17 +02:00
Fabian Reinartz 0ff354341b discovery/kubernetes: remove unused channel 2016-07-05 14:22:12 +02:00
Fabian Reinartz 7221228843 discovery/kubernetes: select between discovery role
This adds `role` field to the Kubernetes SD config, which indicates
which type of Kubernetes SD should be run.
This no longer allows discovering pods and nodes with the same SD
configuration for example.
2016-07-05 14:22:12 +02:00
Fabian Reinartz e0f8caacd7 discovery/kubernetes: extract service endpoint discovery
This extract discovery of services and their endpoints into its own
type.
2016-07-05 10:26:23 +02:00
Fabian Reinartz fdbe28df85 discovery/kubernetes: extract node discovery
This change extracts node discovery into its own type.
2016-07-01 19:31:04 +02:00
Fabian Reinartz 8a97c211a8 discovery/kubernetes: extract pod discovery
This change extracts pod discovery into its own type.
2016-07-01 19:30:00 +02:00
Fabian Reinartz e03e138d34 discovery: consolidate constructors into single file 2016-07-01 19:30:00 +02:00
Fabian Reinartz 57333d1831 discovery/kubernetes: add missing locking 2016-07-01 17:07:13 +02:00
Fabian Reinartz 44036a08d0 Merge pull request #1725 from nicholascapo/use-consul-service-address
discovery: use consul service address if available
2016-06-30 09:30:14 +02:00
Nicholas Capo 84334a8410 discovery: use consul service address if available 2016-06-15 19:27:05 -05:00
Fabian Reinartz 4aeab798e8 Merge pull request #1738 from prometheus/release-0.19
Forward-merge 0.19 fixes into master
2016-06-14 18:11:47 +02:00
Fabian Reinartz 3c80609fce Merge pull request #1737 from prometheus/fabxc-0.19.3
Bump version to 0.19.3
2016-06-14 18:04:56 +02:00
Fabian Reinartz 4c864c8a88 retrieval: don't sync to uninitialized scrape pool
This change does just signal a scrape target update to the scraping loop
once an initial target set is fetched.
Before, the scrape pool was directly synced, causing a race against an
uninitialized scrape pool.

Fixes #1703
2016-06-14 14:18:40 +02:00
Fabian Reinartz d0eeae9d0e retrieval: don't sync to uninitialized scrape pool
This change does just signal a scrape target update to the scraping loop
once an initial target set is fetched.
Before, the scrape pool was directly synced, causing a race against an
uninitialized scrape pool.

Fixes #1703
2016-06-14 14:04:22 +02:00
beorn7 03adbe57e4 discovery/marathon: Fix race conditions in test
The concurrency applied before is in most cases not even needed. With
a cap=1 channel, most tests are much cleaner.

TestMarathonSDRunAndStop was trickier. It could even have blocked
before.

This also includes a general refactoring of the whole file.
2016-06-14 13:12:53 +02:00
Björn Rabenstein 2ea3a837c3 Merge pull request #1731 from prometheus/release-0.19
Forward-merge 0.19 fixes to master
2016-06-14 09:21:52 +02:00
rohit01 47dd5f74ba discovery/marathon: #1722 - ignore apps with zero ports 2016-06-14 04:44:00 +05:30
Fabian Reinartz 0f21bd31ca config: deprecate `target_groups` for `static_configs`
This change deprecates the `target_groups` option in favor
of `static_configs`. The old configuration is still accepted
but prints a warning.
Configuration loading errors if both options are set.
2016-06-08 15:55:25 +02:00
Brian Brazil 05b918a024 Merge pull request #1713 from mattbostock/document_drop
Relabel: Document whole label set is dropped
2016-06-07 17:22:34 +01:00
Matt Bostock 329a00e44f Relabel: Document whole label set is dropped
From the documentation and current tests, it wasn't immediately clear to
me whether the `target` being dropped as the result of a 'drop' action
was a label key-value pair or the entire labelset.

Add a test that documents this behaviour.

Documentation: https://prometheus.io/docs/operating/configuration/
2016-06-07 17:13:30 +01:00
Jimmi Dyson 206bcfcdaa
Kubernetes SD: Remove kubeletPort config option 2016-06-07 12:34:55 +01:00
Jimmi Dyson d48297c904
Kubernetes SD: Add labels for all node addresses and discover node port if available 2016-06-07 12:34:49 +01:00
Fabian Reinartz 26b1c89469 Merge pull request #1702 from pdbogen/master
Initial local pods before using the pod list to initialize pod targets
2016-06-06 15:14:49 +02:00
Patrick Bogen 1e6770cdc5 Initial local pods before using the pod list to initialize pod targets; include more logging in pod target creation 2016-06-02 17:49:21 -07:00
Ali Reza c81b4e8a87 change config names to files for consistency 2016-05-30 07:47:58 +07:00
Gregory G. Tseng 4ceedffe86 Unexport testing constant 2016-05-26 14:42:17 -07:00
Gregory G. Tseng 7997c14b0d Add ServerName into TLS Config 2016-05-26 14:24:49 -07:00
Fabian Reinartz 74c448386c Merge pull request #1665 from prometheus/fabxc-retrpanic
Fix kubernetes SD crash
2016-05-25 17:13:27 -07:00
Fabian Reinartz 12b03db373 retrieval: handle nil target groups from updates 2016-05-25 16:59:16 -07:00
Fabian Reinartz ea36efbbd1 retrieval: document panic behavior 2016-05-25 16:17:25 -07:00
Fabian Reinartz a5ba166935 retrieval: don't panic on non-HTTP scheme 2016-05-25 16:05:20 -07:00
Tobias Schmidt 0c6ed9d437 Fix type usage in Kubernetes discovery
The event types have been recently changed to be exported types which
wasn't reflected in the new pod discovery.
2016-05-20 11:03:19 -04:00
Fabian Reinartz f7ed2ff706 Merge pull request #1644 from prometheus/beorn7/logging
Add missing logging of out-of-order samples
2016-05-20 05:52:00 -07:00
Fabian Reinartz dec56838fc Merge pull request #1449 from pdbogen/master
k8s pod discovery
2016-05-20 05:45:09 -07:00
Patrick Bogen 89940eb48d Write tests to include testing determinancy of various slice orders; ensure that container order is deterministic 2016-05-19 10:57:23 -07:00
Patrick Bogen b3350d872a Add one label named for each port name, mapping it to port number; add corresponding tests; prefix port list label with a comma 2016-05-19 10:37:11 -07:00
beorn7 d43c0159aa Fix style issues in retrieval/... 2016-05-19 17:14:04 +02:00
beorn7 45e5775f9b Add missing logging of out-of-order samples
So far, out-of-order samples during rule evaluation were not logged,
and neither scrape health samples. The latter are unlikely to cause
any errors. That's why I'm logging them always now. (It's alway highly
irregular should it happen.) For rules, I have used the same plumbing
as for samples, just with a different wording in the message to mark
them as a result of rule evaluation.
2016-05-19 16:22:53 +02:00
Patrick Bogen ae413704e8 kubernetes pod-level discovery 2016-05-18 17:18:52 -07:00
Fabian Reinartz 2ca9ee7b0d Merge pull request #1612 from prometheus/stn-dns-mv
[discovery][dns] move dns to own package
2016-05-06 12:23:59 +02:00
stuart nelson d959d2b90a discovery/dns: Maintain argument order consistency 2016-05-06 11:14:26 +02:00
stuart nelson d02591814b discovery/dns: move dns to own package 2016-05-06 11:14:26 +02:00
Shawn Smith d7481f266e fix typo 2016-05-04 13:42:27 +09:00
Tobias Schmidt 3ce895e86d Merge pull request #1608 from prometheus/fix-printf
Fix format argument in retrieval test.
2016-05-01 17:44:16 -04:00
Julius Volz 97b018d26d Fix format argument in retrieval test. 2016-05-01 23:37:45 +02:00
Fabian Reinartz f94fc76608 Merge pull request #1592 from prometheus/fabxc-consul-ref
discovery: sanitize Consul service discovery
2016-04-30 21:18:33 +02:00
Fabian Reinartz 289f306dd9 Merge pull request #1590 from prometheus/fabxc-marathon-ref
Fix basic issues in marathon SD
2016-04-30 21:17:22 +02:00
Fabian Reinartz 76076bfb47 discovery: simplify client initialization 2016-04-30 21:07:49 +02:00
Fabian Reinartz b5bfb502df discovery: properly check context on chan send 2016-04-30 11:57:20 +02:00
Fabian Reinartz 9f8feb9ff6 discovery: consolidate Marathon SD files 2016-04-30 11:56:11 +02:00
Fabian Reinartz 086f7caceb discovery: extract Consul shouldWatch logic 2016-04-30 11:50:19 +02:00
Fabian Reinartz e805e68c01 discovery: sanitize Consul service discovery
This commits simplifies the SD's structure and ensures that all
channel sends are checked against a canceled context.
2016-04-30 11:50:19 +02:00
Fabian Reinartz 5837e6a97f discovery: move consul SD into own package 2016-04-25 16:56:27 +02:00
beorn7 d566808d40 Bring back logging of discarded samples
But only on DEBUG level.

Also, count and report the two cases of out-of-order timestamps on the
one hand and same timestamp but different value on the other hand
separately.
2016-04-25 16:43:52 +02:00
Fabian Reinartz 585ab6b163 Merge pull request #1494 from iamseth/master
Add discovery capability for Microsoft Azure
2016-04-21 13:49:44 +02:00
Jonathan Boulle 38098f8c95 Add missing license headers
Prometheus is Apache 2 licensed, and most source files have the
appropriate copyright license header, but some were missing it without
apparent reason. Correct that by adding it.
2016-04-13 16:08:22 +02:00
Seth Miller 0988e3b937 Add support for Azure discovery
This change adds the ability to do target discovery with Microsoft's Azure platform.
2016-04-06 22:47:02 -05:00
Fabian Reinartz 769389e559 Fix potential race in ctx intialization 2016-04-05 20:27:31 +02:00