prometheus/config/testdata/conf.good.yml

# my global config
global:
  scrape_interval:     15s
  evaluation_interval: 30s
  # scrape_timeout is set to the global default (10s).

  external_labels:
    monitor: codelab
    foo:     bar

rule_files:
- "first.rules"
- "my/*.rules"

remote_write:
  - url: http://remote1/push
    write_relabel_configs:
    - source_labels: [__name__]
      regex:         expensive.*
      action:        drop
  - url: http://remote2/push

remote_read:
  - url: http://remote1/read
    read_recent: true
  - url: http://remote3/read
    read_recent: false
    required_matchers:
      job: special

scrape_configs:
- job_name: prometheus

  honor_labels: true
  # scrape_interval is defined by the configured global (15s).
  # scrape_timeout is defined by the global default (10s).

  # metrics_path defaults to '/metrics'
  # scheme defaults to 'http'.

  file_sd_configs:
    - files:
      - foo/*.slow.json
      - foo/*.slow.yml
      - single/file.yml
      refresh_interval: 10m
    - files:
      - bar/*.yaml

  static_configs:
  - targets: ['localhost:9090', 'localhost:9191']
    labels:
      my:   label
      your: label

  relabel_configs:
  - source_labels: [job, __meta_dns_name]
    regex:         (.*)some-[regex]
    target_label:  job
    replacement:   foo-${1}
    # action defaults to 'replace'
  - source_labels: [abc]
    target_label:  cde
  - replacement:   static
    target_label:  abc
  - regex:
    replacement:   static
    target_label:  abc

  bearer_token_file: valid_token_file


- job_name: service-x

  basic_auth:
    username: admin_name
    password: "multiline\nmysecret\ntest"

  scrape_interval: 50s
  scrape_timeout:  5s

  sample_limit: 1000

  metrics_path: /my_path
  scheme: https

  dns_sd_configs:
  - refresh_interval: 15s
    names:
    - first.dns.address.domain.com
    - second.dns.address.domain.com
  - names:
    - first.dns.address.domain.com
    # refresh_interval defaults to 30s.

  relabel_configs:
  - source_labels: [job]
    regex:         (.*)some-[regex]
    action:        drop
  - source_labels: [__address__]
    modulus:       8
    target_label:  __tmp_hash
    action:        hashmod
  - source_labels: [__tmp_hash]
    regex:         1
    action:        keep
  - action:        labelmap
    regex:         1
  - action:        labeldrop
    regex:         d
  - action:        labelkeep
    regex:         k

  metric_relabel_configs:
  - source_labels: [__name__]
    regex:         expensive_metric.*
    action:        drop

- job_name: service-y

  consul_sd_configs:
  - server: 'localhost:1234'
    token: mysecret
    services: ['nginx', 'cache', 'mysql']
    tag: "canary"
    node_meta:
      rack: "123"
    allow_stale: true
    scheme: https
    tls_config:
      ca_file: valid_ca_file
      cert_file: valid_cert_file
      key_file:  valid_key_file
      insecure_skip_verify: false

  relabel_configs:
  - source_labels: [__meta_sd_consul_tags]
    separator:     ','
    regex:         label:([^=]+)=([^,]+)
    target_label:  ${1}
    replacement:   ${2}

- job_name: service-z

  tls_config:
    cert_file: valid_cert_file
    key_file: valid_key_file

  bearer_token: mysecret

- job_name: service-kubernetes

  kubernetes_sd_configs:
  - role: endpoints
    api_server: 'https://localhost:1234'

    basic_auth:
      username: 'myusername'
      password: 'mysecret'

- job_name: service-kubernetes-namespaces

  kubernetes_sd_configs:
  - role: endpoints
    api_server: 'https://localhost:1234'
    namespaces:
      names:
        - default

- job_name: service-marathon
  marathon_sd_configs:
  - servers:
    - 'https://marathon.example.com:443'

    tls_config:
      cert_file: valid_cert_file
      key_file: valid_key_file

- job_name: service-ec2
  ec2_sd_configs:
    - region: us-east-1
      access_key: access
      secret_key: mysecret
      profile: profile

- job_name: service-azure
  azure_sd_configs:
    - subscription_id: 11AAAA11-A11A-111A-A111-1111A1111A11
      tenant_id: BBBB222B-B2B2-2B22-B222-2BB2222BB2B2
      client_id: 333333CC-3C33-3333-CCC3-33C3CCCCC33C
      client_secret: mysecret
      port: 9100

- job_name: service-nerve
  nerve_sd_configs:
    - servers:
      - localhost
      paths:
      - /monitoring

- job_name: 0123service-xxx
  metrics_path: /metrics
  static_configs:
    - targets:
      - localhost:9090

- job_name: 測試
  metrics_path: /metrics
  static_configs:
    - targets:
      - localhost:9090

- job_name: service-triton
  triton_sd_configs:
  - account: 'testAccount'
    dns_suffix: 'triton.example.com'
    endpoint: 'triton.example.com'
    port: 9163
    refresh_interval: 1m
    version: 1
    tls_config:
      cert_file: testdata/valid_cert_file
      key_file: testdata/valid_key_file

alerting:
  alertmanagers:
  - scheme: https
    static_configs:
    - targets:
      - "1.2.3.4:9093"
      - "1.2.3.5:9093"
      - "1.2.3.6:9093"
Switch config to YAML format. 10 years ago			`# my global config`
Replace example config with new YAML format. 10 years ago			`global:`
Switch config to YAML format. 10 years ago			`scrape_interval: 15s`
			`evaluation_interval: 30s`
			`# scrape_timeout is set to the global default (10s).`

Rename global "labels" config option to "external_labels". 9 years ago			`external_labels:`
config: extend and format config example/test. 10 years ago			`monitor: codelab`
			`foo: bar`
Switch config to YAML format. 10 years ago
			`rule_files:`
config: extend and format config example/test. 10 years ago			`- "first.rules"`
			`- "my/*.rules"`
Switch config to YAML format. 10 years ago
Add support for remote write relabelling. Switch back to a single remote writer, as we were only ever meant to have one and the relabel semantics are clearer that way. 8 years ago			`remote_write:`
Re-add multiple remote writers Each remote write endpoint gets its own set of relabeling rules. This is based on the (yet-to-be-merged) https://github.com/prometheus/prometheus/pull/2419, which removes legacy remote write implementations. 8 years ago			`- url: http://remote1/push`
			`write_relabel_configs:`
			`- source_labels: [__name__]`
			`regex: expensive.*`
			`action: drop`
			`- url: http://remote2/push`
Add support for remote write relabelling. Switch back to a single remote writer, as we were only ever meant to have one and the relabel semantics are clearer that way. 8 years ago
Remote storage reads based on oldest timestamp in primary storage (#3129) Currently all read queries are simply pushed to remote read clients. This is fine, except for remote storage for wich it unefficient and make query slower even if remote read is unnecessary. So we need instead to compare the oldest timestamp in primary/local storage with the query range lower boundary. If the oldest timestamp is older than the mint parameter, then there is no need for remote read. This is an optionnal behavior per remote read client. Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com> 7 years ago			`remote_read:`
			`- url: http://remote1/read`
			`read_recent: true`
			`- url: http://remote3/read`
			`read_recent: false`
Add remote read filter option For special remote read endpoints which have only data for specific queries, it is desired to limit the number of queries sent to the configured remote read endpoint to reduce latency and performance overhead. 7 years ago			`required_matchers:`
			`job: special`
Remote storage reads based on oldest timestamp in primary storage (#3129) Currently all read queries are simply pushed to remote read clients. This is fine, except for remote storage for wich it unefficient and make query slower even if remote read is unnecessary. So we need instead to compare the oldest timestamp in primary/local storage with the query range lower boundary. If the oldest timestamp is older than the mint parameter, then there is no need for remote read. This is an optionnal behavior per remote read client. Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com> 7 years ago
Switch config to YAML format. 10 years ago			`scrape_configs:`
config: extend and format config example/test. 10 years ago			`- job_name: prometheus`
Switch config to YAML format. 10 years ago
retrieval: add honor label handling and parametrized querying. This commit adds the honor_labels and params arguments to the scrape config. This allows to specify query parameters used by the scrapers and handling scraped labels with precedence. 10 years ago			`honor_labels: true`
config: extend and format config example/test. 10 years ago			`# scrape_interval is defined by the configured global (15s).`
			`# scrape_timeout is defined by the global default (10s).`
Switch config to YAML format. 10 years ago
config: extend and format config example/test. 10 years ago			`# metrics_path defaults to '/metrics'`
			`# scheme defaults to 'http'.`
Switch config to YAML format. 10 years ago
config: extend and format config example/test. 10 years ago			`file_sd_configs:`
change config names to files for consistency 9 years ago			`- files:`
config: extend and format config example/test. 10 years ago			`- foo/*.slow.json`
			`- foo/*.slow.yml`
			`- single/file.yml`
			`refresh_interval: 10m`
change config names to files for consistency 9 years ago			`- files:`
config: extend and format config example/test. 10 years ago			`- bar/*.yaml`

config: deprecate `target_groups` for `static_configs` This change deprecates the `target_groups` option in favor of `static_configs`. The old configuration is still accepted but prints a warning. Configuration loading errors if both options are set. 9 years ago			`static_configs:`
config: extend and format config example/test. 10 years ago			`- targets: ['localhost:9090', 'localhost:9191']`
Switch config to YAML format. 10 years ago			`labels:`
config: extend and format config example/test. 10 years ago			`my: label`
			`your: label`

			`relabel_configs:`
Update two more __meta_dns_srv_name references. Although they are only in examples/tests and don't affect anything, they could be confusing (the label has been renamed in the rest of the code a while ago). 9 years ago			`- source_labels: [job, __meta_dns_name]`
Change config regexes to full-string matches. This anchors all regular expressions entered via the config to match a full string vs. a substring. THIS IS A BREAKING CHANGE! Fixes part of https://github.com/prometheus/prometheus/issues/996 9 years ago			`regex: (.*)some-[regex]`
config: extend and format config example/test. 10 years ago			`target_label: job`
			`replacement: foo-${1}`
			`# action defaults to 'replace'`
Add new defaults for relabel configurations 9 years ago			`- source_labels: [abc]`
			`target_label: cde`
			`- replacement: static`
			`target_label: abc`
Handle null Regex in the config as the empty regex. (#2150) 8 years ago			`- regex:`
			`replacement: static`
			`target_label: abc`
config: extend and format config example/test. 10 years ago
Resolve relative paths on configuration loading This moves the concern of resolving the files relative to the config file into the configuration loading itself. It also fixes #921 which did not load the cert and token files relatively. 9 years ago			`bearer_token_file: valid_token_file`

config: extend and format config example/test. 10 years ago
			`- job_name: service-x`

			`basic_auth:`
config: hide authentication credentials in String() output 10 years ago			`username: admin_name`
Replace regex with Secret type and remarshal config to hide secrets (#2775) 8 years ago			`password: "multiline\nmysecret\ntest"`
config: extend and format config example/test. 10 years ago
			`scrape_interval: 50s`
			`scrape_timeout: 5s`

Add sample_limit to scrape config. This imposes a hard limit on the number of samples ingested from the target. This is counted after metric relabelling, to allow dropping of problemtic metrics. This is intended as a very blunt tool to prevent overload due to misbehaving targets that suddenly jump in sample count (e.g. adding a label containing email addresses). Add metric to track how often this happens. Fixes #2137 8 years ago			`sample_limit: 1000`

config: extend and format config example/test. 10 years ago			`metrics_path: /my_path`
			`scheme: https`

			`dns_sd_configs:`
			`- refresh_interval: 15s`
			`names:`
			`- first.dns.address.domain.com`
			`- second.dns.address.domain.com`
			`- names:`
			`- first.dns.address.domain.com`
			`# refresh_interval defaults to 30s.`

			`relabel_configs:`
			`- source_labels: [job]`
Change config regexes to full-string matches. This anchors all regular expressions entered via the config to match a full string vs. a substring. THIS IS A BREAKING CHANGE! Fixes part of https://github.com/prometheus/prometheus/issues/996 9 years ago			`regex: (.*)some-[regex]`
config: extend and format config example/test. 10 years ago			`action: drop`
Add 'hashmod' relabel action. This takes the modulus of a hash of some labels. Combined with a keep relabel action, this allows for sharding of targets across multiple prometheus servers. 10 years ago			`- source_labels: [__address__]`
			`modulus: 8`
config: Update tests/examples to use __tmp_ 9 years ago			`target_label: __tmp_hash`
Add 'hashmod' relabel action. This takes the modulus of a hash of some labels. Combined with a keep relabel action, this allows for sharding of targets across multiple prometheus servers. 10 years ago			`action: hashmod`
config: Update tests/examples to use __tmp_ 9 years ago			`- source_labels: [__tmp_hash]`
Change config regexes to full-string matches. This anchors all regular expressions entered via the config to match a full string vs. a substring. THIS IS A BREAKING CHANGE! Fixes part of https://github.com/prometheus/prometheus/issues/996 9 years ago			`regex: 1`
Add 'hashmod' relabel action. This takes the modulus of a hash of some labels. Combined with a keep relabel action, this allows for sharding of targets across multiple prometheus servers. 10 years ago			`action: keep`
Allow labelmap action 9 years ago			`- action: labelmap`
			`regex: 1`
Add labeldrop and labelkeep actions. (#2279) Introduce two new relabel actions. labeldrop, and labelkeep. These can be used to filter the set of labels by matching regex - labeldrop: drops all labels that match the regex - labelkeep: drops all labels that do not match the regex 8 years ago			`- action: labeldrop`
			`regex: d`
			`- action: labelkeep`
			`regex: k`
config: extend and format config example/test. 10 years ago
Allow ingested metrics to be relabeled. The main purpose of this is to allow for blacklisting of expensive metrics as a tactical option. It could also find uses for renaming and removing labels from federation. 10 years ago			`metric_relabel_configs:`
			`- source_labels: [__name__]`
Change config regexes to full-string matches. This anchors all regular expressions entered via the config to match a full string vs. a substring. THIS IS A BREAKING CHANGE! Fixes part of https://github.com/prometheus/prometheus/issues/996 9 years ago			`regex: expensive_metric.*`
Allow ingested metrics to be relabeled. The main purpose of this is to allow for blacklisting of expensive metrics as a tactical option. It could also find uses for renaming and removing labels from federation. 10 years ago			`action: drop`
config: extend and format config example/test. 10 years ago
			`- job_name: service-y`

			`consul_sd_configs:`
			`- server: 'localhost:1234'`
Hide consul token on UI. (#2797) 8 years ago			`token: mysecret`
Allow ingested metrics to be relabeled. The main purpose of this is to allow for blacklisting of expensive metrics as a tactical option. It could also find uses for renaming and removing labels from federation. 10 years ago			`services: ['nginx', 'cache', 'mysql']`
consul: improve consul service discovery (#3814) * consul: improve consul service discovery Related to #3711 - Add the ability to filter by tag and node-meta in an efficient way (`/catalog/services` allow filtering by node-meta, and returns a `map[string]string` or `service`->`tags`). Tags and nore-meta are also used in `/catalog/service` requests. - Do not require a call to the catalog if services are specified by name. This is important because on large cluster `/catalog/services` changes all the time. - Add `allow_stale` configuration option to do stale reads. Non-stale reads can be costly, even more when you are doing them to a remote datacenter with 10k+ targets over WAN (which is common for federation). - Add `refresh_interval` to minimize the strain on the catalog and on the service endpoint. This is needed because of that kind of behavior from consul: https://github.com/hashicorp/consul/issues/3712 and because a catalog on a large cluster would basically change all the time. No need to discover targets in 1sec if we scrape them every minute. - Added plenty of unit tests. Benchmarks ---------- ```yaml scrape_configs: - job_name: prometheus scrape_interval: 60s static_configs: - targets: ["127.0.0.1:9090"] - job_name: "observability-by-tag" scrape_interval: "60s" metrics_path: "/metrics" consul_sd_configs: - server: consul.service.par.consul.prod.crto.in:8500 tag: marathon-user-observability # Used in After refresh_interval: 30s # Used in After+delay relabel_configs: - source_labels: [__meta_consul_tags] regex: ^(.,)?marathon-user-observability(,.)?$ action: keep - job_name: "observability-by-name" scrape_interval: "60s" metrics_path: "/metrics" consul_sd_configs: - server: consul.service.par.consul.prod.crto.in:8500 services: - observability-cerebro - observability-portal-web - job_name: "fake-fake-fake" scrape_interval: "15s" metrics_path: "/metrics" consul_sd_configs: - server: consul.service.par.consul.prod.crto.in:8500 services: - fake-fake-fake ``` Note: tested with ~1200 services, ~5000 nodes. \| Resource \| Empty \| Before \| After \| After + delay \| \| -------- \|:-----:\|:------:\|:-----:\|:-------------:\| \|/service-discovery size\|5K\|85MiB\|27k\|27k\|27k\| \|`go_memstats_heap_objects`\|100k\|1M\|120k\|110k\| \|`go_memstats_heap_alloc_bytes`\|24MB\|150MB\|28MB\|27MB\| \|`rate(go_memstats_alloc_bytes_total[5m])`\|0.2MB/s\|28MB/s\|2MB/s\|0.3MB/s\| \|`rate(process_cpu_seconds_total[5m])`\|0.1%\|15%\|2%\|0.01%\| \|`process_open_fds`\|16\|1236\|22\|22\| \|`rate(prometheus_sd_consul_rpc_duration_seconds_count{call="services"}[5m])`\|~0\|1\|1\|0.03\| \|`rate(prometheus_sd_consul_rpc_duration_seconds_count{call="service"}[5m])`\|0.1\|80\|0.5\|0.5\| \|`prometheus_target_sync_length_seconds{quantile="0.9",scrape_job="observability-by-tag"}`\|N/A\|200ms\|0.2ms\|0.2ms\| \|Network bandwidth\|~10kbps\|~2.8Mbps\|~1.6Mbps\|~10kbps\| Filtering by tag using relabel_configs uses 100kiB and 23kiB/s per service per job and quite a lot of CPU. Also sends and additional 1Mbps of traffic to consul. Being a little bit smarter about this reduces the overhead quite a lot. Limiting the number of `/catalog/services` queries per second almost removes the overhead of service discovery. * consul: tweak `refresh_interval` behavior `refresh_interval` now does what is advertised in the documentation, there won't be more that one update per `refresh_interval`. It now defaults to 30s (which was also the current waitTime in the consul query). This also make sure we don't wait another 30s if we already waited 29s in the blocking call by substracting the number of elapsed seconds. Hopefully this will do what people expect it does and will be safer for existing consul infrastructures. 7 years ago			`tag: "canary"`
			`node_meta:`
			`rack: "123"`
			`allow_stale: true`
Adding consul capability to connect via tls 8 years ago			`scheme: https`
			`tls_config:`
			`ca_file: valid_ca_file`
			`cert_file: valid_cert_file`
			`key_file: valid_key_file`
			`insecure_skip_verify: false`
Configuration options for bearer tokens, client certs & CA certs Fixes #918, fixes #917 9 years ago
Add interpolation to good test config. Fix regex 8 years ago			`relabel_configs:`
			`- source_labels: [__meta_sd_consul_tags]`
formating and text fixes, adjust regexp 8 years ago			`separator: ','`
Add interpolation to good test config. Fix regex 8 years ago			`regex: label:([^=]+)=([^,]+)`
			`target_label: ${1}`
			`replacement: ${2}`

Configuration options for bearer tokens, client certs & CA certs Fixes #918, fixes #917 9 years ago			`- job_name: service-z`

Move TLS options to scrape config Fixes #1013, fixes #989 9 years ago			`tls_config:`
			`cert_file: valid_cert_file`
			`key_file: valid_key_file`
Configuration options for bearer tokens, client certs & CA certs Fixes #918, fixes #917 9 years ago
Replace regex with Secret type and remarshal config to hide secrets (#2775) 8 years ago			`bearer_token: mysecret`
Initial Kubernetes discovery Fixes #904 10 years ago
			`- job_name: service-kubernetes`

			`kubernetes_sd_configs:`
config: rename role 'endpoint' to 'endpoints' 8 years ago			`- role: endpoints`
config: adapt unit tests 8 years ago			`api_server: 'https://localhost:1234'`
Kubernetes SD authentication options cleanup 9 years ago
			`basic_auth:`
			`username: 'myusername'`
Replace regex with Secret type and remarshal config to hide secrets (#2775) 8 years ago			`password: 'mysecret'`
Fix missing unmarshal for Marathon SD config. 9 years ago
Allow limiting Kubernetes service discover to certain namespaces Allow namespace discovery to be more easily extended in the future by using a struct rather than just a list. Rename fields for kubernetes namespace discovery 8 years ago			`- job_name: service-kubernetes-namespaces`

			`kubernetes_sd_configs:`
			`- role: endpoints`
			`api_server: 'https://localhost:1234'`
			`namespaces:`
			`names:`
			`- default`

Fix missing unmarshal for Marathon SD config. 9 years ago			`- job_name: service-marathon`
			`marathon_sd_configs:`
			`- servers:`
Add tls support 8 years ago			`- 'https://marathon.example.com:443'`

			`tls_config:`
			`cert_file: valid_cert_file`
			`key_file: valid_key_file`
Add SD for Amazon EC2 instances 9 years ago
			`- job_name: service-ec2`
			`ec2_sd_configs:`
			`- region: us-east-1`
			`access_key: access`
Replace regex with Secret type and remarshal config to hide secrets (#2775) 8 years ago			`secret_key: mysecret`
ec2_sd_configs: Support profiles for configuring the ec2 service 8 years ago			`profile: profile`
Support AirBnB's Smartstack Nerve client for SD nerve's registration format differs from serverset. With this commit there is now a dedicated treecache file in util, and two separate files for serverset and nerve. Reference: https://github.com/airbnb/nerve 9 years ago
Add support for Azure discovery This change adds the ability to do target discovery with Microsoft's Azure platform. 9 years ago			`- job_name: service-azure`
			`azure_sd_configs:`
			`- subscription_id: 11AAAA11-A11A-111A-A111-1111A1111A11`
			`tenant_id: BBBB222B-B2B2-2B22-B222-2BB2222BB2B2`
			`client_id: 333333CC-3C33-3333-CCC3-33C3CCCCC33C`
Replace regex with Secret type and remarshal config to hide secrets (#2775) 8 years ago			`client_secret: mysecret`
Add support for Azure discovery This change adds the ability to do target discovery with Microsoft's Azure platform. 9 years ago			`port: 9100`

Support AirBnB's Smartstack Nerve client for SD nerve's registration format differs from serverset. With this commit there is now a dedicated treecache file in util, and two separate files for serverset and nerve. Reference: https://github.com/airbnb/nerve 9 years ago			`- job_name: service-nerve`
			`nerve_sd_configs:`
			`- servers:`
			`- localhost`
			`paths:`
			`- /monitoring`
Allow number to be the first letter as well for `job_name` 8 years ago
			`- job_name: 0123service-xxx`
			`metrics_path: /metrics`
			`static_configs:`
			`- targets:`
			`- localhost:9090`

			`- job_name: 測試`
			`metrics_path: /metrics`
			`static_configs:`
			`- targets:`
			`- localhost:9090`
config: add Alertmanager configuration 8 years ago
Add Joyent Triton discovery 8 years ago			`- job_name: service-triton`
			`triton_sd_configs:`
			`- account: 'testAccount'`
			`dns_suffix: 'triton.example.com'`
			`endpoint: 'triton.example.com'`
			`port: 9163`
			`refresh_interval: 1m`
			`version: 1`
			`tls_config:`
			`cert_file: testdata/valid_cert_file`
			`key_file: testdata/valid_key_file`

config: add Alertmanager configuration 8 years ago			`alerting:`
			`alertmanagers:`
			`- scheme: https`
			`static_configs:`
			`- targets:`
			`- "1.2.3.4:9093"`
			`- "1.2.3.5:9093"`
			`- "1.2.3.6:9093"`