Commit Graph

232 Commits (cef8aca8e8989ced6c1a493a3f3dd5e485206f92)

Author SHA1 Message Date
Łukasz Mierzwa 870627fbed Add enable_compression scrape config option
Currently Prometheus will always request gzip compression from the target when sending scrape requests.
HTTP compression does reduce the amount of bytes sent over the wire and so is often desirable.
The downside of compression is that it requires extra resources - cpu & memory.

This also affects the resource usage on the target since it has to compress the response
before sending it to Prometheus.

This change adds a new option to the scrape job configuration block: enable_compression.
The default is true so it remains the same as current Prometheus behaviour.

Setting this option to false allows users to disable compression between Prometheus
and the scraped target, which will require more bandwidth but it lowers the resource
usage of both Prometheus and the target.

Fixes #12319.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2023-11-20 12:02:55 +00:00
Bartlomiej Plotka 624b973ebf
Added ability to specify scrape protocols to accept during HTTP content type negotiation. (#12738)
* Added ability to specify scrape protocols to accept during HTTP content type negotiation.


This is done via new option in GlobalConfig and ScrapeConfig: "scrape_protocol"

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Fixed readability and log message.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
2023-10-10 11:16:55 +01:00
Julien Pivotto d659e672f1
Merge pull request #12816 from slashpai/lightsail
LIGHTSAIL SD: Validate HTTP config
2023-09-08 15:01:35 +02:00
Julien Pivotto 738e0dbaf0
Merge pull request #12814 from slashpai/puppetdb
PUPPETDB SD: Validate HTTP config
2023-09-08 15:00:21 +02:00
Julien Pivotto a2208b2849
Merge pull request #12815 from slashpai/Uyuni
UYUNI SD: Validate HTTP config
2023-09-08 14:59:50 +02:00
Jayapriya Pai 57988fda75
LIGHTSAIL SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:54:58 +05:30
Jayapriya Pai 344f9d8d15
PUPPETDB SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:42:01 +05:30
Jayapriya Pai 34c8dcc73f
UYUNI SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 17:34:41 +05:30
Julien Pivotto 75470ba815
Merge pull request #12812 from slashpai/azure
AZURE SD: Validate HTTP config
2023-09-08 14:03:19 +02:00
Jayapriya Pai e44d5dfb3a
AZURE SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 13:10:31 +05:30
Jayapriya Pai 7f085aeb6a
EC2 SD: Validate HTTP config
Related-to #12810

Signed-off-by: Jayapriya Pai <janantha@redhat.com>
2023-09-08 11:46:07 +05:30
Julien Pivotto 8498b96a2b HTTP SD: Validate HTTP config
Fixes #12762

Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-09-03 23:06:30 +02:00
rakshith210 b1675e23af
Add Azure AD package for remote write (#11944)
* Add Azure AD package for remote write
* Made AzurePublic default and updated configuration.md
* Updated config structure and removed getToken at initialization
* Changed passing context from request

Signed-off-by: Rakshith Padmanabha <rapadman@microsoft.com>
Signed-off-by: rakshith210 <rakshith.me@gmail.com>
2023-06-01 15:20:10 -06:00
Arianna Vespri a16b876a05
Add limits to global config (#12126)
* Add limits to global config

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Move changes into Validate func

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Make comments consistent wrt 0 meaning no limit

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

* Document global limits

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>

---------

Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>
2023-05-30 09:22:23 +01:00
Julien Pivotto 0dc31ade41 Add support for consul path_prefix
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-05-17 00:14:58 +02:00
Julien Pivotto 599b70a05d Add include scrape configs
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2023-03-06 23:35:39 +01:00
Julien Levesy 8c5da5e078
feat(config): allow no remote write configs
Signed-off-by: Julien Levesy <jlevesy@gmail.com>
2022-12-13 18:41:06 +01:00
Levi Harrison f81fae2414
Add common HTTP client to AWS SDs (#11611)
* Common client in EC2 and Lightsail

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Azure -> AWS

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2022-11-22 10:09:14 -05:00
Julien Pivotto 005ede70de relabel: add keepequal/dropequal relabel action
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-11-10 14:41:02 +01:00
Marine Bal 16c3aa75c0
Add service discovery for OvhCloud (#10802)
* feat(ovhcloud): add ovhcloud management

Signed-off-by: Marine Bal <marine.bal@corp.ovh.com>
Co-authored-by: Arnaud Sinays <sinaysarnaud@gmail.com>
2022-11-03 10:20:09 +01:00
Ganesh Vernekar f371d7f0fb
Add docs for out of order ingestion (#11340)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Co-authored-by: Levi Harrison <levisamuelharrison@gmail.com>
2022-09-28 20:01:21 +05:30
Bryan Boreham 31bb281185 config: in tests use labels.FromStrings
Replacing code which assumes the internal structure of `Labels`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-09-09 13:34:49 +02:00
Bram Vogelaar 4456dcc26e feat(nomad): add nomad service discovery
Signed-off-by: Bram Vogelaar <bram@attachmentgenie.com>
2022-06-30 14:17:08 +02:00
Julien Pivotto e4a09f2b4b
uyuni: Use default HTTP client and set relative paths (#10814)
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-06-08 11:29:44 +02:00
David Dymko 3ef153b00c vultr integration
Signed-off-by: David Dymko <dymkod@gmail.com>
2022-05-24 18:38:06 +02:00
Felix Ehrenpfort ce3bc818a8
Add service discovery for IONOS Cloud (#10514)
* Add service discovery for IONOS Cloud

Signed-off-by: Felix Ehrenpfort <felix@ehrenpfort.de>
2022-05-20 14:57:23 +02:00
Julien Pivotto 71dbb4d091
Add lowercase and uppercase relabel action (#10641)
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-05-03 10:09:53 +02:00
Matthieu MOREL e2ede285a2
refactor: move from io/ioutil to io and os packages (#10528)
* refactor: move from io/ioutil to io and os packages
* use fs.DirEntry instead of os.FileInfo after os.ReadDir

Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>
2022-04-27 11:24:36 +02:00
Julien Pivotto 09da88114d Support overriding minimum TLS version
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-04-20 23:41:30 +02:00
Julien Pivotto 98039cddfa
Update Prometheus common (#10492)
* Update Prometheus common

- Oauth2 supports proxy URL
- HTTP2 can be disabled

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-03-30 12:49:03 +02:00
David N Perkins b13aec9167
Merge pull request #10365 from David-N-Perkins/azure-resource-group-filter
Azure Service Discovery resource group filter
2022-03-29 01:21:23 +02:00
Julien Pivotto fb2da1f26a
Followup on tracing (#10338)
* Simplify code by letting common deal with empty TLS config
* Improve error message if we notice a user is putting an authorization
header into its configuration.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-22 21:44:36 +01:00
Matej Gera 0acbe5e3f5
Tracing: Add additional options to align with the upstream exporter (#10276)
* Enhance configuration

Signed-off-by: Matej Gera <matejgera@gmail.com>
2022-02-22 17:07:30 +01:00
DrAuYueng 5a6e26556b
Add an option to use the external labels as selectors for the remote read endpoint (#10254)
* An option to ignore external_labels

Signed-off-by: DrAuYueng <ouyang1204@gmail.com>
2022-02-16 22:12:47 +01:00
Julien Pivotto 9a2e93228e
Switch to grafana/regexp everywhere (#10268)
Let's have a consistent library for regexp.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-13 00:58:27 +01:00
beorn7 35010daa90 Merge branch 'release-2.33' into beorn7/cleaning-up-cherrypicking-fallout 2022-02-02 16:49:40 +01:00
Julien Pivotto 9d63502204 k8s: improve 'own_namespace'
Fail configuration unmarshalling if kubeconfig or api url are set with
"own namespace"

Only read namespace file if needed.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-01 14:59:09 +01:00
Julien Pivotto 8cb733d04c
Followup on OpenTelemetry migration (#10203)
* Followup on OpenTelemetry migration

- tracing_config: Change with_insecure to insecure, default to false.
- tracing_config: Call SetDirectory to make TLS certificates relative to the Prometheus
  configuration
- documentation: Change bool to boolean in the configuration
- documentation: document type float
- tracing: Always restart the tracing manager when TLS config is set to
  reload certificates
- tracing: Always set TLS config, which could be used e.g. in case of
  potential redirects.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>\\
2022-01-29 23:56:44 +01:00
Matej Gera 2c61d29b2a
Tracing: Migrate to OpenTelemetry library (#9724)
Signed-off-by: Matej Gera <matejgera@gmail.com>
2022-01-25 11:08:04 +01:00
Filip Petkovski 4855a0c067
Allow escaping a dollar sign when expanding external labels (#10129)
* Allow escaping a dollar sign when expanding external labels

There is currently no mechanism to natively escape a dollar sign
in the os.Expand function. As a workaround, this commit modifies
the external label expansion logic to treat a double dollar ($$)
as a mechanism for escaping the dollar character.

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
2022-01-17 16:43:55 +01:00
Witek Bedyk 14986e52cf
Fix Uyuni SD initialization (#9924)
* Fix Uyuni SD initialization

The change prevents null pointer exception during SD initialization.

Signed-off-by: Witek Bedyk <witold.bedyk@suse.com>
2021-12-04 21:14:47 +01:00
beorn7 c954cd9d1d Move packages out of deprecated pkg directory
This creates a new `model` directory and moves all data-model related
packages over there:
  exemplar labels relabel rulefmt textparse timestamp value

All the others are more or less utilities and have been moved to `util`:
  gate logging modetimevfs pool runtime

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
Mateusz Gozdek 1a6c2283a3 Format Go source files using 'gofumpt -w -s -extra'
Part of #9557

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
Arthur Silva Sens be2599c853
config: Make remote-write required for Agent mode (#9618)
* config: Make remote-write required for Agent mode

Signed-off-by: ArthurSens <arthursens2005@gmail.com>
2021-10-30 01:41:40 +02:00
Julien Pivotto 77f411b2ec
Enable tls_config in oauth2 (#9550)
* Enable tls_config in oauth2

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-20 23:10:18 +02:00
Levi Harrison 89a6ebd799
Add common HTTP client to Azure SD (#9267)
* Add `proxy_url` option to Azure SD

Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-10-20 23:09:58 +02:00
Witek Bedyk cda2dbbef6
Add Uyuni service discovery (#8190)
* Add Uyuni service discovery

Signed-off-by: Witek Bedyk <witold.bedyk@suse.com>

Co-authored-by: Joao Cavalheiro <jcavalheiro@suse.de>
Co-authored-by: Marcelo Chiaradia <mchiaradia@suse.com>
Co-authored-by: Stefano Torresi <stefano@torresi.io>
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
2021-10-19 01:00:44 +02:00
Julien Pivotto 9d65017798 config: fix puppetdb tests
This PR fixes the tests in main. The last merge introduced a failing
test in the config package.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-09-16 18:31:28 +02:00
Julien Pivotto 8920024323 Add PuppetDB service discovery
We have been Puppet user for 10 years and we are users of
https://github.com/camptocamp/prometheus-puppetdb-sd

However, that file_sd implementation contains business logic and
assumptions around e.g. the modules which you are using.

This pull request adds a simple PuppetDB service discovery, which will
enable more use cases than the upstream sd.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-09-16 16:54:26 +02:00
DrAuYueng e8be1d0a5c
Check relabel action at yaml unmarshal stage (#9224)
Signed-off-by: DrAuYueng <ouyang1204@gmail.com>
2021-08-31 17:52:57 +02:00