Commit Graph

333 Commits (c3cc25f55243bbe4e6b84c4e1617dd473695251b)

Author SHA1 Message Date
Mateusz Gozdek 1a6c2283a3 Format Go source files using 'gofumpt -w -s -extra'
Part of #9557

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
Robert Fratto bc72a718c4
Initial draft of prometheus-agent (#8785)
* Initial draft of prometheus-agent

This commit introduces a new binary, prometheus-agent, based on the
Grafana Agent code. It runs a WAL-only version of prometheus without the
TSDB, alerting, or rule evaluations. It is intended to be used to
remote_write to Prometheus or another remote_write receiver.

By default, prometheus-agent will listen on port 9095 to not collide
with the prometheus default of 9090.

Truncation of the WAL cooperates on a best-effort case with Remote
Write. Every time the WAL is truncated, the minimum timestamp of data to
truncate is determined by the lowest sent timestamp of all samples
across all remote_write endpoints. This gives loose guarantees that data
from the WAL will not try to be removed until the maximum sample
lifetime passes or remote_write starts functionining.

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* add tests for Prometheus agent (#22)

* add tests for Prometheus agent

* add tests for Prometheus agent

* rearranged tests as per the review comments

* update tests for Agent

* changes as per code review comments

Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>

* incremental changes to prometheus agent

Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>

* changes as per code review comments

Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>

* Commit feedback from code review

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* Port over some comments from grafana/agent

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* Rename agent.Storage to agent.DB for tsdb consistency

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* Consolidate agentMode ifs in cmd/prometheus/main.go

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* Document PreAction usage requirements better for agent mode flags

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* remove unnecessary defaultListenAddr

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

* `go fmt ./tsdb/agent` and fix lint errors

Signed-off-by: Robert Fratto <robertfratto@gmail.com>

Co-authored-by: SriKrishna Paparaju <paparaju@gmail.com>
2021-10-29 16:25:05 +01:00
Pablo Caderno b878527151
Added deferred close to 'http.File' opened (#9443) (#9461)
Signed-off-by: Pablo Caderno <kaderno@gmail.com>
2021-10-06 13:02:46 +02:00
George Robinson 049b4f4f13
Support customization of template options in TemplateExpander (#9290)
Signed-off-by: George Robinson <george.robinson@grafana.com>
2021-09-13 17:19:08 +05:30
Levi Harrison 4a4882d4c7 Replace godoc.org links
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-17 07:18:51 -04:00
Levi Harrison b5f6f8fb36 Switched to go-kit/log
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-11 12:28:36 -04:00
Levi Harrison 7bc11dcb06
React UI: Add Starting Screen (#8662)
* Added walreplay API endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added starting page to react-ui

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Documented the new endpoint

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed typos

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>

* Removed logo

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed isResponding to isUnexpected

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed width of progress bar

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added DB stats object

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Updated starting page to work with new fields

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 2)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 3)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 4)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (and also implementing a method this time) (pt. 5)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed const to let

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Passing nil (pt. 6)

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Remove SetStats method

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added comma

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed api

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed to triple equals

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed data response types

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Don't return pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Changed version

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed interface issue

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed pointer

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Fixed copying lock value error

Signed-off-by: Levi Harrison <git@leviharrison.dev>

Co-authored-by: Julius Volz <julius.volz@gmail.com>
2021-06-05 15:29:32 +01:00
Julien Pivotto db7f0bcec2
Merge pull request from GHSA-vx57-7f4q-fpc7
* Do not remove /new because it is not part of the route parameter (CVE-2021-29622)

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Release 2.27.1

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-05-18 06:47:45 -06:00
Callum Styan 289ba11b79
Add circular in-memory exemplars storage (#6635)
* Add circular in-memory exemplars storage

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>

* Fix some comments, clean up exemplar metrics struct and exemplar tests.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix exemplar query api null vs empty array issue.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com>
Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-03-16 15:17:45 +05:30
Tom Wilkie ce97cdd477
Move remote read handler to remote package. (#8536)
* Move remote read handler to remote package.

This follows the pattern I started with the remote write handler.  The api/v1 package is getting pretty cluttered.  Moving code to other packages helps reduce this size and also makes it reusable - eg Cortex doesn't do streaming remote writes yet, and will very soon.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Deal with a nil remoteReadHandler for tests.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Remove the global metrics.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Fix test.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Review feedback.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-26 16:43:19 +00:00
Peter Štibraný a2e5ad6174 Split queryable and appendable arguments in api_v1.NewAPI.
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
2021-02-15 12:30:00 +01:00
Tom Wilkie d479151f1f Various enhancements and refactorings for remote write receiver:
- Remove unrelated changes
- Refactor code out of the API module - that is already getting pretty crowded.
- Don't track reference for AddFast in remote write.  This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series.  For now, its easier and safer to always use the 'slow' path.
- Return 400 on out of order samples.
- Use remote.DecodeWriteRequest in the remote write adapters.
- Put this behing the 'remote-write-server' feature flag
- Add some (very) basic docs.
- Used named return & add test for commit error propagation

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-08 20:41:23 +00:00
Julien Pivotto ac2626757c Update exporter-toolkit to 0.5.0
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-01-13 21:49:54 +01:00
Julien Pivotto 5b4f46a348 Add TLS and basic authentication
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-28 21:33:44 +01:00
Ben Kochie 5055dfbbe4 Listen on web early in startup
Avoid starting up components like the TSDB if we can't bind
to the web listening port.

Signed-off-by: Ben Kochie <superq@gmail.com>
2020-12-28 20:13:05 +01:00
Mitsuo Heijo 8b64b70fe4
Guard closing quitCh with sync.Once to prevent double close (#8242)
* Guard closing quitCh with sync.Once to prevent double close

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2020-12-02 09:39:54 +01:00
Julien Pivotto cda52234eb
Fix panic with double close() of channel on /-/quit/ (#8166)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-11-10 00:09:39 +01:00
Julius Volz 3470ee1fbf
Make React UI the default, keep old UI under /classic (#8142)
The React app's assets are now served under /assets, while all old
custom web assets (including the ones for console templates) are now
served from /classic/static.

I tested different combinations of --web.external-url and
--web.route-prefix with proxies in front, and I couldn't find a problem
yet with the routing. Console templates also still work.

While migrating old endpoints to /classic, I noticed that /version was
being treated like a lot of the old UI pages, with readiness check
handler in front of it, etc. I kept it in /version and removed that
readiness wrapper, since it doesn't seem to be needed for that endpoint.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2020-11-03 14:51:48 +01:00
James Ranson 1cffda5de7
react updates for pathPrefix (#7979)
* dynamically determine path prefix

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* minor changes per PR review

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* use Context for apiPath and pathPrefix

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* remove unhandled "/version" path

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* only process index once instead of on every req

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* remove unneeded tag fragment

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* switch api path to const

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* revert

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* update tests

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* linter updates

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* simplify

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>

* updates per peer review

Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
2020-10-22 17:22:32 +02:00
Dustin Hooten 916dbd4c8a
React UI: Implement missing TSDB head stats section (#7876)
* React UI: Implement missing TSDB head stats section

Signed-off-by: Dustin Hooten <dhooten@splunk.com>

* Add break

Signed-off-by: Dustin Hooten <dhooten@splunk.com>
2020-09-29 22:05:33 +02:00
Julien Pivotto e208afcc95
web: Remove APIv2 (#7935)
* web: Remove APIv2

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-09-15 09:30:55 +02:00
Julien Pivotto e96d786fb8
Stop CMUX and GRPC servers even with stale connections (#7810)
I have received a recent report for a user which confirms that sometime
the GRPC server does not stop propery.

It appears that there are 2 issues:

1. The cmux server can refuse to stop if there are stale connections.
For that we set the ReadTimeout.
2. The GRPC server graceful stop can never finish.

What this PR avoids is:

```
goroutine 227 [semacquire, 2 minutes]:
sync.runtime_Semacquire(0xc00059a75c)
    /usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc00059a75c)
    /usr/local/go/src/sync/waitgroup.go:130 +0x64
google.golang.org/grpc.(*Server).GracefulStop(0xc00059a600)
```

This PR stops the GRPC server after 15s. Related to the go routing dumps
in #6747.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-17 09:50:32 +01:00
Javier Palomo Almena b58a613443
Replace sync/atomic with uber-go/atomic (#7683)
* storage: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* tsdb: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* cmd: Replace usage of sync/atomic with uber-go/atomic

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* scripts: Verify that we are not using restricted packages

It checks that we are not directly importing 'sync/atomic'.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Reorganise imports in blocks

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* notifier/test: Apply PR suggestions

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* storage/remote: avoid storing references on newEntry

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* Revert "scripts: Verify that we are not using restricted packages"

This reverts commit 278d32748e.

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>

* web: Group imports accordingly

Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
2020-07-30 13:15:42 +05:30
Julien Pivotto a3ac9fa5f5
Add custom title to React UI (#7607)
* Backport custom title to React UI

Backports #4841 into the new UI.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Fix typo

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-21 10:48:29 +02:00
Sylvain Rabot 6a00626ee9
Display dates as well as timestamps in status page (#7544)
* Display dates as well as timestamps in the status page

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>

* Trim trailing whitespaces

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2020-07-10 00:08:43 +02:00
gotjosh bfcd9282a8
API: Allow AlertmanagerRetriever and RulesRetriever to receive a Context (#7256)
* API: Allow AlertmanagerRetriever and RulesRetriever to receive a
Context

Signed-off-by: gotjosh <josue@grafana.com>

* Use single line functions where possible

Signed-off-by: gotjosh <josue@grafana.com>
2020-05-18 19:02:32 +01:00
Ben Ye 1e4e37144d
Fixed wrongly handled not ready TSDB on web and API. (#7182)
* fix federate endpoint panic

Signed-off-by: yeya24 <yb532204897@gmail.com>

* Fixed all cases of not ready TSDB being wrongly handled.

* Fixed issue for federation.
* Ensured this will never happen again thanks to interfaces
* Fixes same issue for stats.
* Added tests for readiness.
* Fixed bug in stats. It was:
   status.MaxTime = db.Head().MaxTime()
   status.MinTime = db.Head().MaxTime()


Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Brian's comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Addressed Brian's comments.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-04-29 17:16:14 +01:00
gotjosh 24af5049bb
API: Allow `TargetRetriever` to receive a Context (#7125)
Fixes #7103

Signed-off-by: gotjosh <josue@grafana.com>
2020-04-16 09:30:47 +01:00
Marek Slabicki 8224ddec23
Capitalizing first letter of all log lines (#7043)
Signed-off-by: Marek Slabicki <thaniri@gmail.com>
2020-04-11 09:22:18 +01:00
Ben Ye ecda6013ed
Use only local tsdb for federation (#7096)
Signed-off-by: yeya24 <yb532204897@gmail.com>
2020-04-07 16:42:42 +01:00
Julian Taylor 05442b31c8
register federation failure metrics (#7081)
Closes gh-7080

Signed-off-by: Julian Taylor <juliantaylor108@gmail.com>
2020-04-06 09:05:01 +01:00
Brian Brazil 7646cbca32
Use .UTC everywhere we use time.Unix (#7066)
time.Unix attaches the local timezone, which can then
leak out (e.g. in the alert json). While this is harmless,
we should be consistent.

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-03-29 17:35:39 +01:00
Peter Štibraný 318cd413fc Don't return error in ContextFromRequest function.
Previously it could return error if RemoteAddr didn't
have correct format, but since this field has no specified
format, that was little too strict.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
2020-02-18 15:58:14 +01:00
Bartlomiej Plotka 48ead578a0 Moved tsdbconfig to main.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-18 11:25:36 +00:00
Bartlomiej Plotka 8a775bc468 Moved unit agnostic options to separate pkg.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 59c9d6ef45 Addressed Brian's comments, moved metrics to main.go
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka cfba92a133 Addressed comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:57 +00:00
Bartlomiej Plotka 34426766d8 Unify Iterator interfaces. All point to storage now.
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things.
All todos I added will be fixed in follow up PRs.

* querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged
with storage interface.go. All imports that.
* querier.SeriesIterator replaced by chunkenc.Iterator
* Added chunkenc.Iterator.Seek method and tests for xor implementation (?)
* Since we properly handle SelectParams for Select methods I adjusted min max
based on that. This should help in terms of performance for queries with functions like offset.
* added Seek to deletedIterator and test.
* storage/tsdb was removed as it was only a unnecessary glue with incompatible structs.

No logic was changed, only different source of abstractions, so no need for benchmarks.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-02-17 18:03:54 +00:00
Harkishen Singh 489a9aa7b9
Adds normalization of localhost urls in targets page react (#6794)
* support for globalurls in targets page react

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* fixed tests

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* removed fmts

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* implemented suggestions

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* formatted

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* implemented suggestions. fixed tests.

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* formated go code

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>

* implemented suggestions

Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>
2020-02-17 18:19:15 +01:00
Julien Pivotto ff0003e072
Make lookbackDelta a option of QueryEngine (#6746)
* Make lookbackDelta a option of QueryEngine

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* julius' suggestion

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* remove trivial getter

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Assume lookback delta is always > 0

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* add debug log

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* don't expose loopback delta

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Specify that lookack delta is also used in federation

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Fix federation test

While we have added some logic to the promql engine to keep it backwards
compatible and have a 5 minute loopback by default, the web/ package is
likely to really be internal to Prometheus and we should not add the
same kind of heuritstics here.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* loopback delta: Fix debug log

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-02-10 00:58:23 +01:00
Drumil Patel 687a962bd1 Add conditional rendering of Navlink for Consoles (#6761)
* Add conditional rendering of Navlink for Consoles

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>

* Replacing if else with only if conditional rendering

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>

* Add tests and removing global declaration in Navbar

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>

* Correct Navbar Testcases and add types for ConsolesLink

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>

* Change names for Console link as per-naming convention

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>

* Change prop names to AppProps and NavbarProps respectively

Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
2020-02-08 11:00:47 +01:00
Julien Pivotto 29086533fd web: use chained WithInstrumentation (#6661)
* web: use chained WithInstrumentation

Requires an update of github.com/prometheus/common
* Update yaml.v2

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-19 00:07:10 +00:00
Julien Pivotto e7f7b6a06f Query Log: Add source IP from console queries (#6593)
* Query Log: Add source IP from console queries

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-01-10 12:56:36 +00:00
Julius Volz d2cb05846b
React UI: Add /tsdb-status to React-handled routes (#6313)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-11-20 15:50:52 +01:00
Julius Volz 8ee3b8c848
React UI: Fix "<path-prefix>/new" -> "<path-prefix>/new/" redirect (#6319)
Fixes https://github.com/prometheus/prometheus/issues/6163

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-11-14 14:58:22 +01:00
Sharad Gaur e94503ff5c Head Cardinality Status Page (#6125)
* Adding TSDB Head Stats like cardinality to Status Page

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Moving mutx to Head

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Renaming variabls

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Renaming variabls and html

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Removing unwanted whitespaces

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Adding Tests, Banchmarks and Max Heap for Postings Stats

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Adding more tests for postingstats and web handler

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Adding more tests for postingstats and web handler

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Remove generated asset file that is no longer used

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>

* Changing comment and variable name for more readability

Signed-off-by: Sharad Gaur <sgaur@splunk.com>

* Using time.Duration in postings status function and removing refresh button from web page

Signed-off-by: Sharad Gaur <sgaur@splunk.com>
2019-11-04 19:06:13 -07:00
Julius Volz 95554074d8
React UI: Support custom path prefixes (#6264)
* React UI: Support custom path prefixes

The challenge was that the path prefix can be set dynamically as a flag
on Prometheus, but the React app bundle is statically compiled in to
expect a given path prefix. By adding a placeholder value to the React
app's index.html and replacing it in Prometheus with the right path
prefix during serving, this injects Prometheus's path prefix into the
React app via a global const.

Threading the path prefix into the different React components could have
been done with React's Contexts (https://reactjs.org/docs/context.html),
but I found the consumer side of context values to be a bit cumbersome
(wrapping entire components in context consumers), so I ended up
preferring direct threading of the path prefix values to components that
needed them. Also, using contexts in tests is more verbose than just
passing in path prefix values directly.

Fixes https://github.com/prometheus/prometheus/issues/6163

Signed-off-by: Julius Volz <julius.volz@gmail.com>

* Review feedback

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-11-04 09:17:50 +01:00
Boyko cb7cbad5f9 WIP: status page - API and UI (#6243)
* status page initial commit

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* refactor useFetch

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* refactoring

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* adding tests

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* snapshot testing

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* fix wrong go files formatting

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* change the snapshot library

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* update api paths

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* move test folder outside src

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* useFetches tests

Signed-off-by: blalov <boyko.lalov@tick42.com>

* sticky navbar

Signed-off-by: Boyko Lalov <boyskila@gmail.com>
Signed-off-by: blalov <boyko.lalov@tick42.com>

* handle runtimeInfo error on Gather() and add json tags to RuntimeInfo struct

Signed-off-by: blalov <boyko.lalov@tick42.com>

* refactor alert managers section

Signed-off-by: blalov <boyko.lalov@tick42.com>
2019-11-02 16:53:32 +01:00
Michał 040425b0eb filtering alerts by state and display count of alerts in each state (#5758)
* local storage selectedTab on targets tab was renamed

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>

* added filters when displaying alerts

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>

* function was simplified

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>

* fixed rebase

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>

* minor rename

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>

* Active -> Pending

Signed-off-by: Michał Szczygieł <1153719+mszczygiel@users.noreply.github.com>
2019-10-29 11:09:52 +01:00
Julius Volz e8027ba515
React UI: Serve React UI under /new (#6229)
This makes React UI URLs look nicer than the previous
/static/graph-new/app.html, but internally still serves all React UI
files from the compiled-in static assets directory.

Also, to allow future usage of the React / Reach router, we need to
serve the main React app's index.html on certain sub-paths that
correspond to current Prometheus's UI pages, instead of trying to serve
actual files that match the provided path name.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2019-10-28 10:45:53 +01:00
Sven Nebel 27bbc235e0 Improve message when Lifecycle API is not enabled
Signed-off-by: Sven Nebel <nebel.sven@gmail.com>
2019-09-25 10:57:21 +01:00
Sven Nebel d186d85649 Return 403 with PUT verb if lifecycle is not enabled
Signed-off-by: Sven Nebel <nebel.sven@gmail.com>
2019-09-25 10:57:11 +01:00
Bartek Płotka 48b2c9c8ea
remote-read: streamed chunked server side; Extended protobuf; Added chunked, checksumed reader (#5703)
Part of: https://github.com/prometheus/prometheus/issues/4517 and https://github.com/improbable-eng/thanos/issues/488

Changes:
* Extended protobuf for chunked remote read and negotation.
* Added checksumed, chunked Writer/Reader.
* Added Server side implementation for chunked streamed remote-read.


Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-08-19 21:16:10 +01:00
Ganesh Vernekar 5ecef3542d
Cleanup after merging tsdb into prometheus
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Simon Pasquier be67b8d460
web: fix flaky TestHTTPMetrics() (#5695)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-06-24 15:48:15 +02:00
Simon Pasquier d7f38dfdde
web: add 'code' label to HTTP metrics (#5640)
* web: add prometheus_http_requests_total metrics

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Add unit test for requestCounter metric

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-06-19 17:56:04 +02:00
pbhudiaBAE 43953b105b Sorting alerts by group name in /alerts (#5448)
* Working group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Working categorised by group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Changed group sorting in web

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Fixed group sorting and comments

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Fixed group sorting and comments with gofmt

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* Added file and group name

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>

* reverted back to full path to yml file

Signed-off-by: Pritam Bhudia <pritam.bhudia@baesystems.com>
2019-05-14 23:14:27 +02:00
mrasu 3a7a743bc4 Consistent Content-Type for js and css (#5551)
Currently, When `/etc/mime.types` has a unusual mime type, web of prometheus uses the type and you may get unexpected result.
With this change, web returns consistent Content-Type header for static js and css files

To reproduce:
1. Add a type at the end of `/etc/mime` like `text/x-js js`
2. Run prometheus
3. Request js file like `http://localhost:9090/static/vendor/js/jquery.min.js`
4. You will see Content-Type of the response is `text/x-js` instead of `application/javascript`

Signed-off-by: mrasu <m.rasu.hitsuji@gmail.com>
2019-05-09 16:10:15 +01:00
Bjoern Rabenstein 38d518c0fe Rework #5009 after comments
Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>
2019-04-17 01:40:10 +02:00
Sylvain Rabot 335a34486e Add external labels to template expansion
This affects the expansion of templates in alert labels and
annotations and console templates.

Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
2019-04-17 01:40:10 +02:00
Simon Pasquier 81c4248081
*: bump gRPC and protobuf dependencies (#5367)
The goal is to remove almost all references to the
golang.org/x/net/context package.

github.com/gogo/protobuf => v1.2.1
google.golang.org/grpc => v1.19.1
github.com/grpc-ecosystem/grpc-gateway => v1.18.5

It also replaces github.com/cockroachdb/cmux by github.com/soheilhy/cmux
because of [1] which fixes #3909 incidentally.

[1] https://github.com/grpc/grpc-go/issues/2636

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-04-04 11:55:32 +02:00
Ganesh Vernekar 0afd172224
Merge pull request #5416 from prometheus/release-2.8
Merge 2.8.1 into master
2019-03-29 15:20:49 +05:30
Simon Pasquier 7328cb6cc2 web/ui: display the job label in /targets
PR #4806 was removed accidentally by #5192 (Bootstrap upgrade).

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-03-26 16:52:48 +01:00
Tariq Ibrahim 8fdfa8abea refine error handling in prometheus (#5388)
i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors.
ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives.
iii) Does away with the use of fmt package for errors in favour of pkg/errors

Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-26 00:01:12 +01:00
Bharath 91306bdf24 Support non POST methods for Lifecycle and Admin APIs (#5376)
Signed-off-by: Bharath Thiruveedula <bharath_ves@hotmail.com>
2019-03-20 17:33:45 +00:00
Julien Pivotto 6c72cdb1e1 Add GODEBUG to the runtime page (#5324)
This is useful, e.g. with go 1.12 to see if TLS1.3 has been enabled.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-03-18 11:10:13 +00:00
Tariq Ibrahim d6f0e14ac6 fix redundant err!=nil check in web.go (#5346)
Signed-off-by: tariqibrahim <tariq181290@gmail.com>
2019-03-12 16:25:46 +00:00
Krasi Georgiev 9d96ada510 Display correct values for the retention in the flags web gui. (#5322)
* Display correct values for the retention in the flags web gui.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* adding a log entry

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* added the retention info to the runtime status page

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* simplify the retention display

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-03-11 22:48:57 +05:30
Simon Pasquier a60431f3cd Merge v2.7.1 into master (#5170)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-01 09:54:12 +01:00
Hrishikesh Barman a1f34bec2e Added CORS Origin flag (#5011)
Signed-off-by: Hrishikesh Barman <hrishikeshbman@gmail.com>
2019-01-17 15:01:06 +00:00
Simon Pasquier 375ad1185c
*: bump gRPC dependencies (#5075)
* *: bump gRPC dependencies

This change updates the gRPC dependencies to more recent versions:

* github.com/gogo/protobuf => v1.2.0
* github.com/grpc-ecosystem/grpc-gateway => v1.6.3
* google.golang.org/grpc => v1.17.0

In addition scripts/genproto.sh leverages Go modules information instead of
hardcoding SHA1 commits. This ensures that the code is generated from
the exact same sources.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Run 'make proto' in CI

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Revert tabs -> spaces change

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix 'make proto' step

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* 'go get' grpc/protobuf dependencies

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Prepopulate cache with go mod download

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-15 15:32:05 +01:00
Simon Pasquier f678e27eb6
*: use latest release of staticcheck (#5057)
* *: use latest release of staticcheck

It also fixes a couple of things in the code flagged by the additional
checks.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use official release of staticcheck

Also run 'go list' before staticcheck to avoid failures when downloading packages.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 14:47:38 +01:00
Simon Pasquier 77ee41084f
web: group targets by job then instance (#4898)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 15:24:15 +01:00
Simon Pasquier 632626b553
web: deduplicate handler label for HTTP metrics (#4752)
* web: deduplicate handler label for HTTP metrics

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* remove instrumentHandlerFunc

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-30 13:30:32 +01:00
Simon Pasquier 0bb810d126
discovery/marathon: fix leaked connections (#4915)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-27 14:58:27 +01:00
achiuBAE a9050c45f6 Allow setting the Prometheus instance document title through a flag. (#4841)
* web: added ability to set page title through flag.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Reformatted variable names and Flag description for readability.

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* assets_vfsdata.go

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* Flag name changed from web.ui-title to web.page-title

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>

* make assets

Signed-off-by: Andrew Chiu <andrew.chiu2@baesystems.com>
2018-11-21 12:45:06 +08:00
Simon Pasquier 87abb279e4
web: log stack traces on panic (#4221)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-20 14:25:03 +01:00
Simon Pasquier 6fa8de132b
web/v1/api: add tests for admin actions (#4767)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-11-15 14:22:16 +01:00
Krasi Georgiev 47a673c3a0
process scrape loops reloading in parallel (#4526)
The scrape manage receiver's channel now just saves the target sets
and another backgorund runner updates the scrape loops every 5 seconds.
This is so that the scrape manager doesn't block the receiving channel
when it does the long background reloading of the scrape loops.

Active and dropped targets are now saved in each scrape pool instead of
the scrape manager. This is mainly to avoid races when getting the
targets via the web api.

When reloading the scrape loops now happens in parallel to speed up the
final disared state and this also speeds up the prometheus's shutting
down.

Also updated some funcs signatures in the web package for consistency.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2018-09-26 12:20:56 +03:00
Tom Wilkie 4c52400708
Limit concurrent remote reads. (#4656)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-25 20:07:34 +01:00
Simon Pasquier 2d7f562ed6 web: fix asset paths for Windows platforms (#4616)
* web: fix asset paths for Windows platforms

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* web: add tests

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-09-19 12:50:53 +05:30
Tom Wilkie 457e4bb58e
Limit the number of samples remote read can return. (#4532)
* Limit the number of samples remote read can return.

- Return 413 entity too large.
- Limit can be set be a flag.  Allow 0 to mean no limit.
- Include limit in error message.
- Set default limit to 50M (* 16 bytes = 800MB).

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-09-05 15:50:50 +02:00
Simon Pasquier 75bd348135 web: clean up api/v2 (#4554)
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-08-29 12:55:46 +05:30
Simon Pasquier 3581377e5d Replace go-bindata with vfsgen (#4430)
Looking at https://tech.townsourced.com/post/embedding-static-files-in-go/ (which was mentioned in the issue), vfsgen has all the needed features.

In particular:

- Reproducible builds (no issue with timestamping).
- Well maintained and relatively popular.
- Integration with go generate.
- Self-contained (no external dependency).

* [WIP] Replace go-bindata by vfsgen

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Add license + remove doc.go

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Generate templates assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use new templates assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* split static assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Idempotent make assets

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Update vendor/

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* vendor vfsgendev

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Update README.md

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Simplify assets generation

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix README.md

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Use generate helper program instead of vfsgen

This avoids installing vfsgendev in the target environment.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Remove unused vfsgen package

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* vendoring shurcooL/vfsgen

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Fix go generate command

Signed-off-by: Simon Pasquier <spasquie@redhat.com>

* Sync web/ui/assets_vfsdata.go

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-08-24 09:03:10 +02:00
Benji Visser 8bb6e0dd6e Show rule evaluation errors on rules page (#4457)
* adding information about the health and errors for Rules

adding Health() and LastError() to the Rule interface. This will allow
us to easily surface information about rules.

Signed-off-by: noqcks <benny@noqcks.io>

* updating rules.html with fields for Rule errors and health state

Signed-off-by: noqcks <benny@noqcks.io>

* fix code comment grammar & access Rule health/error info using a mutex

Signed-off-by: noqcks <benny@noqcks.io>

* s/Errors/Error/ in rules.html to remain consistent with targets.html

Signed-off-by: noqcks <benny@noqcks.io>

* adding periods to code comments in reporting/alerting

Signed-off-by: noqcks <benny@noqcks.io>

* putting health/error below mutex in struct field

Signed-off-by: noqcks <benny@noqcks.io>
2018-08-07 00:33:45 +02:00
Julius Volz 159e1537d2
Remove /heap endpoint (#4460)
It was added 5 years ago by Matt and I'm not sure anyone ever used
it after public release (since we have /debug/pprof/heap as well).

It also lacked error checking and allows people to write to disk over HTTP.

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2018-08-04 21:31:28 +02:00
mg03 31f8ca0dfb
api v1 alerts/rules json endpoint
Signed-off-by: mg03 <mgeng03@gmail.com>
2018-07-30 15:29:44 +02:00
Tom Wilkie ccb2ee607b Log errors encountered when marshalling and writing responses.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2018-07-06 18:44:45 +01:00
Fabian Reinartz e7cfc7dae5 web: restore old path prefix behavior
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-15 09:08:14 +01:00
Fabian Reinartz 8a4bda8d57 web: remove security headers
Signed-off-by: Fabian Reinartz <freinartz@google.com>
2018-06-12 08:59:13 -04:00
Fabian Reinartz 9c83e815d1 Merge branch 'release-2.2' into update 2018-06-05 05:29:11 -04:00
Simon Pasquier 097439b0f9 web: limit the number of dropped targets (#4212)
Displaying all the dropped targets in the service-discovery page hurts
the Prometheus server as well as the browser when thousands of dropped
targets exist. This change limits this number to 1,000 and display the
number of active/total targets per scrape configuration.

Add warning when more than 100 targets are dropped

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-05 10:20:32 +01:00
Eric V c207920500 Added HTTP security headers to web.go (#3583)
Signed-off-by: Eric Valenzuela <evalenzuela@thousandeyes.com>
2018-05-03 07:27:34 +01:00
beorn7 94ff07b81d Merge branch 'release-2.2'
Signed-off-by: beorn7 <beorn@soundcloud.com>
2018-04-10 16:50:35 +02:00
Krasi Georgiev ddd46de6f4 Races/3994 (#4005)
Fix race by properly locking access to scrape pools. Use separate mutex for information needed by UI so that UI isn't blocked when targets are being updated.
2018-04-09 15:18:25 +01:00
Ganesh Vernekar b44ce11d1b Added test to check pathPrefix 2018-03-30 11:55:54 +05:30
Ganesh Vernekar cd2820e165 Fix pathPrefix bug from PR-4025 2018-03-30 11:04:15 +05:30
Ganesh Vernekar f30b37e00b Fixed pathPrefix for web pages 2018-03-29 18:02:25 +05:30
Krasi Georgiev 5fec98d0a7 simplify server error handling (#4006) 2018-03-25 10:05:59 +01:00
Marek Siarkowicz bb86c3f62b Report internal runtime information on status page (#3921)
Add information about tsdb, wal and config reload
2018-03-21 16:08:37 +00:00