prometheus

Commit Graph

Author	SHA1	Message	Date
Baskar Shanmugam	905a0bd63a	Added 'limit' query parameter support to /api/v1/status/tsdb endpoint (#12336 ) * Added 'topN' query parameter support to /api/v1/status/tsdb endpoint Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Updated query parameter for tsdb status to 'limit' Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Corrected Stats() parameter name from topN to limit Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> * Fixed p.Stats CI failure Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com> --------- Signed-off-by: Baskar Shanmugam <baskar.shanmugam.career@gmail.com>	2 years ago
Nicolas Dumazet	9594fa4dbd	/-/{healthy,ready}/ respond to HEAD (#11160 ) Some frameworks issue HEAD requests to determine health. This resolves prometheus/prometheus#11159 Signed-off-by: Nicolas Dumazet <nicdumz.commits@gmail.com> Signed-off-by: Nicolas Dumazet <nicdumz.commits@gmail.com>	2 years ago
Matthew	ee11771323	fix: add response closures to web_test.go (#10675 ) * fix: add response closures to web_test.go Signed-off-by: mtfoley <mtfoley.mae@gmail.com>	2 years ago
Łukasz Mierzwa	44e5f220c0	Move prometheus_ready metric to web package (#10729 ) This moves prometheus_ready to the web package and links it with the ready variable that decides if HTTP requests should return 200 or 503. This is a follow up change from #10682 Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>	3 years ago
Matthieu MOREL	e2ede285a2	refactor: move from io/ioutil to io and os packages (#10528 ) * refactor: move from io/ioutil to io and os packages * use fs.DirEntry instead of os.FileInfo after os.ReadDir Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>	3 years ago
Augustin Husson	727cdbff4c	remove old UI (#10208 ) * remove old ui Signed-off-by: Augustin Husson <husson.augustin@gmail.com> * add coments and removed unused struct Signed-off-by: Augustin Husson <husson.augustin@gmail.com> * removed tplFunc Signed-off-by: Augustin Husson <husson.augustin@gmail.com>	3 years ago
Shihao Xia	f5c3ea1bf2	fix missing db.Close() in web_test.go in order to avoid goroutine leak (#10035 ) Signed-off-by: Shihao Xia <charlesxsh@hotmail.com>	3 years ago
renzheng.wang	3462b79f4b	fix TestAgentAPIEndPoints (#9882 ) * fix TestAgentAPIEndPoints Signed-off-by: renzheng.wang <wangrzneu@gmail.com> * fix TestAgentAPIEndPoints Signed-off-by: renzheng.wang <wangrzneu@gmail.com>	3 years ago
Matheus Alcantara	c965a7555b	web: use t.TempDir instead of ioutil.TempDir on tests (#9981 ) Signed-off-by: Matheus Alcantara <matheusssilv97@gmail.com>	3 years ago
Julien Pivotto	c78fcd29ba	Adapt UI for Prometheus Agent (#9851 ) * Adapt UI for Prometheus Agent UI is not my strongest skill, but I'd like to have something minimal for the initial release of the agent. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Address review comments Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Add tests Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Add tests, serve only current mode paths Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Update js style, add agent test Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Mateusz Gozdek	79a753db7e	web: randomize used port in TestAgentAPIEndPoints test To make sure we hit the right target and allow running tests in parallel. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	b2c933e317	web: randomize used listening ports for tests So they can be parallelized to speed up tests execution. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Robert Fratto	bc72a718c4	Initial draft of prometheus-agent (#8785 ) * Initial draft of prometheus-agent This commit introduces a new binary, prometheus-agent, based on the Grafana Agent code. It runs a WAL-only version of prometheus without the TSDB, alerting, or rule evaluations. It is intended to be used to remote_write to Prometheus or another remote_write receiver. By default, prometheus-agent will listen on port 9095 to not collide with the prometheus default of 9090. Truncation of the WAL cooperates on a best-effort case with Remote Write. Every time the WAL is truncated, the minimum timestamp of data to truncate is determined by the lowest sent timestamp of all samples across all remote_write endpoints. This gives loose guarantees that data from the WAL will not try to be removed until the maximum sample lifetime passes or remote_write starts functionining. Signed-off-by: Robert Fratto <robertfratto@gmail.com> * add tests for Prometheus agent (#22) * add tests for Prometheus agent * add tests for Prometheus agent * rearranged tests as per the review comments * update tests for Agent * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * incremental changes to prometheus agent Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * Commit feedback from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Port over some comments from grafana/agent Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Rename agent.Storage to agent.DB for tsdb consistency Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Consolidate agentMode ifs in cmd/prometheus/main.go Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Document PreAction usage requirements better for agent mode flags Signed-off-by: Robert Fratto <robertfratto@gmail.com> * remove unnecessary defaultListenAddr Signed-off-by: Robert Fratto <robertfratto@gmail.com> * `go fmt ./tsdb/agent` and fix lint errors Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: SriKrishna Paparaju <paparaju@gmail.com>	3 years ago
Levi Harrison	7bc11dcb06	React UI: Add Starting Screen (#8662 ) * Added walreplay API endpoint Signed-off-by: Levi Harrison <git@leviharrison.dev> * Added starting page to react-ui Signed-off-by: Levi Harrison <git@leviharrison.dev> * Documented the new endpoint Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fixed typos Signed-off-by: Levi Harrison <git@leviharrison.dev> Co-authored-by: Julius Volz <julius.volz@gmail.com> * Removed logo Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed isResponding to isUnexpected Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed width of progress bar Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed width of progress bar Signed-off-by: Levi Harrison <git@leviharrison.dev> * Added DB stats object Signed-off-by: Levi Harrison <git@leviharrison.dev> * Updated starting page to work with new fields Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil (pt. 2) Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil (pt. 3) Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil (and also implementing a method this time) (pt. 4) Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil (and also implementing a method this time) (pt. 5) Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed const to let Signed-off-by: Levi Harrison <git@leviharrison.dev> * Passing nil (pt. 6) Signed-off-by: Levi Harrison <git@leviharrison.dev> * Remove SetStats method Signed-off-by: Levi Harrison <git@leviharrison.dev> * Added comma Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed api Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed to triple equals Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fixed data response types Signed-off-by: Levi Harrison <git@leviharrison.dev> * Don't return pointer Signed-off-by: Levi Harrison <git@leviharrison.dev> * Changed version Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fixed interface issue Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fixed pointer Signed-off-by: Levi Harrison <git@leviharrison.dev> * Fixed copying lock value error Signed-off-by: Levi Harrison <git@leviharrison.dev> Co-authored-by: Julius Volz <julius.volz@gmail.com>	4 years ago
Julien Pivotto	5b4f46a348	Add TLS and basic authentication Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ben Kochie	5055dfbbe4	Listen on web early in startup Avoid starting up components like the TSDB if we can't bind to the web listening port. Signed-off-by: Ben Kochie <superq@gmail.com>	4 years ago
Julien Pivotto	cda52234eb	Fix panic with double close() of channel on /-/quit/ (#8166 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julius Volz	3470ee1fbf	Make React UI the default, keep old UI under /classic (#8142 ) The React app's assets are now served under /assets, while all old custom web assets (including the ones for console templates) are now served from /classic/static. I tested different combinations of --web.external-url and --web.route-prefix with proxies in front, and I couldn't find a problem yet with the routing. Console templates also still work. While migrating old endpoints to /classic, I noticed that /version was being treated like a lot of the old UI pages, with readiness check handler in front of it, etc. I kept it in /version and removed that readiness wrapper, since it doesn't seem to be needed for that endpoint. Signed-off-by: Julius Volz <julius.volz@gmail.com>	4 years ago
Julien Pivotto	6c56a1faaa	Testify: move to require (#8122 ) * Testify: move to require Moving testify to require to fail tests early in case of errors. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * More moves Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	1282d1b39c	Refactor test assertions (#8110 ) * Refactor test assertions This pull request gets rid of assert.True where possible to use fine-grained assertions. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	e208afcc95	web: Remove APIv2 (#7935 ) * web: Remove APIv2 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	6573bf42f2	web: Close and exhaust all response bodies in tests + cleanup snapshot directory (#7850 ) * web: Close and exhaust all response bodies in tests Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Remove empty snapshot directory Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix lint Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Li Zhijian	369245e606	cleanup one more tempfile (#7839 ) Introduced by a recent PR(#7810) Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>	4 years ago
Li Zhijian	6f6ef1451e	Cleanup tempfiles (#7696 ) * cleanup tempfiles for web_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> * cleanup tempfiles for api_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> * cleanup tempfiles for file_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>	4 years ago
Julien Pivotto	e96d786fb8	Stop CMUX and GRPC servers even with stale connections (#7810 ) I have received a recent report for a user which confirms that sometime the GRPC server does not stop propery. It appears that there are 2 issues: 1. The cmux server can refuse to stop if there are stale connections. For that we set the ReadTimeout. 2. The GRPC server graceful stop can never finish. What this PR avoids is: ``` goroutine 227 [semacquire, 2 minutes]: sync.runtime_Semacquire(0xc00059a75c) /usr/local/go/src/runtime/sema.go:56 +0x42 sync.(WaitGroup).Wait(0xc00059a75c) /usr/local/go/src/sync/waitgroup.go:130 +0x64 google.golang.org/grpc.(Server).GracefulStop(0xc00059a600) ``` This PR stops the GRPC server after 15s. Related to the go routing dumps in #6747. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ganesh Vernekar	d4b9fe801f	M-map full chunks of Head from disk (#6679 ) When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory Prom startup now happens in these stages - Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks. - Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series. If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss. [Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks. [The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files. In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file. Prombench results _WAL Replay_ 1h Wal reply time 30% less wal reply time - 4m31 vs 3m36 2h Wal reply time 20% less wal reply time - 8m16 vs 7m _Memory During WAL Replay_ High Churn: 10-15% less RAM - 32gb vs 28gb 20% less RAM after compaction 34gb vs 27gb No Churn: 20-30% less RAM - 23gb vs 18gb 40% less RAM after compaction 32.5gb vs 20gb Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ben Ye	1e4e37144d	Fixed wrongly handled not ready TSDB on web and API. (#7182 ) * fix federate endpoint panic Signed-off-by: yeya24 <yb532204897@gmail.com> * Fixed all cases of not ready TSDB being wrongly handled. * Fixed issue for federation. * Ensured this will never happen again thanks to interfaces * Fixes same issue for stats. * Added tests for readiness. * Fixed bug in stats. It was: status.MaxTime = db.Head().MaxTime() status.MinTime = db.Head().MaxTime() Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Ben Ye	ecda6013ed	Use only local tsdb for federation (#7096 ) Signed-off-by: yeya24 <yb532204897@gmail.com>	5 years ago
Bartlomiej Plotka	a20bebf7eb	Moved readyStorage to main. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	34426766d8	Unify Iterator interfaces. All point to storage now. This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things. All todos I added will be fixed in follow up PRs. * querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged with storage interface.go. All imports that. * querier.SeriesIterator replaced by chunkenc.Iterator * Added chunkenc.Iterator.Seek method and tests for xor implementation (?) * Since we properly handle SelectParams for Select methods I adjusted min max based on that. This should help in terms of performance for queries with functions like offset. * added Seek to deletedIterator and test. * storage/tsdb was removed as it was only a unnecessary glue with incompatible structs. No logic was changed, only different source of abstractions, so no need for benchmarks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Harkishen Singh	489a9aa7b9	Adds normalization of localhost urls in targets page react (#6794 ) * support for globalurls in targets page react Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * fixed tests Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * removed fmts Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * formatted Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions. fixed tests. Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * formated go code Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>	5 years ago
Ganesh Vernekar	5ecef3542d	Cleanup after merging tsdb into prometheus Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Simon Pasquier	be67b8d460	web: fix flaky TestHTTPMetrics() (#5695 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Simon Pasquier	d7f38dfdde	web: add 'code' label to HTTP metrics (#5640 ) * web: add prometheus_http_requests_total metrics Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Add unit test for requestCounter metric Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Minh-Long Do	b26b5c9e96	Add rendering test of template based web endpoints (#5188 ) Signed-off-by: Minh-Long Do <minhlong.langos@gmail.com>	6 years ago
Simon Pasquier	6fa8de132b	web/v1/api: add tests for admin actions (#4767 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Simon Pasquier	2d7f562ed6	web: fix asset paths for Windows platforms (#4616 ) * web: fix asset paths for Windows platforms Signed-off-by: Simon Pasquier <spasquie@redhat.com> * web: add tests Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Fabian Reinartz	e7cfc7dae5	web: restore old path prefix behavior Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Fabian Reinartz	8a4bda8d57	web: remove security headers Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Eric V	c207920500	Added HTTP security headers to web.go (#3583 ) Signed-off-by: Eric Valenzuela <evalenzuela@thousandeyes.com>	7 years ago
beorn7	94ff07b81d	Merge branch 'release-2.2' Signed-off-by: beorn7 <beorn@soundcloud.com>	7 years ago
Ganesh Vernekar	b44ce11d1b	Added test to check pathPrefix	7 years ago
Anton Tereshchenkov	4cb8f6c260	web: remove unused MetricsPath option (#3964 )	7 years ago
Krasi Georgiev	4be9a40b89	fatalf is not thread safe so using panic instead (#3680 )	7 years ago
Krasi Georgiev	c94fa731aa	bypass the proxy for the tests	7 years ago
Krasi Georgiev	0e58cda1fe	use Fatalf instead of panic	7 years ago
Krasi Georgiev	61d793cb1f	handle web_test webhandler errors.	7 years ago
Krasi Georgiev	aca8f85699	fixed the tests	7 years ago
Fabian Reinartz	53875da7fb	Merge pull request #3286 from prometheus/dev-2.0 Dev 2.0	7 years ago

1 2

63 Commits (d3906655bb3d2e1ffbf2f7765a958d88092d8615)