prometheus

Commit Graph

Author	SHA1	Message	Date
beorn7	a1e595edac	Fix two trivial lint warnings Not sure why those show up for me locally but not if run by the CI. Signed-off-by: beorn7 <beorn@grafana.com>	3 years ago
Dieter Plaetinck	cda025b5b5	TSDB: demistify SeriesRefs and ChunkRefs (#9536 ) * TSDB: demistify seriesRefs and ChunkRefs The TSDB package contains many types of series and chunk references, all shrouded in uint types. Often the same uint value may actually mean one of different types, in non-obvious ways. This PR aims to clarify the code and help navigating to relevant docs, usage, etc much quicker. Concretely: * Use appropriately named types and document their semantics and relations. * Make multiplexing and demuxing of types explicit (on the boundaries between concrete implementations and generic interfaces). * Casting between different types should be free. None of the changes should have any impact on how the code runs. TODO: Implement BlockSeriesRef where appropriate (for a future PR) Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * feedback Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * agent: demistify seriesRefs and ChunkRefs Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
johncming	b882d2b7c7	tsdb/wal: Avoid writing closed channel. (#9566 ) Signed-off-by: johncming <johncming@yahoo.com>	3 years ago
chenlujjj	d18e42c650	refine comments of Checkpoint function (#9655 ) Signed-off-by: chenlujjj <953546398@qq.com>	3 years ago
Marco Pracucci	309b094b92	Optimized MemPostings.EnsureOrder() (#9673 ) * Optimizes MemPostings.EnsureOrder() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Ignore linter warning Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Marco Pracucci	9f5ff5b269	Allow to disable trimming when querying TSDB (#9647 ) * Allow to disable trimming when querying TSDB Signed-off-by: Marco Pracucci <marco@pracucci.com> * Addressed review comments Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added unit test Signed-off-by: Marco Pracucci <marco@pracucci.com> * Renamed TrimDisabled to DisableTrimming Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Marco Pracucci	edd05d7010	Add Head.AppendableMinValidTime() (#9643 ) Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Mateusz Gozdek	b7bdf6fab2	Fix imports formatting According to `2829908806 (r58457095)`. Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Mateusz Gozdek	1a6c2283a3	Format Go source files using 'gofumpt -w -s -extra' Part of #9557 Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>	3 years ago
Julien Pivotto	6e1d6edb33	Exclude agent from windows tests (#9645 ) We are aware of the issue, but while we are working on it, having main tests broken is an annoyance. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
chenlujjj	660329d5b3	add tombstoneFormatVersionSize & tombstonesCRCSize constants (#9625 ) Signed-off-by: chenlujjj <953546398@qq.com>	3 years ago
Praveen Ghuge	64d9b41998	Use testing.T.TempDir() instead of ioutil.TempDir() in tsdb/wal unit tests (#9602 ) Signed-off-by: Praveen Ghuge <praveen.ghuge@outlook.com>	3 years ago
Robert Fratto	bc72a718c4	Initial draft of prometheus-agent (#8785 ) * Initial draft of prometheus-agent This commit introduces a new binary, prometheus-agent, based on the Grafana Agent code. It runs a WAL-only version of prometheus without the TSDB, alerting, or rule evaluations. It is intended to be used to remote_write to Prometheus or another remote_write receiver. By default, prometheus-agent will listen on port 9095 to not collide with the prometheus default of 9090. Truncation of the WAL cooperates on a best-effort case with Remote Write. Every time the WAL is truncated, the minimum timestamp of data to truncate is determined by the lowest sent timestamp of all samples across all remote_write endpoints. This gives loose guarantees that data from the WAL will not try to be removed until the maximum sample lifetime passes or remote_write starts functionining. Signed-off-by: Robert Fratto <robertfratto@gmail.com> * add tests for Prometheus agent (#22) * add tests for Prometheus agent * add tests for Prometheus agent * rearranged tests as per the review comments * update tests for Agent * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * incremental changes to prometheus agent Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * changes as per code review comments Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com> * Commit feedback from code review Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com> Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Port over some comments from grafana/agent Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Rename agent.Storage to agent.DB for tsdb consistency Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Consolidate agentMode ifs in cmd/prometheus/main.go Signed-off-by: Robert Fratto <robertfratto@gmail.com> * Document PreAction usage requirements better for agent mode flags Signed-off-by: Robert Fratto <robertfratto@gmail.com> * remove unnecessary defaultListenAddr Signed-off-by: Robert Fratto <robertfratto@gmail.com> * `go fmt ./tsdb/agent` and fix lint errors Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: SriKrishna Paparaju <paparaju@gmail.com>	3 years ago
Xiaochao Dong	c2d1c85857	close tsdb.head in test case (#9580 ) Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>	3 years ago
Furkan Türkal	0c07663b70	fix: possible race on shared variables in test (#9470 ) Fixes #9433 Signed-off-by: Furkan <furkan.turkal@trendyol.com>	3 years ago
Dieter Plaetinck	d5bfbe3114	improve bstream comments and doc (#9560 ) * improve bstream comments and doc Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * feedback Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Julien Pivotto	73255e15f6	Address golint failures from revive Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	3 years ago
Serge Catudal	8c3eca84db	Fix remote write receiver endpoint for exemplars (#9414 ) Signed-off-by: Serge Catudal <serge.catudal@gmail.com>	3 years ago
Dieter Plaetinck	d5afe0a577	TSDB: Use a dedicated head chunk reference type (#9501 ) * Use dedicated Ref type Throughout the code base, there are reference types masked as regular integers. Let's use dedicated types. They are equivalent, but clearer semantically. This also makes it trivial to find where they are used, and from uses, find the centralized docs. Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * postpone some work until after possible return Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * clarify Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * rename feedback Signed-off-by: Dieter Plaetinck <dieter@grafana.com> * skip header is up to caller Signed-off-by: Dieter Plaetinck <dieter@grafana.com>	3 years ago
Ganesh Vernekar	10d4cb6dc0	Merge remote-tracking branch 'upstream/main' into release-2.30-merge	3 years ago
Ganesh Vernekar	10bc6e80ee	Fix panic on failed snapshot replay and don't hard fail replay on disabled exemplars (#9438 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	b30db03f35	Cut v2.30.2 (#9426 ) * Don't error on overlapping m-mapped chunks during WAL replay (#9381) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Reduce log level during WAL replay on overlapping m-map chunks (#9425) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Cut v2.30.2 Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	a7d499e19a	Reduce log level during WAL replay on overlapping m-map chunks (#9425 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8c597e5166	Don't error on overlapping m-mapped chunks during WAL replay (#9381 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	1fb3c1b598	Replace calls to strings.Compare (#9397 ) < is clearer and faster. As the documentation says, "Basically no one should use strings.Compare." Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	2bcd9f2f69	Link 2 more TSDB blog posts in tsdb/README.md (#9371 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Darshan Chaudhary	1f688657bf	Call delete on head if interval overlaps (#9151 ) * Call delete on head if interval overlaps Signed-off-by: darshanime <deathbullet@gmail.com> * Garbage collect tombstones during head gc Signed-off-by: darshanime <deathbullet@gmail.com> * Truncate tombstones before min time during head gc Signed-off-by: darshanime <deathbullet@gmail.com> * Lock less by deleting all keys in a single pass Signed-off-by: darshanime <deathbullet@gmail.com> * Pass map to DeleteTombstones Signed-off-by: darshanime <deathbullet@gmail.com> * Create new slice to replace old one Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago
Ganesh Vernekar	30534e99d9	Take snapshot only after closing the WAL (#9328 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8944520ccc	Fix deletion of old snapshots (#9314 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	2327236bb5	Decrement active_appenders metric when no samples added (#9230 ) * Decrement active_appenders metric when no samples added Also add a test that the metric is incremented and decremented as expected with and without samples. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Fix comment Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	87d909df4a	Remove symbols map from TSDB head (#9301 ) This saves memory, effort and locking. Since every symbol is also added to postings, `Symbols()` can be implemented there instead. This now has to build a map for deduplication, but `Symbols()` is only called for compaction, and `gc()` used to rebuild the symbols map after every compaction so not an additional cost. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Callum Styan	93886d8417	Fix div by 0 panic is resize function. (#9286 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Ganesh Vernekar	35b1a82594	Exemplars in snapshot (#9255 ) * Exemplars in snapshot Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix lint Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Add docs Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix lint Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix comments Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Levi Harrison	06afe6162c	Also ignore `func1` Signed-off-by: Levi Harrison <git@leviharrison.dev>	3 years ago
Julien Pivotto	d5676fb9e0	Merge pull request #9254 from prometheus/superq/go1.17 Build with Go 1.17 / npm 7 / node 16	3 years ago
SuperQ	e167a45c65	Add new Go build tags. Add new go:build comments based on 1.17 formatting[0]. [0]: https://golang.org/doc/go1.17#gofmt Signed-off-by: SuperQ <superq@gmail.com>	3 years ago
Callum Styan	cc55e57c1b	Fix a data race in the loadWAL function caused by reusing the same error var in multiple goroutines (#9259 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Ganesh Vernekar	8a5d8c15e3	Do not replay checkpoint if it is covered by snapshot (#9226 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Bryan Boreham	9dfdc3eb36	Speed up BenchmarkPostings_Stats (#9213 ) The previous code re-used series IDs 1-1000 many times over, so a lot of time was spent ensuring the lists of series were in ascending order. The intended use of `MemPostings.Add()` is that all series IDs are unique, and changing the benchmark to do this lets it finish ten times faster. (It doesn't affect the benchmark result, just the setup code) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	328a74ca36	Fix bugs and add enhancements to the chunk snapshot (#9185 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Jupiter	84ab705318	32 should better be replaced by "symbolFactor" (#9203 ) Signed-off-by: tanghengjian <1040104807@qq.com>	3 years ago
Marco Pracucci	84e786ebc1	Fixed Decoder.Series() error checking (#9201 ) Signed-off-by: Marco Pracucci <marco@pracucci.com>	3 years ago
Julien Pivotto	cab96a06ef	Merge release 2.29 in main (#9196 ) * PromQL: Fix start and end keywords masking label and metric names This commit fixes an issue with the "at modifier" that introduced two new keywords: `start` and `end`. In grouping options and in metric names, these keywords took precedence over metric or label names, so that those metrics and labels could no longer be referenced. Signed-off-by: Clayton Peters <clayton.peters@man.com> * Add in additional tests for metrics and/or labels called start/end. Signed-off-by: Clayton Peters <clayton.peters@man.com> * : Cut 2.29.0-rc.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> VERSION: bump to 2.29.0-rc.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> * Remove experimental wording on size-based retention Followup of #9004 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Fix PR reference in changelog Signed-off-by: George Brighton <george@gebn.co.uk> * Describe EC2 availability zone IDs at most once per refresh (#9142) Signed-off-by: George Brighton <george@gebn.co.uk> * Describe EC2 availability zones at most once per SD load Closes #9142. Signed-off-by: George Brighton <george@gebn.co.uk> * Incorporate feedback Signed-off-by: George Brighton <george@gebn.co.uk> * Integrate feedback Signed-off-by: George Brighton <george@gebn.co.uk> * Add a compatibility note for macOS users. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * : Cut v2.29.0-rc.1 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> Fix `kuma_sd` targetgroup reporting (#9157) * Bundle all xDS targets into a single group Signed-off-by: austin ce <austin.cawley@gmail.com> * : cut v2.29.0-rc.2 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> Rename links Signed-off-by: Levi Harrison <git@leviharrison.dev> * bump codemirror-promql to 0.17.0 Signed-off-by: Augustin Husson <husson.augustin@gmail.com> * : cut v2.29.0 Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> tsdb: align atomically accessed int64 (#9192) This prevents a panic in 32-bit archs: https://pkg.go.dev/sync/atomic#pkg-note-BUG Fixed #9190 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> * Release 2.29.1 (#9193) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu> Co-authored-by: Clayton Peters <clayton.peters@man.com> Co-authored-by: Frederic Branczyk <fbranczyk@gmail.com> Co-authored-by: George Brighton <george@gebn.co.uk> Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com> Co-authored-by: Levi Harrison <git@leviharrison.dev> Co-authored-by: Augustin Husson <husson.augustin@gmail.com>	3 years ago
Bryan Boreham	040ef175eb	Optimise WAL loading by removing extra map and caching min-time (#9160 ) * BenchmarkLoadWAL: close WAL after use So that goroutines are stopped and resources released Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * BenchmarkLoadWAL: make series IDs co-prime with #workers Series are distributed across workers by taking the modulus of the ID with the number of workers, so multiples of 100 are a poor choice. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * BenchmarkLoadWAL: simulate mmapped chunks Real Prometheus cuts chunks every 120 samples, then skips those samples when re-reading the WAL. Simulate this by creating a single mapped chunk for each series, since the max time is all the reader looks at. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Fix comment Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Remove series map from processWALSamples() The locks that is commented to reduce contention in are now sharded 32,000 ways, so won't be contended. Removing the map saves memory and goes just as fast. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * loadWAL: Cache the last mmapped chunk time So we can skip calling append() for samples it will reject. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Improvements from code review Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Full stops and capitals on comments Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Cache max time in both places mmappedChunks is updated Including refactor to extract function `setMMappedChunks`, to reduce code duplication. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Update head min/max time when mmapped chunks added This ensures we have the correct values if no WAL samples are added for that series. Note that `mSeries.maxTime()` was always `math.MinInt64` before, since that function doesn't consider mmapped chunks. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Bryan Boreham	7407457243	Avoid deadlock when processing duplicate series record (#9170 ) * Avoid deadlock when processing duplicate series record `processWALSamples()` needs to be able to send on its output channel before it can read the input channel, so reads to allow this in case the output channel is full. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * processWALSamples: update comment Previous text seems to relate to an earlier implementation. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	3 years ago
Ganesh Vernekar	ee7e0071d1	Snapshot in-memory chunks on shutdown for faster restarts (#7229 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	848cb5a6d6	Enhanced WAL replay for duplicate series record (#7438 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
Ganesh Vernekar	8002a3ab80	Breakdown tsdb/head.go into multiple files (#9147 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	3 years ago
jinglina	1a430e5f89	remove redundant parentheses (#9134 ) Signed-off-by: jinglina <jinglinax@163.com>	3 years ago
Darshan Chaudhary	c4f2e9eec5	Add present_over_time (#9097 ) * Add present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Add tests for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Address PR comments Signed-off-by: darshanime <deathbullet@gmail.com> * Add documentation for present_over_time Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation Signed-off-by: darshanime <deathbullet@gmail.com> * Update documentation comment Signed-off-by: darshanime <deathbullet@gmail.com>	3 years ago

1 2 3 4 5 ...

399 Commits (9d1c1cd551301f92995e6f64dc67e743a737a64e)