prometheus

Commit Graph

Author	SHA1	Message	Date
Brian Brazil	fdf1c29224	Fix metric description of prometheus_tsdb_symbol_table_size_bytes. (#8080 ) This is how much memory we use to load in the on-disk symbol tables, not the size of the tables themselves. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	4 years ago
Ganesh Vernekar	ce4b3ac282	Simplify TestHeadReadWriter_Truncate (#7437 ) * Simplify TestHeadReadWriter_Truncate Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	2624d827fa	Read repair empty last file in chunks_head (#8061 ) * Read repair empty file in chunks_head Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor and introduce repairLastChunkFile Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Attempt windows test fix Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Bartlomiej Plotka	2fe1e9fa93	Create a checkpoint only at the end of Compact call (#8067 ) * Create a checkpoint only at the end of Compact call Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix Bartek's offline reviews Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Introduce TruncateInMemory and TruncateWAL Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Small enhancements and test fixing attempts Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add TestOneCheckpointPerCompactCall Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Don't truncate WAL on block compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Simplified the algo. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Better protection around calling truncateWAL, truncate WAL on Head compaction error Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	d9cd913219	Add links to TSDB blog post series in tsdb/README.md (#8078 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	59733b1a26	TSDB: use `blocks` instead of db.blocks in condition (#8068 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ganesh Vernekar	5de7d15879	Consistent naming and better names for fields in TSDB format docs (#8077 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
jessicagreben	36ac0b68f1	merge master, fix conflicts	4 years ago
Bartlomiej Plotka	9981b3f3ee	labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025 ) * labels: Reduce allocated memory by Hash method in edge cases; Added tests. Old: /tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$ goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/pkg/labels BenchmarkLabels_Hash BenchmarkLabels_Hash/typical_labels_under_1KB BenchmarkLabels_Hash/typical_labels_under_1KB-12 5366161 259 ns/op 0 B/op 0 allocs/op BenchmarkLabels_Hash/bigger_labels_over_1KB BenchmarkLabels_Hash/bigger_labels_over_1KB-12 1700371 767 ns/op 2048 B/op 1 allocs/op BenchmarkLabels_Hash/extremely_large_label_value_10MB BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 356 3743115 ns/op 10523442 B/op 1 allocs/op PASS New: /tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$ goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/pkg/labels BenchmarkLabels_Hash BenchmarkLabels_Hash/typical_labels_under_1KB BenchmarkLabels_Hash/typical_labels_under_1KB-12 4758883 259 ns/op 0 B/op 0 allocs/op BenchmarkLabels_Hash/bigger_labels_over_1KB BenchmarkLabels_Hash/bigger_labels_over_1KB-12 3324492 357 ns/op 80 B/op 1 allocs/op BenchmarkLabels_Hash/extremely_large_label_value_10MB BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 1087 1083949 ns/op 9734 B/op 1 allocs/op PASS Process finished with exit code 0 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Kemal's comment. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Moved to v2 xxhash for improvements. New: /tmp/___BenchmarkLabels_Hash_in_github_com_prometheus_prometheus_pkg_labels -test.v -test.bench ^\QBenchmarkLabels_Hash\E$ -test.run ^$ -test.benchtime 10s goos: linux goarch: amd64 pkg: github.com/prometheus/prometheus/pkg/labels BenchmarkLabels_Hash BenchmarkLabels_Hash/typical_labels_under_1KB BenchmarkLabels_Hash/typical_labels_under_1KB-12 53447894 221 ns/op 0 B/op 0 allocs/op BenchmarkLabels_Hash/bigger_labels_over_1KB BenchmarkLabels_Hash/bigger_labels_over_1KB-12 42341754 326 ns/op 0 B/op 0 allocs/op BenchmarkLabels_Hash/extremely_large_label_value_10MB BenchmarkLabels_Hash/extremely_large_label_value_10MB-12 10000 1248546 ns/op 0 B/op 0 allocs/op PASS Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Removed old xxhash package. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Pined minor version. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Ganesh Vernekar	4ed3ec7b46	More info in mmap error message (#8058 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
frbimo	260cd84d8f	log on empty head compaction result (#8037 ) Signed-off-by: frbimo <fr.bimo@gmail.com>	4 years ago
Gayathri Venkatesh	73e2ce1bd6	Do not ignore reload errors in compactHead (#8051 ) * Modified unknownRefs to unknownRefs.Load() Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Modified db.go Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Revert "Modified unknownRefs to unknownRefs.Load()" This reverts commit 79caf595fa9b9193878dc0dd9dec17d58851ae42. Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Made changes to reload error in db.go Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>	4 years ago
Goutham Veeramachaneni	1cc02930b6	Implement series API query hint to not hit chunks (#8050 ) Fixes https://github.com/prometheus/prometheus/issues/5547 Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	4 years ago
Ling Jin	9145200842	tsdb: fix unkown ref in log (#8048 ) Signed-off-by: JinLingChristopher <jinl1037@hotmail.com>	4 years ago
Arthur Silva Sens	4f45e201cc	Promtool tsdb list now prints block sizes (#7993 ) * promtool tsdb list now prints blocks' size Signed-off-by: arthursens <arthursens2005@gmail.com>	4 years ago
Jess G	90680b092c	Add a tsdb writer interface (#7950 ) * add a tsdb writer interface Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use block writer in tsdbutil block creator Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * pass thru context Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use block writer in blockutil and also in tests Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * fix link err Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * add err handling for defer Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * add blockwriter test Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * mv initHead into NewBlockWriter, update tests Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use blocksize arg, make test more concise Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
garanews	c38816828f	fix few typo (#8023 ) Signed-off-by: garanews <puntogtg@tiscali.it>	4 years ago
Brian Brazil	073e93c768	Gracefully handle unknown WAL record types. (#8004 ) As we're looking to expand what's in the WAL, having old Prometheus servers ignore the new record types rather than treating them as corruption allows for better upgrade/downgrade paths. Adjust some tests accordingly, so they're still testing what they're meant to test. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	4 years ago
Ganesh Vernekar	50ae4e298b	Fix magic number in docs (#7998 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	f0d87b5d86	Encode and Decode method for tombstones (#7967 ) * Encode and Decode method for tombstones Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Max Neverov	7e1c27b853	Add tsdb startup duration metric (#7737 ) * Add tsdb wal replay duration metric Signed-off-by: Max Neverov <neverov.max@gmail.com>	4 years ago
Jorge Vallecillo	7aa5fb01bf	tsdb/chunks/head_chunks_test.go: Fix typo (#7953 ) tsdb/db_test.go: Fix typo Signed-off-by: Jorge Vallecillo <jorgevallecilloc@gmail.com>	4 years ago
Tobias Klauser	b5b3bc34da	tsdb/fileutil: Use FcntlFstore from golang.org/x/sys/unix (#7949 ) Direct syscalls using syscall.Syscall(SYS_*, ...) should no longer be used on darwin, see [1]. Instead, use the FcntlFstore libSystem wrapper provided by the golang.org/x/sys/unix package to implement preallocFixed. [1] https://golang.org/doc/go1.12#darwin Signed-off-by: Tobias Klauser <tklauser@distanz.ch>	4 years ago
Xiaochao Dong	a282d25099	tsdb: remove duplicate values set to reduce memory usage(map overhead) (#7915 ) Signed-off-by: Xiaochao Dong (@damnever) <dxc.wolf@gmail.com>	4 years ago
zhulongcheng	5ea6bdca1f	tsdb: sync temporary checkpoint directory before rename (#7004 ) Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>	4 years ago
johncming	75ae384192	tsdb: remove redundant fields. (#7869 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	2255b6f62f	Refactor WAL.Segments method to be part of the wal package (#6477 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
johncming	252265fa81	tsdb/chunks: remove un-used code. (#6944 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	c806262206	Fix 'chunks.HeadReadWriter: maxt of the files are not set' error (#7856 ) * Fix chunks.HeadReadWriter: maxt of the files are not set Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Yukun Sun	cfd4e05c9e	fix: return a corruption error when iterator function find a chunk that is out of sequence (#7855 ) Signed-off-by: sunyukun <sunyukun@didiglobal.com> Co-authored-by: sunyukun <sunyukun@didiglobal.com>	4 years ago
Max Neverov	bb5c6b38e2	Fix Possible Race Condition in TSDB (#7815 ) * Replace tsdb chunk mapper size with atomic; protect mmappedChunkFiles with read path mutex on DeleteCorrupted Signed-off-by: Max Neverov <neverov.max@gmail.com> * PR fixes Signed-off-by: Max Neverov <neverov.max@gmail.com>	4 years ago
johncming	3e7b463908	add testcase for rangeForTimestamp. (#6454 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Guangwen Feng	9439b2150d	Fix golint warning caused by misspell (#7842 ) Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	4 years ago
johncming	b521612042	tsdb: simplify code. (#7792 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
gastonqiu	5a7d398d19	Chore: Log segment number when segment read failed (#7687 ) * Chore: Log segment number when segment read failed To manually fix the WAL files, it is good to know where the corrupt happened so we should log the segment number when the read failed. Related Issue #7506 Signed-off-by: gaston.qiu <gaston.qiu@umbocv.com>	4 years ago
johncming	d19fc71903	tsdb: use NewRangeHead instead. (#7793 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Harold Dost	21a753c4e2	Make file permissions set to allow for wider umask options. (#7782 ) 0644 -> 0666 on all non vendored code. Fixes #7717 Signed-off-by: Harold Dost <harolddost@gmail.com>	4 years ago
Bartlomiej Plotka	f16cbc20d6	tsdb: Bug fix for further continued deletions after crash deletions; added more tests. (#7777 ) * tsdb: Bug fix for further continued after crash deletions; added more tests. Additionally: Added log line for block removal. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comment. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Bartlomiej Plotka	4ae2ef94e0	tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772 ) ## Changes: * Rename dir when deleting * Ignoring blocks with broken meta.json on start (we do that on reload) * Compactor writes <ulid>.tmp-for-creation blocks instead of just .tmp * Delete tmp-for-creation and tmp-for-deletion blocks during DB open. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Zhou Hao	40ace418d1	fix misspell (#7764 ) Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>	4 years ago
Frederic Branczyk	e0cf219f0d	tsdb: Save allocations on labels by re-using label array Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Robert-André Mauchin	ed6ce7ac98	Convert int to string using rune() (#7707 ) See https://github.com/golang/go/issues/32479 Fix #7706. Signed-off-by: Robert-André Mauchin <zebob.m@gmail.com>	4 years ago
Bartlomiej Plotka	28c5cfaf0d	tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714 ) I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (: ## Changes * Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code. * Moved `fanout test` code from weird separate package to storage. * Unskiped one test: TestFanout_SelectSorted/chunk_querier * Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
johncming	ac677ed8b3	promql: delete redundant return value. (#7721 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Julien Pivotto	30e079bbd5	TSDB: Fix master tests (#7705 ) Now appenders take a context. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	e6d7cc5fa4	tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069 ) * tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating. Chained to https://github.com/prometheus/prometheus/pull/7059 * NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it. * Added single SeriesEntry / ChunkEntry for all series implementations. * Unified all vertical, and non vertical for compact and querying to single merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before) * Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples. * Refactored endpoint tests and querier tests to include subtests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments from Brian and Beorn. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed snapshot test and added chunk iterator support for DBReadOnly. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed race when iterating over Ats first. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed populate block tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed endpoints test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added test & fixed case of head open chunk. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed DBReadOnly tests and bug producing 1 sample chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added cases for partial block overlap for multiple full chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added extra tests for chunk meta after compaction. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed small vertical merge bug and added more tests for that. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Annanay	48b9afd14b	Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	0b4d448d29	Fix tests Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	263d2aa5f5	Fix failing tests Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	9bba8a6eae	Merge branch 'master' into appender-context Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	89129cd39a	Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Javier Palomo Almena	b58a613443	Replace sync/atomic with uber-go/atomic (#7683 ) * storage: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * cmd: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * scripts: Verify that we are not using restricted packages It checks that we are not directly importing 'sync/atomic'. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Reorganise imports in blocks Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier/test: Apply PR suggestions Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * storage/remote: avoid storing references on newEntry Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Revert "scripts: Verify that we are not using restricted packages" This reverts commit `278d32748e`. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Group imports accordingly Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
Javier Palomo Almena	348ff4285f	tsdb: Replace sync/atomic with uber-go/atomic in tsdb (#7659 ) * tsdb/chunks: Replace sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb/heaad: Replace sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * vendor: Make go.uber.org/atomic a direct dependency There is no modifications to go.sum and vendor/ because it was already vendored. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Remove comments referring to the sync/atomic alignment bug Related: https://golang.org/pkg/sync/atomic/#pkg-note-BUG Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
jessicagreben	7504b5ce7c	add rule importer with tsdb block writer Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>	4 years ago
Annanay	f40e4579b7	gofmt Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	7f98a744e5	Add context to Appender interface Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Ben Ye	50c261502e	add tsdb cmds into promtool (#6088 ) Signed-off-by: yeya24 <yb532204897@gmail.com> update tsdb cli in makefile and promu Signed-off-by: yeya24 <yb532204897@gmail.com> remove building tsdb bin Signed-off-by: yeya24 <yb532204897@gmail.com> remove useless func Signed-off-by: yeya24 <yb532204897@gmail.com> refactor analyzeBlock Signed-off-by: yeya24 <yb532204897@gmail.com> Fix Makefile Signed-off-by: Simon Pasquier <spasquie@redhat.com>	4 years ago
johncming	9801f52b0a	tsdb/chunks: fix bug of data race(#7643 ). (#7646 ) Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	4a8531a64b	BlocksToDelete function in DB options (#7638 ) * Optional retention filter for DB Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Specify len for the map creation Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	ffc925dd21	TSDB: Error when we commit/rollback twice (#7593 ) * TSDB: Error when we commit/rollback twice Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	cfe30a7b62	TSDB: Use t.Cleanup to delete temporary files (#7620 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	62805b2fe9	tsdb: test for leaks (#7566 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Krasimir Georgiev	ccab2b30c9	Test no panic after a WAL corruption (#7625 ) * no panic the head memseries has chunks in it Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> * fix a panic when querying after a wal corruption. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> * review nits Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> * Add test for reading the data after a wal corruption. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Update tsdb/db_test.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Update tsdb/db_test.go Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> * spellings Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	4 years ago
Julien Pivotto	9b8cc663f7	Merge pull request #7623 from prometheus/release-2.20 Release 2.20	4 years ago
Krasi Georgiev	d30492cbb0	Avoid panic when the headChunk is nil during isolation. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>	4 years ago
Zhou Hao	ddedf454d0	add os.RemoveAll err verification (#7540 ) * add os.RemoveAll err verification for watcher_test Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com> * add os.RemoveAll err verification for db_test Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com> * add os.RemoveAll err verification for write_test Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com> * add os.RemoveAll err verification for queue_manager_test Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com> * tsdb/wal/watcher_test: add close operation before delete Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>	4 years ago
Ganesh Vernekar	1760c7474c	Replay m-map chunks irrespective of WAL (#7589 ) * Replay m-map chunks irrespective of WAL Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * More logs Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Björn Rabenstein	e0067a7bd8	Merge pull request #7573 from codesome/mmap-empty-files Avoid empty mmap files by using .tmp files to write headers	4 years ago
Ganesh Vernekar	b8a7e80f9b	Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	ea013343ca	Log when starting to create a checkpoint (#7581 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	7a763ff61e	Avoid empty mmap files by using .tmp files to write headers Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Bartlomiej Plotka	823b218e1b	Fixed race between compact (gc, populate) and head append causing unknown symbol error. (#7560 ) * Fixed race between compact (gc, populate) and head append causing unknown symbol error. Fixes https://github.com/prometheus/prometheus/issues/7373 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Bartlomiej Plotka	492061b24c	Revert "Fix unknown symbol error during head compaction (#7526 )" (#7556 ) This reverts commit `30505a202a`.	4 years ago
Ganesh Vernekar	30505a202a	Fix unknown symbol error during head compaction (#7526 ) * Fix race during head compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Comment out the test Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Skip test instead of commenting it out Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Marco Pracucci	2f6bf7de4c	Optimise labels regex matchers containing a literal within the pattern (#7503 ) * Added labels matchers regex fast path for literals within the regex Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Harkishen Singh	f32307b656	Increments WAL corruption metric on WAL corruption during checkpointing (#7491 ) * Increments wal corruption metric on error during checkpointing Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com> * check for wal corruption error Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>	4 years ago
Ganesh Vernekar	e65e2e0dac	Fix panic from db metrics (#7501 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Bartlomiej Plotka	1861bf38f5	tombstones: Fixed Add method in order to support trimming time series; Simplified the algo. (#7471 ) * tombstones: Fixed Add method in order to support edge trimming; Simplified the algo. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Removed duplicated test case. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed comment, removed "edge" mention. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Removed trimming word. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Marco Pracucci	cef4dd6fff	Optimized label regex matcher with literal prefix and/or suffix (#7453 ) * Optimized label regex matcher with literal prefix and/or suffix Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added license Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added more tests cases with newlines Signed-off-by: Marco Pracucci <marco@pracucci.com> * Restored deleted test Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Ganesh Vernekar	082c17b691	Introduce SortedLabelValues/LabelValues to speedup queries for high cardinality (#7448 ) * Introduce LabelValuesUnsorted to speedup queries for high cardinality Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add sort check Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Bartlomiej Plotka	b788986717	storage: Adjusted fully storage layer support for chunk iterators: Remote read client, readyStorage, fanout. (#7059 ) * Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334 * Added ChunkQueryable implementation to fanout and readyStorage. * Added more comments. * Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for both chunks and series, for both querying and compacting (!) + made sure duplicates are merged. * Added ErrChunkSeriesSet * Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion. * Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one. * Improved tests. * Split remote client into Write (old storage) and read. * Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved all config based options to sampleAndChunkQueryableClient to aboid boilerplate. In next commit: Changes for TSDB. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Joe Lei	74a73ba1cf	fix analyze limit not work expected (#7430 ) Signed-off-by: joelei <thezero12@hotmail.com>	5 years ago
Ganesh Vernekar	b7c46a8c79	Merge remote-tracking branch 'upstream/master' into merge-release-2.19 Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ganesh Vernekar	48fae12b89	Fix unsequential m-map files (#7414 ) * Fix unsequential m-map files Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Marco Pracucci	3b529ddbce	Cleanup bstream_test.go based on post-merge feedback received on #7390 (#7413 ) * Fixed bstream test license Signed-off-by: Marco Pracucci <marco@pracucci.com> * Simplified bstreamReader.loadNextBuffer() Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed date in license Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Simon Pasquier	d634785944	tsdb/docs: fix head chunks directory + link from README (#7309 ) Signed-off-by: Simon Pasquier <spasquie@redhat.com>	5 years ago
Simon Pasquier	2f12049371	tsdb: improve logs when encountering corruption (#7308 ) * tsdb: improve logs when encountering corruption Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Wrap corrupted block errors Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Add file path to head chunks Signed-off-by: Simon Pasquier <spasquie@redhat.com>	5 years ago
Marco Pracucci	f42ed03dc5	Optimized bstream reader used by XORChunk iterator (#7390 ) * Optimized bstream reader Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed linter Signed-off-by: Marco Pracucci <marco@pracucci.com> * Added license to new file Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed type cast Signed-off-by: Marco Pracucci <marco@pracucci.com> * Changed comments Signed-off-by: Marco Pracucci <marco@pracucci.com> * Improved comments and rolledback no-op changes Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed race condition Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Julien Pivotto	f893786153	Fix TSDB test failure (#7394 ) PR #7338 was not rebased on top of master and interface had changed. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Krasimir Georgiev	ab6203b7c7	add head compaction test (#7338 )	5 years ago
Ganesh Vernekar	9593b64ce6	Merge branch 'master' into to-merge-release-2.19	5 years ago
Kemal Akkoyun	66dfb951c4	: Consistent Error/Warning handling for SeriesSet iterator: Allowing Async Select (#7251 ) Add errors and Warnings to SeriesSet Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Change Querier interface and refactor accordingly Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor promql/engine to propagate warnings at eval stage Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Make sure all the series from all Selects are pre-advanced Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Separate merge series sets Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Clean Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor merge querier failure handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactored and simplified fanout with improvements from incoming chunk iterator PRs. * Secondary logic is hidden, instead of weird failed series set logic we had. * Fanout is well commented * Fanout closing record all errors * MergeQuerier improved API (clearer) * deferredGenericMergeSeriesSet is not needed as we return no samples anyway for failed series sets (next = false). Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix formatting Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix CI issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Added final tests for error handling. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. * Moved hints in populate to be allocated only when needed. * Used sync.Once in secondary Querier to achieve all-or-nothing partial response logic. * Select after first Next is done will panic. NOTE: in lazySeriesSet in theory we could just panic, I think however we can totally just return error, it will panic in expand anyway. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Utilize errWithWarnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix recently introduced expansion issue Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add tests for secondary querier error handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Implement lazy merge Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add name to test cases Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Reorganize Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Remove redundant warnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix rebase mistake Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Ganesh Vernekar	1627d234da	Moves the atomically accessed member to the top of the struct (#7365 ) * Moves the 64bit atomically accessed field to the top of the struct. Signed-off-by: Bryan Varner <1652015+bvarner@users.noreply.github.com> * Moves the 64bit atomically accessed field to the top of the struct. Signed-off-by: Bryan Varner <1652015+bvarner@users.noreply.github.com> * Fixing up go fmt formatting issues. Signed-off-by: Bryan Varner <1652015+bvarner@users.noreply.github.com> Co-authored-by: Bryan Varner <1652015+bvarner@users.noreply.github.com>	5 years ago
Peter Štibraný	ff80690a6e	Optimise lowWatermark in Isolation (#7332 ) * Track open appenders in doubly-linked list to make lowWatermark O(1). * Use RW locks. * Added BenchmarkIsolationWithState. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Jess G	fdc49fae5b	Added time range parameters to labelNames API (#7288 ) * add time range params to labelNames api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * evaluate min/max time range when reading labels from the head Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelValues api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test, add docs Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add a test for head min max range Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test to match comment Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * address CR comments Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * combine vars only used once Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelNames api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * evaluate min/max time range when reading labels from the head Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelValues api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test, add docs Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add a test for head min max range Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test to match comment Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * address CR comments Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * combine vars only used once Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * restart ci Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * use range expectedLabelNames instead of range actualLabelNames in test Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>	5 years ago
Ganesh Vernekar	a1355eb7c7	Remove time based m-map file creation (#7314 ) * Remove time based m-map file creation Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ganesh Vernekar	83619aa9ac	Preallocate m-map file only for Windows (#7306 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Guangwen Feng	2393d6137b	Add unit test case for func Type in record.go (#7082 ) Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	5 years ago
Krasimir Georgiev	f4dd45609a	Use min and maxt of the range head when creating a block (#7282 ) Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>	5 years ago
Krasimir Georgiev	09df8d94e0	More explicit chunks and head error handling. (#7277 )	5 years ago

1 2 3 4 5 ...

310 Commits (731545ad341ed4588a44ff38bc7df06fb21995d0)