250 Commits (4ed3ec7b46c25896930953b234d166c3faa6715b)

Author	SHA1	Message	Date
Ganesh Vernekar	4ed3ec7b46	More info in mmap error message (#8058) ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
frbimo	260cd84d8f	log on empty head compaction result (#8037) ... Signed-off-by: frbimo <fr.bimo@gmail.com>	4 years ago
Gayathri Venkatesh	73e2ce1bd6	Do not ignore reload errors in compactHead (#8051) ... * Modified unknownRefs to unknownRefs.Load() Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Modified db.go Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Revert "Modified unknownRefs to unknownRefs.Load()" This reverts commit 79caf595fa9b9193878dc0dd9dec17d58851ae42. Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com> * Made changes to reload error in db.go Signed-off-by: GayathriVenkatesh <gayaa2010@gmail.com>	4 years ago
Goutham Veeramachaneni	1cc02930b6	Implement series API query hint to not hit chunks (#8050) ... Fixes https://github.com/prometheus/prometheus/issues/5547 Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	4 years ago
Ling Jin	9145200842	tsdb: fix unkown ref in log (#8048) ... Signed-off-by: JinLingChristopher <jinl1037@hotmail.com>	4 years ago
Arthur Silva Sens	4f45e201cc	Promtool tsdb list now prints block sizes (#7993) ... * promtool tsdb list now prints blocks' size Signed-off-by: arthursens <arthursens2005@gmail.com>	4 years ago
Jess G	90680b092c	Add a tsdb writer interface (#7950) ... * add a tsdb writer interface Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use block writer in tsdbutil block creator Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * pass thru context Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use block writer in blockutil and also in tests Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * fix link err Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * add err handling for defer Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * add blockwriter test Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * mv initHead into NewBlockWriter, update tests Signed-off-by: jessicagreben <jessicagrebens@gmail.com> * use blocksize arg, make test more concise Signed-off-by: jessicagreben <jessicagrebens@gmail.com>	4 years ago
garanews	c38816828f	fix few typo (#8023) ... Signed-off-by: garanews <puntogtg@tiscali.it>	4 years ago
Brian Brazil	073e93c768	Gracefully handle unknown WAL record types. (#8004) ... As we're looking to expand what's in the WAL, having old Prometheus servers ignore the new record types rather than treating them as corruption allows for better upgrade/downgrade paths. Adjust some tests accordingly, so they're still testing what they're meant to test. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	4 years ago
Ganesh Vernekar	50ae4e298b	Fix magic number in docs (#7998) ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Ganesh Vernekar	f0d87b5d86	Encode and Decode method for tombstones (#7967) ... * Encode and Decode method for tombstones Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Max Neverov	7e1c27b853	Add tsdb startup duration metric (#7737) ... * Add tsdb wal replay duration metric Signed-off-by: Max Neverov <neverov.max@gmail.com>	4 years ago
Jorge Vallecillo	7aa5fb01bf	tsdb/chunks/head_chunks_test.go: Fix typo (#7953) ... tsdb/db_test.go: Fix typo Signed-off-by: Jorge Vallecillo <jorgevallecilloc@gmail.com>	4 years ago
Tobias Klauser	b5b3bc34da	tsdb/fileutil: Use FcntlFstore from golang.org/x/sys/unix (#7949) ... Direct syscalls using syscall.Syscall(SYS_*, ...) should no longer be used on darwin, see [1]. Instead, use the FcntlFstore libSystem wrapper provided by the golang.org/x/sys/unix package to implement preallocFixed. [1] https://golang.org/doc/go1.12#darwin Signed-off-by: Tobias Klauser <tklauser@distanz.ch>	4 years ago
Xiaochao Dong	a282d25099	tsdb: remove duplicate values set to reduce memory usage(map overhead) (#7915) ... Signed-off-by: Xiaochao Dong (@damnever) <dxc.wolf@gmail.com>	4 years ago
zhulongcheng	5ea6bdca1f	tsdb: sync temporary checkpoint directory before rename (#7004) ... Signed-off-by: zhulongcheng <zhulongcheng.dev@gmail.com>	4 years ago
johncming	75ae384192	tsdb: remove redundant fields. (#7869) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	2255b6f62f	Refactor WAL.Segments method to be part of the wal package (#6477) ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
johncming	252265fa81	tsdb/chunks: remove un-used code. (#6944) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	c806262206	Fix 'chunks.HeadReadWriter: maxt of the files are not set' error (#7856) ... * Fix chunks.HeadReadWriter: maxt of the files are not set Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Yukun Sun	cfd4e05c9e	fix: return a corruption error when iterator function find a chunk that is out of sequence (#7855) ... Signed-off-by: sunyukun <sunyukun@didiglobal.com> Co-authored-by: sunyukun <sunyukun@didiglobal.com>	4 years ago
Max Neverov	bb5c6b38e2	Fix Possible Race Condition in TSDB (#7815) ... * Replace tsdb chunk mapper size with atomic; protect mmappedChunkFiles with read path mutex on DeleteCorrupted Signed-off-by: Max Neverov <neverov.max@gmail.com> * PR fixes Signed-off-by: Max Neverov <neverov.max@gmail.com>	4 years ago
johncming	3e7b463908	add testcase for rangeForTimestamp. (#6454) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Guangwen Feng	9439b2150d	Fix golint warning caused by misspell (#7842) ... Signed-off-by: Guangwen Feng <fenggw-fnst@cn.fujitsu.com>	4 years ago
johncming	b521612042	tsdb: simplify code. (#7792) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
gastonqiu	5a7d398d19	Chore: Log segment number when segment read failed (#7687) ... * Chore: Log segment number when segment read failed To manually fix the WAL files, it is good to know where the corrupt happened so we should log the segment number when the read failed. Related Issue #7506 Signed-off-by: gaston.qiu <gaston.qiu@umbocv.com>	4 years ago
johncming	d19fc71903	tsdb: use NewRangeHead instead. (#7793) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Harold Dost	21a753c4e2	Make file permissions set to allow for wider umask options. (#7782) ... 0644 -> 0666 on all non vendored code. Fixes #7717 Signed-off-by: Harold Dost <harolddost@gmail.com>	4 years ago
Bartlomiej Plotka	f16cbc20d6	tsdb: Bug fix for further continued deletions after crash deletions; added more tests. (#7777) ... * tsdb: Bug fix for further continued after crash deletions; added more tests. Additionally: Added log line for block removal. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comment. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Bartlomiej Plotka	4ae2ef94e0	tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) ... ## Changes: * Rename dir when deleting * Ignoring blocks with broken meta.json on start (we do that on reload) * Compactor writes <ulid>.tmp-for-creation blocks instead of just .tmp * Delete tmp-for-creation and tmp-for-deletion blocks during DB open. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Zhou Hao	40ace418d1	fix misspell (#7764) ... Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>	4 years ago
Frederic Branczyk	e0cf219f0d	tsdb: Save allocations on labels by re-using label array ... Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>	4 years ago
Robert-André Mauchin	ed6ce7ac98	Convert int to string using rune() (#7707) ... See https://github.com/golang/go/issues/32479 Fix #7706. Signed-off-by: Robert-André Mauchin <zebob.m@gmail.com>	4 years ago
Bartlomiej Plotka	28c5cfaf0d	tsdb: Moved code merge series and iterators to differen files; cleanup. No functional changes just move! (#7714) ... I did not want to move those in previous PR to make it easier to review. Now small cleanup time for readability. (: ## Changes * Merge series goes to `storage/merge.go` leaving `fanout.go` for just fanout code. * Moved `fanout test` code from weird separate package to storage. * Unskiped one test: TestFanout_SelectSorted/chunk_querier * Moved block series set codes responsible for querying blocks to `querier.go` from `compact.go` Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
johncming	ac677ed8b3	promql: delete redundant return value. (#7721) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Julien Pivotto	30e079bbd5	TSDB: Fix master tests (#7705) ... Now appenders take a context. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Bartlomiej Plotka	e6d7cc5fa4	tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) ... * tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating. Chained to https://github.com/prometheus/prometheus/pull/7059 * NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it. * Added single SeriesEntry / ChunkEntry for all series implementations. * Unified all vertical, and non vertical for compact and querying to single merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before) * Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples. * Refactored endpoint tests and querier tests to include subtests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments from Brian and Beorn. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed snapshot test and added chunk iterator support for DBReadOnly. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed race when iterating over Ats first. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed populate block tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed endpoints test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added test & fixed case of head open chunk. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed DBReadOnly tests and bug producing 1 sample chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added cases for partial block overlap for multiple full chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added extra tests for chunk meta after compaction. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed small vertical merge bug and added more tests for that. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	4 years ago
Annanay	48b9afd14b	Address comments ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	0b4d448d29	Fix tests ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	263d2aa5f5	Fix failing tests ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	9bba8a6eae	Merge branch 'master' into appender-context ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	89129cd39a	Address comments ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Javier Palomo Almena	b58a613443	Replace sync/atomic with uber-go/atomic (#7683) ... * storage: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * cmd: Replace usage of sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * scripts: Verify that we are not using restricted packages It checks that we are not directly importing 'sync/atomic'. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Reorganise imports in blocks Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * notifier/test: Apply PR suggestions Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * storage/remote: avoid storing references on newEntry Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * Revert "scripts: Verify that we are not using restricted packages" This reverts commit 278d32748e. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * web: Group imports accordingly Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
Javier Palomo Almena	348ff4285f	tsdb: Replace sync/atomic with uber-go/atomic in tsdb (#7659) ... * tsdb/chunks: Replace sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb/heaad: Replace sync/atomic with uber-go/atomic Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * vendor: Make go.uber.org/atomic a direct dependency There is no modifications to go.sum and vendor/ because it was already vendored. Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com> * tsdb: Remove comments referring to the sync/atomic alignment bug Related: https://golang.org/pkg/sync/atomic/#pkg-note-BUG Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>	4 years ago
Annanay	f40e4579b7	gofmt ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Annanay	7f98a744e5	Add context to Appender interface ... Signed-off-by: Annanay <annanayagarwal@gmail.com>	4 years ago
Ben Ye	50c261502e	add tsdb cmds into promtool (#6088) ... Signed-off-by: yeya24 <yb532204897@gmail.com> update tsdb cli in makefile and promu Signed-off-by: yeya24 <yb532204897@gmail.com> remove building tsdb bin Signed-off-by: yeya24 <yb532204897@gmail.com> remove useless func Signed-off-by: yeya24 <yb532204897@gmail.com> refactor analyzeBlock Signed-off-by: yeya24 <yb532204897@gmail.com> Fix Makefile Signed-off-by: Simon Pasquier <spasquie@redhat.com>	4 years ago
johncming	9801f52b0a	tsdb/chunks: fix bug of data race(#7643). (#7646) ... Signed-off-by: johncming <johncming@yahoo.com>	4 years ago
Ganesh Vernekar	4a8531a64b	BlocksToDelete function in DB options (#7638) ... * Optional retention filter for DB Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Specify len for the map creation Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	4 years ago
Julien Pivotto	ffc925dd21	TSDB: Error when we commit/rollback twice (#7593) ... * TSDB: Error when we commit/rollback twice Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago