223 Commits (66666f091a1f5fe479c1e53754986074febf37ce)

Author	SHA1	Message	Date
tianwenjie01	b64054365d	Fix DefaultOptions	7 years ago
Fabian Reinartz	74f67e8271	Fix compacting disable/enable ... Enabling and disabling compaction no longer blocks are potentially causes panics.	7 years ago
Fabian Reinartz	3065be97d8	Fix and document locking order for DB	7 years ago
Fabian Reinartz	9c4235532e	Fix compaction selection after creating new heads ... This fixes the case where between block creations no compaction plans are ran. We were not compacting anything in these cases since the on creation the most recent head block always had a high timestamp of 0.	7 years ago
Goutham Veeramachaneni	7b9c536883	Merge pull request #103 from Gouthamve/block-ranges ... Compaction implementation for block-ranges	7 years ago
Goutham Veeramachaneni	9a2ab732b5	Choose ranges by retention and min-block-duration ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	f1ae239c20	Persist the right MaxTime when snapshotting ... This is because we cut a new block from where the snapshotted block ends if we restore from backups and highTimestamp would be where we should be starting from. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	ce23adb579	Initial compaction implementation for block-ranges ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	1e74c155eb	Return empty string to signal non-caching	8 years ago
Fabian Reinartz	3410559c1b	Compact head block early ... Let older head blocks be compacted once the newest once has samples at 50% of its total range. This allows the memory of the compacted blocks to be released and garbage collected before a new head block gets created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3 and memory spikes are reduced.	8 years ago
Fabian Reinartz	03f95ee096	Fix directory file descriptor leak.	8 years ago
Fabian Reinartz	d4cb579443	Add DB.Dir method	8 years ago
Goutham Veeramachaneni	ff4ccb6eb0	Remove unnecessary error from ToggleCompaction fns ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	261cd9f393	Incorporate feedback. ... Move back to {Enable, Disable}Compactions. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	a110a64abd	Add full Snapshot support ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	a1c8425357	Initial implementation of HeadBlock Snapshots ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	3a5ae6b1a4	Merge pull request #82 from Gouthamve/deletes-1 ... Implement deletions	8 years ago
Fabian Reinartz	b3b9245ad6	Add and fix metrics	8 years ago
Goutham Veeramachaneni	6febabeb28	Final delete fixes. ... * Make sure no reads happen on the block when delete is in progress. * Fix bugs in compaction. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	9bf7aa9af1	Misc. fixes incorporating feedback. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	8434019ad9	Merge branch 'master' into deletes-1 ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	0b70333ef6	Add tests for tombstones. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	d32eb25662	Synchronise deletes and compactions. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	3dbb400bef	Initial implementation of Delete on DB. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	debfe9b1e5	Simplify AddFast and avoid errgroup Commits in general	8 years ago
Fabian Reinartz	fb5c5535fc	Misc cleanup	8 years ago
Fabian Reinartz	39df7e2bba	Switch blocks to ULID directories, drop sequenc numbers	8 years ago
Fabian Reinartz	285bc07030	Switch append refs to string	8 years ago
Fabian Reinartz	8b51b7e2be	Make WAL for HeadBlock composeable.	8 years ago
Fabian Reinartz	5534e6c53c	Make HeadBlock impl public, make interface private	8 years ago
Fabian Reinartz	44769c1654	Allow disabling lock file	8 years ago
Fabian Reinartz	35b62f001e	Change offset table layout, add TOC, ...	8 years ago
Fabian Reinartz	54a7b98cff	Fix and test ref handling in DB appender	8 years ago
Brian Brazil	d04bd947ea	Merge pull request #51 from prometheus/rollback ... Make rollback actually rollback.	8 years ago
Brian Brazil	dba222b53c	Make rollback actually rollback. ... Also add tests for basic commit/rollback functionality. Fixes #39	8 years ago
Fabian Reinartz	46c947d94f	Merge pull request #45 from mattbostock/document_appender_not_goroutine_safe ... db: Add comment: Appender not goroutine safe	8 years ago
Matt Bostock	4d5ca4b9c1	db: Fix typo: reference	8 years ago
Matt Bostock	3dd5a4ee09	db: Add comment: Appender not goroutine safe	8 years ago
Fabian Reinartz	778103b450	Add liecence file and headers	8 years ago
Fabian Reinartz	10c7c9acbe	Adjust import names to new repository organisation	8 years ago
Fabian Reinartz	767b88a3a5	Parallelize commits to prevent deadlocks ... This parallelizes commits to prevent deadlocks across inconsistently locked heads. As commits are currently not fully atomic across heads, this does decrease our guarantees.	8 years ago
Fabian Reinartz	a52980e0a8	Add workaround for deadlocks ... This adds a workaround to avoid deadlocks for inconsistent write lock order across headBlocks. Things keep working if transactions only append data for the same timestamp, which is generally the case for Prometheus. Full behavior should be restored in a subsequent change.	8 years ago
Fabian Reinartz	e478d0e3bc	Actually close olds blocks in reloadBlocks ... This fixes a bug leaking memory because blocks were not actually closed as the closing call references the initial, empty slice	8 years ago
Fabian Reinartz	70909ca8ad	Ensure GC runs after each compactor call ... GC is triggered rarely, which may cause unnecessarily high memory spikes when running several compaction cycles in a row. Explicitly run GC so we don't have idle bytes marked as used from the previous cycle.	8 years ago
Fabian Reinartz	55ee4b5b3b	Merge branch 'master' of github.com:fabxc/tsdb	8 years ago
Fabian Reinartz	c18e055d7c	Fix races and add comments on remaining ones	8 years ago
Fabian Reinartz	e837034360	Merge pull request #14 from Gouthamve/log-update ... Update kit/log To New API	8 years ago
Fabian Reinartz	9c93f8f2aa	Fix various races ... This fixes different race condition encoutnered when running Prometheus. It reduces the overall performance in the synthetic benchmark a fair bit but has no indiciations of impacting a real-world setup notably.	8 years ago
Fabian Reinartz	3635569257	Trigger reload correctly on interrupted compaction	8 years ago
Fabian Reinartz	2c999836fb	Add Queryable interface to Block ... This adds the Queryable interface to the Block interface. Head and persisted blocks now implement their own Querier() method and thus isolate customization (e.g. remapPostings) more cleanly.	8 years ago
Fabian Reinartz	11be2cc585	Add composed Block interfaces, remove head generation ... This adds more lower-leve interfaces which are used to compose to different Block interfaces. The DB only uses interfaces instead of explicit persistedBlock and headBlock. The headBlock generation property is dropped as the use-case can be implemented using block sequence numbers.	8 years ago
Goutham Veeramachaneni	df7db4ac07	Update kit/log To New API ... NewContext has been removed couple of weeks back. Ref: https://github.com/go-kit/kit/releases/tag/v0.4.0	8 years ago
Fabian Reinartz	303a4ec3bc	Sync directory on retention delation	8 years ago
Fabian Reinartz	863d38dfee	Fix unreturned lock	8 years ago
Fabian Reinartz	d0770302ed	Add retention deletion	8 years ago
Fabian Reinartz	4962175218	Fix deadlock between heads and headmtx ... With hundreds of concurrent appenders the locking order between the headBlocks on instantiating appenders and write locking the headmtx is hard to impossible to get consistent. Just never instantiate appenders while holding the headmtx lock in any way.	8 years ago
Fabian Reinartz	65b846ae5b	Remove unreturned locks, detect writes on closed heads	8 years ago
Fabian Reinartz	32c32013a6	Fix loading of persisted blocks	8 years ago
Fabian Reinartz	cb4dde7659	Fix WAL log recovery bug ... This fixes a bug where the last WAL file was closed after consuming it instead of being left open for further writes. Reloading of blocks on startup considers loading head blocks now.	8 years ago
Fabian Reinartz	3910b913bd	Revert interval overlap comparison changes	8 years ago
Fabian Reinartz	87805fb83f	Remove Partitioned* code	8 years ago
Fabian Reinartz	eedbebe1d7	Misc cleanups	8 years ago
Fabian Reinartz	50791a412e	Remove leaky Appender abstraction for hashedAppend	8 years ago
Fabian Reinartz	87d270fe11	Fix interval check, make DB shutdown concurrent	8 years ago
Fabian Reinartz	04783831e2	Correctly close compacted blocks.	8 years ago
Fabian Reinartz	55a9b5428a	Add separate head mutex ... Introduce a seperate mutex for the head blocks to avoid a race where a post-compaction reload may run between switching the DB's base mutex to create a new head block in an appender.	8 years ago
Fabian Reinartz	3e569bc964	Optimize memory allocations in writer	8 years ago
Fabian Reinartz	ffb24a98f4	Add missing unlock, run debug endpoint in benchmark	8 years ago
Fabian Reinartz	92120448c2	Properly cleanup compacted dirs, fixes, docs	8 years ago
Fabian Reinartz	2c3e778d90	Compactor interface	8 years ago
Fabian Reinartz	2c3b56350a	Write meta.json file atomically	8 years ago
Fabian Reinartz	6c4217276f	Periodically trigger compaction ... This addresses an issue where the compaction triggered on cutting a new block doesn't find anything as the writers are still active on the block that should be ready for compaction.	8 years ago
Fabian Reinartz	db5c88ea9a	Misc compaction fixes	8 years ago
Fabian Reinartz	b281e4e39b	Accept prometheus.Registerer in constructor	8 years ago
Fabian Reinartz	306831f151	Add per-block state ULID	8 years ago
Fabian Reinartz	78780cd2ba	Segment chunk file ... This adds write path support for segmented chunk data files. Files of 512MB are pre-allocated and written to. If the file size is exceeded, the next file is started. On completion, files are truncated to their final size.	8 years ago
Fabian Reinartz	a3b47c4929	Create default logger for DB	8 years ago
Fabian Reinartz	f734773214	Replace per-file locking with single PID lock file ... File locks have a multitude of problems that make them hard to use correctly. As they are just advisory, they are only meaningful to prevent accidents like running the same process twice. A simple PID file lock works reliably in those cases and is simpler.	8 years ago
Fabian Reinartz	79944a5912	Break out WAL into segment files	8 years ago
Fabian Reinartz	987a90d149	Don't delete files if retention duration is 0	8 years ago
Fabian Reinartz	9c76624df2	Add initial retention cutoff	8 years ago
Fabian Reinartz	f1435f2e2c	Track appended samples properly in metric	8 years ago
Fabian Reinartz	012cf4ef25	Count writer references on head blocks	8 years ago
Fabian Reinartz	5a1c8eaa0e	Fix missing appends after reference lookups	8 years ago
Fabian Reinartz	f556036037	Close blocks before overwriting with compacted one ... This prevents the meta.json file of a compacted block to be overwritten with data of the old block.	8 years ago
Fabian Reinartz	ac5229e1b4	Correctly write empty blocks and extend appenders by new blocks	8 years ago
Fabian Reinartz	42fa342229	Improve multi-head handling ... This improves handling of multiple head blocks. Configuration is simplified to specify the number of concurrently appendable blocks.	8 years ago
Fabian Reinartz	30efe4a58c	Support writing to multiple head blocks ... This is an initial (and hacky) first pass on allowing appending to multiple blocks simultaniously to avoid dropping samples right after cutting a new head block. It's also required for cases like the PGW, where a scrape may contain varying timestamps.	8 years ago
Fabian Reinartz	a61a31a5d7	compaction: add fast-path for compacting mem blocks	8 years ago
Fabian Reinartz	c20cc44b06	Add docs, write sequence number to meta.json	8 years ago
Fabian Reinartz	30d8866c13	Limit compaction range, make cut and compact concurrent	8 years ago
Fabian Reinartz	67d185ceb9	Compact based on compaction generation	8 years ago
Fabian Reinartz	d4779b374c	Properly track and write meta file	8 years ago
Fabian Reinartz	9ddbd64d00	Move stats into meta.json file, cleanup, docs	8 years ago
Fabian Reinartz	e006bc6dc6	Improve error messages, create regular dir for block	8 years ago
Fabian Reinartz	d2322f6095	Improve compaction processing	8 years ago
Fabian Reinartz	343dd9d94c	Fix wrong byte size in WAL base ref	8 years ago
Fabian Reinartz	dd0b69fe1b	Export ErrNotFound	8 years ago
Fabian Reinartz	1c80c33e72	Fix bug of unsorted postings lists being created ... The former approach created unordered postings list by either map iteration of new series being unsorted (fixable) or concurrent writers creating new series interleaved. We switch back to generating ephemeral references for a single batch. Newly created series have to be re-set upon the next insert.	8 years ago
Fabian Reinartz	c7f5590a71	Ensure order of postings when adding new series	8 years ago