56 Commits (81222849bcf523fabfcf254f3afb4ef2bcd70488)

Author	SHA1	Message	Date
Fabian Reinartz	81222849bc	Filter WAL data in Head, misc fixes	7 years ago
Fabian Reinartz	33e9bdf403	WAL refactoring and truncation fixes and test	7 years ago
Fabian Reinartz	c36d574290	Replace single head lock with granular locks ... This adds various new locks to replace the single big lock on the head. All parts now must be COW as they may be held by clients after initial retrieval. Series by ID and hashes are now held in a stripe lock to reduce contention and total holding time during GC. This should reduce starvation of readers.	7 years ago
Fabian Reinartz	1ddedf2b30	Change series ID from uint32 to uint64	7 years ago
Goutham Veeramachaneni	1698c516ad	[WIP]: WAL implementation ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	893b6ec506	Add tests for GC and chunk truncation	7 years ago
Fabian Reinartz	5cf2662074	Refactor WAL into Head and misc improvements	7 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads ... This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	7 years ago
Alin Sinpalean	8f7934d025	Take the fdatasync() syscall out from under mutex lock to (mostly) avoid scrape and eval from blocking on disk I/O.	7 years ago
Matt Layher	78b15c3434	Add newCRC32 function to simplify hash initialization	7 years ago
Goutham Veeramachaneni	7438ed7035	Expose Intervals type for use by TombstoneReader. ... TombstoneReader is exposed but Intervals is not. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	44e9ae38b5	Incorporate PR feedback. ... * Expose Stone as it is used in an exported method. * Move from tombstoneReader to []Stone for the same reason as above. * Make WAL reading a little cleaner. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	bacb143b7e	Simplify tombstone and WAL Delete formats. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	f29fb62fba	Make TombstoneReader a Getter. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	9bf7aa9af1	Misc. fixes incorporating feedback. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	3eb4119ab1	Make HeadBlock use WAL. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	39df7e2bba	Switch blocks to ULID directories, drop sequenc numbers	8 years ago
Fabian Reinartz	4862b261d0	Abstract WAL into interface	8 years ago
Fabian Reinartz	535532ca02	Export refdSample ... The type was part of a exported method signatures and should therefore be exported as well.	8 years ago
Julius Volz	8d1fb4fa01	Minor comment fixes and additions.	8 years ago
Fabian Reinartz	778103b450	Add liecence file and headers	8 years ago
Fabian Reinartz	10c7c9acbe	Adjust import names to new repository organisation	8 years ago
Fabian Reinartz	55ee4b5b3b	Merge branch 'master' of github.com:fabxc/tsdb	8 years ago
Fabian Reinartz	9c93f8f2aa	Fix various races ... This fixes different race condition encoutnered when running Prometheus. It reduces the overall performance in the synthetic benchmark a fair bit but has no indiciations of impacting a real-world setup notably.	8 years ago
Goutham Veeramachaneni	761e4768f3	Lint and Vet Fixes	8 years ago
Fabian Reinartz	ad5812d83a	Handle WAL corruption by truncating ... This adds handling for various corruption scenarios of the WAL. If corruption is encountered, we truncate the WAL after the last valid entry transparently and continue appending after the offset.	8 years ago
Fabian Reinartz	d3edfb5540	Test multiple full write/restore cylcles in WAL	8 years ago
Fabian Reinartz	cb4dde7659	Fix WAL log recovery bug ... This fixes a bug where the last WAL file was closed after consuming it instead of being left open for further writes. Reloading of blocks on startup considers loading head blocks now.	8 years ago
Fabian Reinartz	55a9b5428a	Add separate head mutex ... Introduce a seperate mutex for the head blocks to avoid a race where a post-compaction reload may run between switching the DB's base mutex to create a new head block in an appender.	8 years ago
Fabian Reinartz	ffb24a98f4	Add missing unlock, run debug endpoint in benchmark	8 years ago
Fabian Reinartz	f734773214	Replace per-file locking with single PID lock file ... File locks have a multitude of problems that make them hard to use correctly. As they are just advisory, they are only meaningful to prevent accidents like running the same process twice. A simple PID file lock works reliably in those cases and is simpler.	8 years ago
Fabian Reinartz	afa084920c	Use Castagnoli polynomial for CRC32s; cache them	8 years ago
Fabian Reinartz	9c7a88223e	Add full encode/decode WAL cycle test	8 years ago
Fabian Reinartz	2c97428a79	Add basic WAL tests	8 years ago
Fabian Reinartz	84e8027a8e	Close consumed WAL segments, verify checksums on read	8 years ago
Fabian Reinartz	eeb03e97e2	Add headers and format flag for WAL segments	8 years ago
Fabian Reinartz	79944a5912	Break out WAL into segment files	8 years ago
Fabian Reinartz	9ddbd64d00	Move stats into meta.json file, cleanup, docs	8 years ago
Fabian Reinartz	343dd9d94c	Fix wrong byte size in WAL base ref	8 years ago
Fabian Reinartz	9cf49f68e9	wal: use larger buffer	8 years ago
Fabian Reinartz	1c80c33e72	Fix bug of unsorted postings lists being created ... The former approach created unordered postings list by either map iteration of new series being unsorted (fixable) or concurrent writers creating new series interleaved. We switch back to generating ephemeral references for a single batch. Newly created series have to be re-set upon the next insert.	8 years ago
Fabian Reinartz	0dffd52238	Use page writer in compaction	8 years ago
Fabian Reinartz	54f5027406	Put WAL lock down into encoder	8 years ago
Fabian Reinartz	300f4e2abf	Use separate lock for series creation ... This uses the head block's own lock to only lock if new series were encountered. In the general append case we just need to hold a	8 years ago
Fabian Reinartz	71efd2e08d	Periodically fsync WAL, make head cut async	8 years ago
Fabian Reinartz	2eb544c98e	Change file names and maker parsing safer	8 years ago
Fabian Reinartz	937cdb579c	Switch to sequential block names ... This changes block directory names from the int64 timestamp to sequential numbering.	8 years ago
Fabian Reinartz	dbd2b21d2e	Make persistence atomic	8 years ago
Fabian Reinartz	62b8ded0a5	Rename iterator value getters to At()	8 years ago
Fabian Reinartz	7280533c42	Add basic shard metrics	8 years ago