70 Commits (74b0336d0694b342e81dfcecf0c3038e2bffc7e3)

Author	SHA1	Message	Date
Simon Pasquier	3e17cd1621	Instrument WAL corruptions	7 years ago
Fabian Reinartz	27f1b8aac3	Merge pull request #162 from BasPH/fsync-duration ... Instrument WAL fsync	7 years ago
Bas Harenslak	5e1c258a98	Instrument WAL fsync	7 years ago
Goutham Veeramachaneni	da565f975e	Merge pull request #161 from prometheus/fileutil ... Remove dependency on etcd/pkg/fileutil	7 years ago
Fabian Reinartz	bbe72dccb9	Remove dependency on etcd/pkg/fileutil	7 years ago
Goutham Veeramachaneni	c35d3a65bd	Add levels to all log lines. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	1e88ba06b4	Use boolean function instead of postings to drop WAL series ... There is not guarantee or requirement for WAL writers to only add series entries in increasing order of IDs. A postings list cannot look back and thus unordered WAL entries would skip over IDs to not truncate from the WAL. We replace it with a simple boolean check function that does not require order.	7 years ago
Fabian Reinartz	162a48e4f2	Create series with ID recorded in WAL when reading it back	7 years ago
Fabian Reinartz	7ada9cd805	Simplify series create logic in head	7 years ago
Fabian Reinartz	6892fc6dcb	Finish old WAL segment async, default to no fsync ... We were still fsyncing while holding the write lock when we cut a new segment. Given we cannot do anything but logging errors, we might just as well complete segments asynchronously. There's not realistic use case where one would fsync after every WAL entry, thus make the default of a flush interval of 0 to never fsync which is a much more likely use case.	7 years ago
Fabian Reinartz	1d5f85817d	Fix various races	7 years ago
Fabian Reinartz	c2916736be	Re-use slices on deocding the WAL. Fix tests.	7 years ago
Fabian Reinartz	0db4c227b7	Fix min/max time handling and concurrent crc32 usage	7 years ago
Fabian Reinartz	970bffec8d	Fix WAL errors and add tests for it	7 years ago
Fabian Reinartz	81222849bc	Filter WAL data in Head, misc fixes	7 years ago
Fabian Reinartz	33e9bdf403	WAL refactoring and truncation fixes and test	7 years ago
Fabian Reinartz	c36d574290	Replace single head lock with granular locks ... This adds various new locks to replace the single big lock on the head. All parts now must be COW as they may be held by clients after initial retrieval. Series by ID and hashes are now held in a stripe lock to reduce contention and total holding time during GC. This should reduce starvation of readers.	7 years ago
Fabian Reinartz	1ddedf2b30	Change series ID from uint32 to uint64	7 years ago
Goutham Veeramachaneni	1698c516ad	[WIP]: WAL implementation ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Fabian Reinartz	893b6ec506	Add tests for GC and chunk truncation	7 years ago
Fabian Reinartz	5cf2662074	Refactor WAL into Head and misc improvements	7 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads ... This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	7 years ago
Alin Sinpalean	8f7934d025	Take the fdatasync() syscall out from under mutex lock to (mostly) avoid scrape and eval from blocking on disk I/O.	7 years ago
Matt Layher	78b15c3434	Add newCRC32 function to simplify hash initialization	7 years ago
Goutham Veeramachaneni	7438ed7035	Expose Intervals type for use by TombstoneReader. ... TombstoneReader is exposed but Intervals is not. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Goutham Veeramachaneni	44e9ae38b5	Incorporate PR feedback. ... * Expose Stone as it is used in an exported method. * Move from tombstoneReader to []Stone for the same reason as above. * Make WAL reading a little cleaner. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	bacb143b7e	Simplify tombstone and WAL Delete formats. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	f29fb62fba	Make TombstoneReader a Getter. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	9bf7aa9af1	Misc. fixes incorporating feedback. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	3eb4119ab1	Make HeadBlock use WAL. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	39df7e2bba	Switch blocks to ULID directories, drop sequenc numbers	8 years ago
Fabian Reinartz	4862b261d0	Abstract WAL into interface	8 years ago
Fabian Reinartz	535532ca02	Export refdSample ... The type was part of a exported method signatures and should therefore be exported as well.	8 years ago
Julius Volz	8d1fb4fa01	Minor comment fixes and additions.	8 years ago
Fabian Reinartz	778103b450	Add liecence file and headers	8 years ago
Fabian Reinartz	10c7c9acbe	Adjust import names to new repository organisation	8 years ago
Fabian Reinartz	55ee4b5b3b	Merge branch 'master' of github.com:fabxc/tsdb	8 years ago
Fabian Reinartz	9c93f8f2aa	Fix various races ... This fixes different race condition encoutnered when running Prometheus. It reduces the overall performance in the synthetic benchmark a fair bit but has no indiciations of impacting a real-world setup notably.	8 years ago
Goutham Veeramachaneni	761e4768f3	Lint and Vet Fixes	8 years ago
Fabian Reinartz	ad5812d83a	Handle WAL corruption by truncating ... This adds handling for various corruption scenarios of the WAL. If corruption is encountered, we truncate the WAL after the last valid entry transparently and continue appending after the offset.	8 years ago
Fabian Reinartz	d3edfb5540	Test multiple full write/restore cylcles in WAL	8 years ago
Fabian Reinartz	cb4dde7659	Fix WAL log recovery bug ... This fixes a bug where the last WAL file was closed after consuming it instead of being left open for further writes. Reloading of blocks on startup considers loading head blocks now.	8 years ago
Fabian Reinartz	55a9b5428a	Add separate head mutex ... Introduce a seperate mutex for the head blocks to avoid a race where a post-compaction reload may run between switching the DB's base mutex to create a new head block in an appender.	8 years ago
Fabian Reinartz	ffb24a98f4	Add missing unlock, run debug endpoint in benchmark	8 years ago
Fabian Reinartz	f734773214	Replace per-file locking with single PID lock file ... File locks have a multitude of problems that make them hard to use correctly. As they are just advisory, they are only meaningful to prevent accidents like running the same process twice. A simple PID file lock works reliably in those cases and is simpler.	8 years ago
Fabian Reinartz	afa084920c	Use Castagnoli polynomial for CRC32s; cache them	8 years ago
Fabian Reinartz	9c7a88223e	Add full encode/decode WAL cycle test	8 years ago
Fabian Reinartz	2c97428a79	Add basic WAL tests	8 years ago
Fabian Reinartz	84e8027a8e	Close consumed WAL segments, verify checksums on read	8 years ago
Fabian Reinartz	eeb03e97e2	Add headers and format flag for WAL segments	8 years ago