diff --git a/README.md b/README.md index 51e6e68b6..7b187c0bc 100644 --- a/README.md +++ b/README.md @@ -3,3 +3,5 @@ This repository contains the new Prometheus storage layer that will be used in its 2.0 release. A writeup of its design can be found [here](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/). + +See also the [format documentation](docs/format/README.md). diff --git a/docs/format/README.md b/docs/format/README.md new file mode 100644 index 000000000..33126f638 --- /dev/null +++ b/docs/format/README.md @@ -0,0 +1,5 @@ +## TSDB format + +* [Index](index.md) +* [Chunks](chunks.md) +* [Tombstones](tombstones.md) diff --git a/docs/format/index.md b/docs/format/index.md index 331c20e16..18600c835 100644 --- a/docs/format/index.md +++ b/docs/format/index.md @@ -54,7 +54,7 @@ Strings are referenced by sequential indexing. The strings are sorted in lexicog │ ├──────────────────────┴───────────────┤ │ │ │ . . . │ │ │ ├──────────────────────┬───────────────┤ │ -│ │ len(str_n) │ str_1 │ │ +│ │ len(str_n) │ str_n │ │ │ └──────────────────────┴───────────────┘ │ ├──────────────────────────────────────────┤ │ CRC32 <4b> │ @@ -119,8 +119,8 @@ After the labels, the number of indexed chunks is encoded, followed by a sequenc ### Label Index -A label index section indexes the existing (combined) values for one or more label names. -The `#names` field determines the number indexed label names, followed by the total number of entries in the `#entries` field. The body holds `#entries` symbol table reference tuples of length of length `#names`. The value tuples are sorted in lexicographically increasing order. +A label index section indexes the existing (combined) values for one or more label names. +The `#names` field determines the number of indexed label names, followed by the total number of entries in the `#entries` field. The body holds #entries / #names tuples of symbol table references, each tuple being of #names length. The value tuples are sorted in lexicographically increasing order. ``` ┌───────────────┬────────────────┬────────────────┐ @@ -139,11 +139,19 @@ The `#names` field determines the number indexed label names, followed by the to └─────────────────────────────────────────────────┘ ``` -The sequence of label index sections is finalized by an offset table pointing to the beginning of each label index section for a given set of label names. +For instance, a single label name with 4 different values will be encoded as: + +``` +┌────┬───┬───┬──────────────┬──────────────┬──────────────┬──────────────┬───────┐ +│ 24 │ 1 │ 4 │ ref(value_0) | ref(value_1) | ref(value_2) | ref(value_3) | CRC32 | +└────┴───┴───┴──────────────┴──────────────┴──────────────┴──────────────┴───────┘ +``` + +The sequence of label index sections is finalized by an [offset table](#offset-table) pointing to the beginning of each label index section for a given set of label names. ### Postings -Postings sections store monotonically increasing lists of series references that contain a given label pair associated with the list. +Postings sections store monotonically increasing lists of series references that contain a given label pair associated with the list. ``` ┌────────────────────┬────────────────────┐ @@ -161,7 +169,7 @@ Postings sections store monotonically increasing lists of series references that └─────────────────────────────────────────┘ ``` -The sequence of postings sections is finalized by an offset table pointing to the beginning of each postings section for a given set of label names. +The sequence of postings sections is finalized by an [offset table](#offset-table) pointing to the beginning of each postings section for a given set of label names. ### Offset Table diff --git a/encoding_helpers.go b/encoding_helpers.go index d0805fe07..ffb58b5c8 100644 --- a/encoding_helpers.go +++ b/encoding_helpers.go @@ -11,7 +11,7 @@ import ( var errInvalidSize = errors.New("invalid size") -// enbuf is a helper type to populate a byte slice with various types. +// encbuf is a helper type to populate a byte slice with various types. type encbuf struct { b []byte c [binary.MaxVarintLen64]byte diff --git a/querier.go b/querier.go index d4dad9306..b5b9ae050 100644 --- a/querier.go +++ b/querier.go @@ -117,7 +117,7 @@ func (q *querier) Close() error { return merr.Err() } -// NewBlockQuerier returns a queries against the readers. +// NewBlockQuerier returns a querier against the reader. func NewBlockQuerier(b BlockReader, mint, maxt int64) (Querier, error) { indexr, err := b.Index() if err != nil {