We know the max size of our map so we can create it with that information and avoid extra allocations
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* refactor: move from io/ioutil to io and os packages
* use fs.DirEntry instead of os.FileInfo after os.ReadDir
Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>
* labels.Equal benchmark for equal, not equal, and differing lengths
Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
* Compare equality of label.Label structs directly
Compare the structs using `==` instead of the name and value
of each label. This is functionally equivalent and about ~10%
faster in my testing.
Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
* Use longer more realistic names and values in benchmark
Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
This removes the dependancy on C leveldb and snappy.
It also takes care of fewer dependencies as they would
anyway not work on any non-Debian, non-Brew system.
Change-Id: Ia70dce1ba8a816a003587927e0b3a3f8ad2fd28c
If the metrics exported by a process already contain any of a target's
base labels (such as "job" or "instance", but also any manually assigned
target-group label), don't overwrite that label, but instead add a new
label consisting of the original label name prepended with "exporter_".
This is to accomodate intermediate exporter jobs, which might indicate
e.g. the jobs and instances for which they are exporting data.
This commit updates the documentation, Makefiles, formatting, and
code semantics to support the 1.1. runtime, which includes ...
1. ``make advice``,
2. ``make format``, and
3. ``go fix`` on various targets.
This commit extracts the model.Values truncation behavior into the actual
tiered storage, which uses it and behaves in a peculiar way—notably the
retention of previous elements if the chunk were to ever go empty. This is
done to enable interpolation between sparse sample values in the evaluation
cycle. Nothing necessarily new here—just an extraction.
Now, the model.Values TruncateBefore functionality would do what a user
would expect without any surprises, which is required for the
DeletionProcessor, which may decide to split a large chunk in two if it
determines that the chunk contains the cut-off time.
This commit reduces the general compile time dependencies to omit
the Protocol Buffer compiler and the Go Protocol Buffer generator
tool. The build steps to furnish them still remain, but they can
optionally be called if data.proto or config.proto are under work.
This commit employs explicit memory freeing for the in-memory storage
arenas. Secondarily, we take advantage of smaller channel buffer sizes
in the test.
This commit introduces to Prometheus a batch database sample curator,
which corroborates the high watermarks for sample series against the
curation watermark table to see whether a curator of a given type
needs to be run.
The curator is an abstract executor, which runs various curation
strategies across the database. It remarks the progress for each
type of curation processor that runs for a given sample series.
A curation procesor is responsible for effectuating the underlying
batch changes that are request. In this commit, we introduce the
CompactionProcessor, which takes several bits of runtime metadata and
combine sparse sample entries in the database together to form larger
groups. For instance, for a given series it would be possible to
have the curator effectuate the following grouping:
- Samples Older than Two Weeks: Grouped into Bunches of 10000
- Samples Older than One Week: Grouped into Bunches of 1000
- Samples Older than One Day: Grouped into Bunches of 100
- Samples Older than One Hour: Grouped into Bunches of 10
The benefits hereof of such a compaction are 1. a smaller search
space in the database keyspace, 2. better employment of compression
for repetious values, and 3. reduced seek times.
Primary changes:
* Strictly typed unmarshalling of metric values
* Schema types are contained by the processor (no "type entity002")
Minor changes:
* Added ProcessorFunc type for expressing processors as simple
functions.
* Added non-destructive `Merge` method to `model.LabelSet`
The Protocol Buffer compiler supports generating a machine-readable
descriptor file encoded as a provided Protocol Buffer message type,
which can be used to decode messages that have been encoded with it
after-the-fact. The generated descriptor also bundles in dependent
message types.
We can use this to perform forensics on old Prometheus clients, if
necessary.
Go's time.Time represents time as UTC in its fundamental data type.
That said, when using ``time.Unix(...)``, it sets the zone for the
time representation to the local. Unfortunately with diagnosis and
our tests, it is a PITA to jump between various zones, even though
the serialized version remains the same.
To keep things easy, all places where times are generated or read
are converted into UTC. These conversions are cheap, for
``Time.In`` merely changes a pointer reference in the struct,
nothing more. This enables me to diagnose test failures with fixture
data very easily.
The curator work can be done easier if dto.SampleKey is no longer
directly accessed but rather has a higher level type around it that
captures a certain modicum of business logic. This doesn't look
terribly interesting today, but it will get more so.
Łukasz Mierzwa