We can instead write it as we go, and then go back and write in the
length at the end.
Also fix the compaction benchmark, which indicates no changes.
For the benchmark, this brings maximum memory usage of the buffers
from ~200kB down to 128B.
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
This adds support for a new query param on the new `api/v1/metadata`
endpoint that provides metadata for a specified metric via the V1 API.
It collapses metadata that is equal across all targets, and aggregates
under the same metric name the ones that differ.
Signed-off-by: gotjosh <josue@grafana.com>
* api: provide per metric metadata
This adds a new endpoint that provides per metric metadata via the V1 API.
It collapses metadata that is equal across all targets, and aggregates under the same metric name the ones that differ.
* Allow tests to be asserted on response length
Some tests e.g. limit on API responses, don't require an assertion on
equality.
This allows us to assert against response length instead of
equality.
Signed-off-by: gotjosh <josue@grafana.com>
* Allows sorting of responses from the API in tests
Fixes flaky test for api/v1/targets/metadata.
Allows sorting of responses from the API. For our tests to be deterministic, we need to ensure the response from the API follows an order. This structure allows us to define one.
Fixes#6431
Signed-off-by: gotjosh <josue@grafana.com>
The most common format (used by go, gcc and clang) for compiler error positions seems to be
`filename:line:char:` or `line:char:` if the filename is unknown.
This PR adapts the PromQL parser to use this convention.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
This PR exports the list of supported PromQL functions and their signatures.
The reason for that is that the PromQL language server likes to use that list.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
This commit introduces several test cases for the current /targets/metadata API endpoint.
To achieve so, we use a mock of the metadataStore and inject it to the targets under test.
Currently, three success cases are covered: with a metric name, with a target matcher, and with both. As for the failure scenario, the one where we couldn't match against a particular metric is covered.
Signed-off-by: gotjosh <josue@grafana.com>
To test the implementation of our metric metadata API, we need to represent various states of metadata in the scrape metadata store. That is currently not possible as the interface and method to set the store are private.
This changes the interface, list and get methods, and the SetMetadaStore function to be public.
Incidentally, the scrapeCache implementation needs to be renamed to match the new signature.
Signed-off-by: gotjosh <josue@grafana.com>
Previously, the struct `testTargetRetriever` had hardcoded active and dropped targets. This made it difficult to change the target information depending on the test case.
This change introduces a way to define them as arguments and pass it to a constructor for building. It lays a foundation for dynamically defining targets with various set of arguments to test different scenarios.
Signed-off-by: gotjosh <josue@grafana.com>
* Fix tsdb panic when querying corrupted chunks.
check that the chunk segment has enough data to read all chunk pieces.
* refactor, simplify and add tests.
* simpfiy WriteChunks implementation
Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>
* Add time units to storage.tsdb.retention.size flag
In an effort to reduce confusion with the `m` option of the
`ParseDuration()` function, this commit adds the available time units to
the `storage.tsdb.retention.time` flag to help showcase that there is no
option for months (which could be assumed to be `m`).
If someone were looking to set the retention to six months, they may
mistakenly do so with `6m`, which would reduce their retention to six
minutes.
Signed-off-by: Brooks Swinnerton <bswinnerton@gmail.com>
The Kubernetes client records workqueue duration and latency metrics as
seconds so there's no need to convert the values from microseconds to
seconds anymore.
The cache metrics (prometheus_sd_kubernetes_cache_*) are removed because
they aren't used anymore by the client though still exposed by its API.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
For sanity I'm not going to try to distinguish the things I'm maintainer
for because I'm the right person (e.g. template) vs things I'm the
maintainer for because someone has to be (e.g. pkg). Also add a general
warning to handle that this is more nuanced than it's worth trying to
capture, and relatedly always going to be out of date.
I'm also not giving full names/emails as we do elsewhere, as that'd
make things only more difficult to read for this particular repository.
I didn't put Bartek down for remote read, as particular functions of
particular files seems a a bit fine grained.
Fixes#4714
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
* move graph related files into own folder
Signed-off-by: blalov <boiskila@gmail.com>
* move graph helper functions into own file
Signed-off-by: blalov <boiskila@gmail.com>
* fix typo in file name
Signed-off-by: blalov <boiskila@gmail.com>
* fix typo in file name and lint fixes
Signed-off-by: blalov <boiskila@gmail.com>
* promql: Allow injecting fake tokens into the generated parser
Yacc grammars do not support having multiple start symbols.
To work around that restriction, it is possible to inject fake tokens into the lexer stream,
as described here https://www.gnu.org/software/bison/manual/html_node/Multiple-start_002dsymbols.html .
This is part of the parser rewrite effort described in #6256.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
For yacc generated parsers there is the convention to capitalize the names of item types provided by the lexer, which makes it easy to distinct lexer tokens (capitalized) from nonterminal symbols (not capitalized) in language grammars.
This convention is also followed by the (non generated) go compiler (see https://golang.org/pkg/go/token/#Token).
Part of the parser rewrite described in #6256.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
This is the first step towards a generated lexer as described in #6256.
It adds methods to the parser struct, that make it implement the yyLexer interface required by a yacc generated parser, as described here: https://godoc.org/golang.org/x/tools/cmd/goyacc .
The yyLexer interface is implemented by the parser struct instead of the lexer struct for the following reasons:
* Both parsers have a lookahead that the lexer does not know about. This solution makes it possible to synchronize these lookaheads when switching parsers.
* The routines to handle parser errors are not accessible to the lexer.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
It is possible that desired shards is always a bit higher than the
number of shards (less than 30%) and by exporting desired shards as the
raw number it will be easy to tell if a Prometheus is in that situation.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
When using both a label and the suffix+label in the
relabel config. It's possible that Prometheus remove
the suffx+label for no obvious reason. It's due to a
collision when merging labels from target and from
the sample.
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
It's currently causing weird unknown problems in React tests that run
successfully everywhere else, and by now all the tests that run on
Travis also run on CircleCI, so Travis is not needed anymore.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* promql: Clean up parser struct
The parser struct used two have two somewhat misused fields:
peekCount int
token [3]item
By reading the code carefully one notices, that peekCount always has the value 0 or 1 and that only the first element of token is ever accessed.
To make this clearer, this commit replaces the token array with a single variable and the peekCount int with a boolean.
Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>
* Update remote storage maintainers to Callum and Chris
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Chat feedback
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Björn Rabenstein