48abdcd83f
bugfix: allow quoted exemplar keys in openmetrics text format
...
Signed-off-by: Tristan Colgate-McFarlane <tristan@cerbos.dev>
2024-11-12 15:29:44 +00:00
8bcb4d865d
feat: normalize "le" and "quantile" labels values upon ingestion
...
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-18 17:37:29 +02:00
032ca9ef96
[PERF] textparse: further optimzations for OM `CreatedTimestamps` ( #15097 )
...
* feat: Added more tests; some changes/optimizations when pair-programming with Bartek.
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: use an efficient replacement to p.Metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: reduce mem allocs + comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use single quotes
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename to seriesHash
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: stronger checking
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: fmt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: pass pointer of buf into seriesHash()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-10 12:01:13 +01:00
f6e110d588
textparse: Refactored main testing utils for reusability; fixed proto Units. ( #15095 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 12:17:44 +01:00
8d281c3491
textparse: Refactored benchmark ( #15083 )
...
* textparse: Refactored benchmark.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Kill sinks, tested, nothing is inlined.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 09:55:26 +01:00
ab64966e9d
fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" ( #15094 )
...
* fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()"
---------
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-06 16:35:29 +00:00
47aeca9663
feat: naive fixes and optimzations for `CreatedTimestamp` function ( #14965 )
...
* enhance: wip ct parse optimizations
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further work on optimization
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further improvements and remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve optimizations and fix some CT parse errors
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check for LsetHash along with name
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup and documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* enhance: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: rename to resetCTParseValues
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post-merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add all possible reserved suffixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* test: separate CT values for each metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-04 13:11:02 +01:00
f1c57a95ed
change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug ( #14738 )
...
* chore: revert TypeRequiresCT to private
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: init NewOpenMetricsParser with skipCT true
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: allow opt-in to OM CT ingestion
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use textparse interface to set om options
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: set skipOMSeries in test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* wip: add tests for OM CR parse
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: merge ct tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add cases for OM text
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check correct test cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use both scrape protocols in config
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: fix inputs and output tests for OM
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipOMSeries to skipOMCTSeries
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* fix: finish refac
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move setup code outside test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: verify _created lines create new metric in certain cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* manager: Fixed CT OMText conversion bug; Refactored tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2024-10-02 11:52:03 +01:00
98cd80b2e2
feat: add microbenchmarks for OM CT parsing ( #14933 )
...
* test: benchmark OM CT parsing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move OM ct benchmark to promparse_test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: stricter comparison
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: use richer OM test data
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move parse-ct test outside of inner loop
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: separate benchmarks for om and prom parsers
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove more unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename to BenchmarkOMParseCreatedTimestamp
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-10-02 07:48:27 +01:00
02c465bf58
textparse: Implement `CreatedTimestamp()` in `openmetricsparse.go` ( #14356 )
...
* feat: initial implement of createedTimestamp() with tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: return ct after finding it
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: multiple changes
- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check if err from p.Next()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: advance parser and parse histograms + summary
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: restore previous tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: retore failing tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return nil when mtype doesn't match
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: update go fmt version
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document deepcopyparser
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: cover edgecase of `gauge_created` in CreatedTimestamp()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: dedeuplicate labeldiff checks
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for new label functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document CreatedTimestamp func
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: optimize `CreatedTimestamp()`
- Use refactored CreatedTimestamp function with bug fixes
- Remove unused code in labels.go
- Improve code documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add tests and lint fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove mName
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for CT parse failures and deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: edit expectCT struct
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add new label in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: use p.builder in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add NewMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: rename var
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add condition for OM fuzzing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: build tags
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: default skipCT to false
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipCT to skipCTSeries
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: formatting
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments and readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: remove NewOpenMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: extract skipCTSeries logic from parseMetricSuffix
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: inline create a NewOpenMetricsParser
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: improve error handling
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return error instead of nil
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: remove skipCT check from tBraceOpen
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* Pair programming with Manik, Arthur and Daniel.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: comments and use helper funcs
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-08-08 12:35:35 +01:00
ba6aa95222
refactor: metrics parsing: extract common checking code
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 16:37:41 +00:00
eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
93b72ec5dd
tsdb: create SymbolTables for labels as required
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
a28d7865ad
UTF-8: Add support for parsing UTF8 metric and label names
...
This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats.
This grammar will also be valid for non-UTF-8 names.
UTF-8 names will not be considered valid unless model.NameValidationScheme is changed.
This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/ .
Part of https://github.com/prometheus/prometheus/issues/13095
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-02-15 14:34:37 -05:00
17f48f2b3b
Tests: use replacement DeepEquals in more places
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-08 19:32:33 +00:00
8065bef172
Move metric type definitions to common/model
...
They are used in multiple repos, so common is a better place for them.
Several packages now don't depend on `model/textparse`, e.g.
`storage/remote`.
Also remove `metadata` struct from `api.go`, since it was identical to
a struct in the `metadata` package.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-19 18:56:54 +00:00
9c4782f1cc
golangci-lint: enable testifylint linter ( #13254 )
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
9225d118dc
Add Exemplars support for all time series
...
This change removes restrictions to allow adding exemplars
to all time series. It also contains some improvements in test values
so that it is easier to track what is tested.
The advantage of doing this is having a little less error-prone tests:
"yy" is not really descriptive but "counter-test" can give people
a better idea about what is tested so it is harder to make mistakes.
Closes gh-11982
Signed-off-by: Jonatan Ivanov <jonatan.ivanov@gmail.com>
2023-03-03 17:09:17 -08:00
0fee615719
model/textparse: improve openmetrics error outputs
...
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-22 17:19:01 +01:00
8d350d9e0c
Update package model/textparse tests for new labels.Labels type
...
We don't want to touch the result labels now we create them differently.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-12-19 15:22:09 +00:00
e934d0f011
Merge 'main' into sparsehistogram
...
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>
2022-10-05 22:14:49 +02:00
96d5a32659
Update go to 1.19, set min version to 1.18 ( #11279 )
...
* Update go to 1.19, set min version to 1.18
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
* Update golangci-lint
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-09-07 11:30:48 +02:00
28f028e938
Merge branch 'main' into sparsehistogram
2022-07-12 19:07:13 +02:00
c2b4de3611
refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages ( #10747 )
...
Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
2022-06-27 21:29:19 +02:00
7ee1836ef5
Merge branch 'main' into sparsehistogram
2022-04-05 18:31:19 +02:00
cd18da3605
Fix OpenMetrics parser to sort uppercase labels correctly. ( #10510 )
...
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-03-31 20:18:53 +02:00
4210aac74a
Merge branch 'main' into sparsehistogram
2022-03-22 14:47:42 +01:00
a9e2ffb717
Specify type in metadata parser errors ( #10269 )
2022-03-08 08:04:11 -05:00
5d4db805ac
Merge branch 'main' into sparsehistogram
2021-11-17 19:57:31 +01:00
c954cd9d1d
Move packages out of deprecated pkg directory
...
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
Tristan Colgate-McFarlane