8bcb4d865d
feat: normalize "le" and "quantile" labels values upon ingestion
...
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-18 17:37:29 +02:00
f6e110d588
textparse: Refactored main testing utils for reusability; fixed proto Units. ( #15095 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 12:17:44 +01:00
8d281c3491
textparse: Refactored benchmark ( #15083 )
...
* textparse: Refactored benchmark.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Kill sinks, tested, nothing is inlined.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 09:55:26 +01:00
ab64966e9d
fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" ( #15094 )
...
* fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()"
---------
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-06 16:35:29 +00:00
98cd80b2e2
feat: add microbenchmarks for OM CT parsing ( #14933 )
...
* test: benchmark OM CT parsing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move OM ct benchmark to promparse_test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: stricter comparison
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: use richer OM test data
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move parse-ct test outside of inner loop
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: separate benchmarks for om and prom parsers
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove more unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename to BenchmarkOMParseCreatedTimestamp
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-10-02 07:48:27 +01:00
315165e49d
refac: make typeRequiresCT private
...
Signed-off-by: Manik Rana <manikrana54@gmail.com>
2024-09-20 21:56:19 +05:30
50cd453c8f
chore: Fix typos ( #14868 )
...
* Fix typos
---------
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-10 22:32:03 +02:00
02c465bf58
textparse: Implement `CreatedTimestamp()` in `openmetricsparse.go` ( #14356 )
...
* feat: initial implement of createedTimestamp() with tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: return ct after finding it
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: multiple changes
- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check if err from p.Next()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: advance parser and parse histograms + summary
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: restore previous tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: retore failing tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return nil when mtype doesn't match
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: update go fmt version
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document deepcopyparser
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: cover edgecase of `gauge_created` in CreatedTimestamp()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: dedeuplicate labeldiff checks
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for new label functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document CreatedTimestamp func
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: optimize `CreatedTimestamp()`
- Use refactored CreatedTimestamp function with bug fixes
- Remove unused code in labels.go
- Improve code documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add tests and lint fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove mName
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for CT parse failures and deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: edit expectCT struct
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add new label in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: use p.builder in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add NewMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: rename var
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add condition for OM fuzzing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: build tags
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: default skipCT to false
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipCT to skipCTSeries
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: formatting
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments and readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: remove NewOpenMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: extract skipCTSeries logic from parseMetricSuffix
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: inline create a NewOpenMetricsParser
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: improve error handling
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return error instead of nil
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: remove skipCT check from tBraceOpen
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* Pair programming with Manik, Arthur and Daniel.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: comments and use helper funcs
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-08-08 12:35:35 +01:00
07355c9199
Bump client_golang to 1.19
...
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
2024-03-06 09:11:13 -03:00
ba6aa95222
refactor: metrics parsing: extract common checking code
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 16:37:41 +00:00
eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
93b72ec5dd
tsdb: create SymbolTables for labels as required
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
5597020a60
Use github.com/klauspost/compress for gzip and zlib
...
klauspost/compress is a high quality drop-in replacement for common Go
compression libraries. Since Prometheus sends out a lot of HTTP requests
that often return compressed output having improved compression
libraries helps to save cpu & memory resources.
On a test Prometheus server I was able to see cpu reduction from 31 to
30 cores.
Benchmark results:
name old time/op new time/op delta
TargetScraperGzip/metrics=1-8 69.4µs ± 4% 69.2µs ± 3% ~ (p=0.122 n=50+50)
TargetScraperGzip/metrics=100-8 84.3µs ± 2% 80.9µs ± 2% -4.02% (p=0.000 n=48+46)
TargetScraperGzip/metrics=1000-8 296µs ± 1% 274µs ±14% -7.35% (p=0.000 n=47+45)
TargetScraperGzip/metrics=10000-8 2.06ms ± 1% 1.66ms ± 2% -19.34% (p=0.000 n=47+45)
TargetScraperGzip/metrics=100000-8 20.9ms ± 2% 17.5ms ± 3% -16.50% (p=0.000 n=49+50)
name old alloc/op new alloc/op delta
TargetScraperGzip/metrics=1-8 6.06kB ± 0% 6.07kB ± 0% +0.24% (p=0.000 n=48+48)
TargetScraperGzip/metrics=100-8 7.04kB ± 0% 6.89kB ± 0% -2.17% (p=0.000 n=49+50)
TargetScraperGzip/metrics=1000-8 9.02kB ± 0% 8.35kB ± 1% -7.49% (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8 18.1kB ± 1% 16.1kB ± 2% -10.87% (p=0.000 n=47+47)
TargetScraperGzip/metrics=100000-8 1.21MB ± 0% 1.01MB ± 2% -16.69% (p=0.000 n=36+50)
name old allocs/op new allocs/op delta
TargetScraperGzip/metrics=1-8 71.0 ± 0% 72.0 ± 0% +1.41% (p=0.000 n=50+50)
TargetScraperGzip/metrics=100-8 81.0 ± 0% 76.0 ± 0% -6.17% (p=0.000 n=50+50)
TargetScraperGzip/metrics=1000-8 92.0 ± 0% 83.0 ± 0% -9.78% (p=0.000 n=50+50)
TargetScraperGzip/metrics=10000-8 93.0 ± 0% 91.0 ± 0% -2.15% (p=0.000 n=50+50)
TargetScraperGzip/metrics=100000-8 111 ± 0% 135 ± 1% +21.89% (p=0.000 n=40+50)
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
2024-02-22 17:08:15 +00:00
a28d7865ad
UTF-8: Add support for parsing UTF8 metric and label names
...
This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats.
This grammar will also be valid for non-UTF-8 names.
UTF-8 names will not be considered valid unless model.NameValidationScheme is changed.
This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/ .
Part of https://github.com/prometheus/prometheus/issues/13095
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-02-15 14:34:37 -05:00
17f48f2b3b
Tests: use replacement DeepEquals in more places
...
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-08 19:32:33 +00:00
8065bef172
Move metric type definitions to common/model
...
They are used in multiple repos, so common is a better place for them.
Several packages now don't depend on `model/textparse`, e.g.
`storage/remote`.
Also remove `metadata` struct from `api.go`, since it was identical to
a struct in the `metadata` package.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-19 18:56:54 +00:00
9c4782f1cc
golangci-lint: enable testifylint linter ( #13254 )
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-12-07 11:35:01 +00:00
fb3eb21230
enable gocritic, unconvert and unused linters
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-13 19:20:22 +00:00
e3b5416d05
model/textparse: improve error outputs
...
Parsing errors in the Prometheus HTTP format parser are very hard to
investigate since they only approximately indicate what is going wrong
in the parser and don't provide any information about the incorrect
input. As such it is very hard to tell what is wrong in the format
exposed by the application.
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-03 16:54:57 +01:00
96b9d8cebb
model/textparse: associate correct token to errors
...
In some cases, the Prometheus HTTP format parser was not returning the
right token in the error output which made debugging impossible.
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-01-02 13:57:27 +01:00
8d350d9e0c
Update package model/textparse tests for new labels.Labels type
...
We don't want to touch the result labels now we create them differently.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-12-19 15:22:09 +00:00
8120af22e2
benchmarks: SetBytes takes bytes per operation
...
Where the code was multiplying bytes by number of operations, this
resulted in absurdly high throughput numbers.
Also, in `BenchmarkParse()`, don't run the `expfmt` case twice.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-10-02 22:11:33 +02:00
96d5a32659
Update go to 1.19, set min version to 1.18 ( #11279 )
...
* Update go to 1.19, set min version to 1.18
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
* Update golangci-lint
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
2022-09-07 11:30:48 +02:00
c2b4de3611
refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages ( #10747 )
...
Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
2022-06-27 21:29:19 +02:00
e2ede285a2
refactor: move from io/ioutil to io and os packages ( #10528 )
...
* refactor: move from io/ioutil to io and os packages
* use fs.DirEntry instead of os.FileInfo after os.ReadDir
Signed-off-by: MOREL Matthieu <matthieu.morel@cnp.fr>
2022-04-27 11:24:36 +02:00
a9e2ffb717
Specify type in metadata parser errors ( #10269 )
2022-03-08 08:04:11 -05:00
c954cd9d1d
Move packages out of deprecated pkg directory
...
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
machine424