4f48e76086
chore: fix some function names in comment
...
Signed-off-by: huochexizhan <huochexizhan@outlook.com>
2024-11-19 12:02:10 +08:00
2182b83271
feat(nhcb): implement created timestamp handling ( #15198 )
...
Call through to the underlaying parser if we are not in a histogram
and the entry is a series or exponential native histogram. Otherwise store
and retrieve CT for NHCB.
* fix(omparser): losing exemplars when CT is parsed
Fixes : #15137
Ignore exemplars while peeking ahead during CT parsing.
Simplify state reset with defer().
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-24 07:38:58 +02:00
d8c1605930
Merge pull request #15164 from machine424/quantile
...
feat: normalize "le" and "quantile" labels values upon ingestion
2024-10-19 21:13:03 +02:00
8bcb4d865d
feat: normalize "le" and "quantile" labels values upon ingestion
...
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-18 17:37:29 +02:00
c3b8582cf4
[PERF] textparse: optimize OM `p.isCreatedSeries()` ( #15150 )
...
* refac: remove p.Metric usage
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* perf: use bytes instead of strings
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-14 10:18:15 +01:00
032ca9ef96
[PERF] textparse: further optimzations for OM `CreatedTimestamps` ( #15097 )
...
* feat: Added more tests; some changes/optimizations when pair-programming with Bartek.
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: use an efficient replacement to p.Metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: reduce mem allocs + comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use single quotes
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename to seriesHash
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: stronger checking
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: fmt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: pass pointer of buf into seriesHash()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-10 12:01:13 +01:00
47aeca9663
feat: naive fixes and optimzations for `CreatedTimestamp` function ( #14965 )
...
* enhance: wip ct parse optimizations
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further work on optimization
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further improvements and remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve optimizations and fix some CT parse errors
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check for LsetHash along with name
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup and documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* enhance: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: rename to resetCTParseValues
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post-merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add all possible reserved suffixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* test: separate CT values for each metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-04 13:11:02 +01:00
f1c57a95ed
change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug ( #14738 )
...
* chore: revert TypeRequiresCT to private
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: init NewOpenMetricsParser with skipCT true
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: allow opt-in to OM CT ingestion
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use textparse interface to set om options
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: set skipOMSeries in test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* wip: add tests for OM CR parse
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: merge ct tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add cases for OM text
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check correct test cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use both scrape protocols in config
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: fix inputs and output tests for OM
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipOMSeries to skipOMCTSeries
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* fix: finish refac
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move setup code outside test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: verify _created lines create new metric in certain cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* manager: Fixed CT OMText conversion bug; Refactored tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2024-10-02 11:52:03 +01:00
315165e49d
refac: make typeRequiresCT private
...
Signed-off-by: Manik Rana <manikrana54@gmail.com>
2024-09-20 21:56:19 +05:30
02c465bf58
textparse: Implement `CreatedTimestamp()` in `openmetricsparse.go` ( #14356 )
...
* feat: initial implement of createedTimestamp() with tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: return ct after finding it
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: multiple changes
- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check if err from p.Next()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: advance parser and parse histograms + summary
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: restore previous tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: retore failing tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unneeded comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return nil when mtype doesn't match
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: update go fmt version
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document deepcopyparser
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: cover edgecase of `gauge_created` in CreatedTimestamp()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: dedeuplicate labeldiff checks
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for new label functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: document CreatedTimestamp func
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: optimize `CreatedTimestamp()`
- Use refactored CreatedTimestamp function with bug fixes
- Remove unused code in labels.go
- Improve code documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add tests and lint fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove mName
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add tests for CT parse failures and deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: edit expectCT struct
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: add new label in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: use p.builder in deepCopy
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add NewMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: comments
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: rename var
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add condition for OM fuzzing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: build tags
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: default skipCT to false
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipCT to skipCTSeries
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: formatting
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments and readability updates
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: remove NewOpenMetricsParserWithOpts
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: extract skipCTSeries logic from parseMetricSuffix
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: inline create a NewOpenMetricsParser
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: improve error handling
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: return error instead of nil
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: remove skipCT check from tBraceOpen
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* Pair programming with Manik, Arthur and Daniel.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: comments and use helper funcs
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-08-08 12:35:35 +01:00
2e30f1231b
docs: textparse.Parser return type mismatch
...
The docs suggest the Next method returns a bool, but that's not the case (`Entry` is an int).
```
// Next advances the parser to the next sample. It returns false if no
// more samples were read or an error occurred.
Next() (Entry, error)
```
The docs were first added in d80a3de23576a4a46cb05de2df752f
2024-02-29 16:45:05 +01:00
eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
a28d7865ad
UTF-8: Add support for parsing UTF8 metric and label names
...
This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats.
This grammar will also be valid for non-UTF-8 names.
UTF-8 names will not be considered valid unless model.NameValidationScheme is changed.
This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/ .
Part of https://github.com/prometheus/prometheus/issues/13095
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-02-15 14:34:37 -05:00
3db4596965
Merge pull request #13299 from vesari/add-unit-parser
...
protobuf: add unit parser
2024-01-09 14:47:50 +01:00
c83e1fc574
textparse: remove MetricType alias
...
No backwards-compatibility; make a clean break.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-19 18:56:54 +00:00
8065bef172
Move metric type definitions to common/model
...
They are used in multiple repos, so common is a better place for them.
Several packages now don't depend on `model/textparse`, e.g.
`storage/remote`.
Also remove `metadata` struct from `api.go`, since it was identical to
a struct in the `metadata` package.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2023-12-19 18:56:54 +00:00
3d6a92a25e
Merge remote-tracking branch 'prometheus/main' into add-unit-parser
2023-12-17 14:11:55 +01:00
5d5acf5744
Add unit protobuf parser
...
Signed-off-by: Arianna Vespri <arianna.vespri@yahoo.it>
2023-12-12 12:40:08 +01:00
5082655392
Append Created Timestamps ( #12733 )
...
* Append created timestamps.
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Log when created timestamps are ignored
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Proposed changes to Append CT PR.
Changes:
* Changed textparse Parser interface for consistency and robustness.
* Changed CT interface to be more explicit and handle validation.
* Simplified test, change scrapeManager to allow testability.
* Added TODOs.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Updates.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Refactor head_appender test
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Fix linter issues
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Use model.Sample in head appender test
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
---------
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2023-12-11 08:43:42 +00:00
ef8e6ae780
Parse created timestamps from Prometheus Protobuf ( #12973 )
...
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
2023-10-18 19:04:02 +01:00
0a513f827d
ci(lint): enable errorlint linter on model
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-09-29 21:11:33 +00:00
0e3f35324b
scrape: Enable ingestion of multiple exemplars per sample
...
This has become a requirement for native histograms, as a single
histogram sample commonly has many buckets, so that providing many
exemplars makes sense.
Since OM text doesn't support native histograms yet, the test had to
be expanded to also support protobuf test cases.
Signed-off-by: beorn7 <beorn@grafana.com>
2023-07-13 14:16:10 +02:00
9225d118dc
Add Exemplars support for all time series
...
This change removes restrictions to allow adding exemplars
to all time series. It also contains some improvements in test values
so that it is easier to track what is tested.
The advantage of doing this is having a little less error-prone tests:
"yy" is not really descriptive but "counter-test" can give people
a better idea about what is tested so it is harder to make mistakes.
Closes gh-11982
Signed-off-by: Jonatan Ivanov <jonatan.ivanov@gmail.com>
2023-03-03 17:09:17 -08:00
0fee615719
model/textparse: improve openmetrics error outputs
...
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-22 17:19:01 +01:00
e3b5416d05
model/textparse: improve error outputs
...
Parsing errors in the Prometheus HTTP format parser are very hard to
investigate since they only approximately indicate what is going wrong
in the parser and don't provide any information about the incorrect
input. As such it is very hard to tell what is wrong in the format
exposed by the application.
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
2023-02-03 16:54:57 +01:00
3e5ad99c33
textparse: Minor doc comment improvement
...
Signed-off-by: beorn7 <beorn@grafana.com>
2023-01-11 00:30:55 +01:00
1f04899ae3
Update package model/textparse for new labels.Labels type
...
Parse metrics using labels.ScratchBuilder, so we reduce assumptions about
internals of Labels.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2022-12-19 15:22:09 +00:00
f7df3b86ba
histograms: parse float histograms from proto definition ( #11149 )
...
* histograms: parse float histograms from proto definition
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Improve comment
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Ignore float buckets
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Refactor Histogram() function
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Fix test_float_histogram
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
* Update model/textparse/protobufparse.go
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Marc Tudurí <marctc@protonmail.com>
* Update protobufparse.go
Signed-off-by: Marc Tudurí <marctc@protonmail.com>
* Update scrape.go
Signed-off-by: Marc Tudurí <marctc@protonmail.com>
* Update scrape/scrape.go
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Marc Tudurí <marctc@protonmail.com>
Signed-off-by: Marc Tuduri <marctc@protonmail.com>
Signed-off-by: Marc Tudurí <marctc@protonmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2022-08-25 20:37:41 +05:30
28f028e938
Merge branch 'main' into sparsehistogram
2022-07-12 19:07:13 +02:00
c2b4de3611
refactor (package model): move from github.com/pkg/errors to 'errors' and 'fmt' packages ( #10747 )
...
Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
Co-Authored-By: Julien Pivotto <roidelapluie@gmail.com>
Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
2022-06-27 21:29:19 +02:00
7ee1836ef5
Merge branch 'main' into sparsehistogram
2022-04-05 18:31:19 +02:00
cd18da3605
Fix OpenMetrics parser to sort uppercase labels correctly. ( #10510 )
...
Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-03-31 20:18:53 +02:00
4210aac74a
Merge branch 'main' into sparsehistogram
2022-03-22 14:47:42 +01:00
a9e2ffb717
Specify type in metadata parser errors ( #10269 )
2022-03-08 08:04:11 -05:00
5d4db805ac
Merge branch 'main' into sparsehistogram
2021-11-17 19:57:31 +01:00
c954cd9d1d
Move packages out of deprecated pkg directory
...
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
huochexizhan