469573b13b
fix(nhcb): do not return nhcb from parse if exponential is present ( #15209 )
...
From: https://github.com/prometheus/prometheus/pull/14978#discussion_r1800755481
Also encode the requirement table set in #13532
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-24 18:14:05 +02:00
2182b83271
feat(nhcb): implement created timestamp handling ( #15198 )
...
Call through to the underlaying parser if we are not in a histogram
and the entry is a series or exponential native histogram. Otherwise store
and retrieve CT for NHCB.
* fix(omparser): losing exemplars when CT is parsed
Fixes : #15137
Ignore exemplars while peeking ahead during CT parsing.
Simplify state reset with defer().
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-24 07:38:58 +02:00
aa81210c8b
NHCB scrape: refactor state handling and speed up scrape test ( #15193 )
...
* NHCB: scrape use state field and not booleans
From comment https://github.com/prometheus/prometheus/pull/14978#discussion_r1800898724
Also make compareLabels read only and move storeLabels to the first
processed classic histogram series.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* Speed up TestConvertClassicHistogramsToNHCB 3x
Reduce the startup time and timeouts
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* lint fix
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
---------
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-22 17:49:25 +01:00
ad4857de52
Merge pull request #14978 from prometheus/nhcb-scrape-impl
...
feat: NHCB: convert classic histograms to nhcb in scrape MVP
2024-10-22 07:55:58 +02:00
70e2d23027
Merge pull request #11474 from clwluvw/group-label
...
[FEATURE] rules: add labels at group level
2024-10-21 14:47:12 +01:00
25ef4d3483
benchmark, rename parser omtext_with_nhcb
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 15:40:48 +02:00
bee1eb7720
goimports run
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 14:02:32 +02:00
555bd6292a
Better docstring on test
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 13:48:21 +02:00
a6947e1e6d
Remove omcounterdata.txt as redundant
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 13:45:33 +02:00
eaee6bacc7
Fix failing benchmarks
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 13:40:16 +02:00
5ee0980cd1
Add unit test to show that current wrapper is sub-optimal
...
https://github.com/prometheus/prometheus/pull/14978#discussion_r1800755481
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 13:35:33 +02:00
482bb453c6
Followup to #15164
...
Update test cases
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 11:03:07 +02:00
8c1b5a6251
Merge branch 'main' into nhcb-scrape-impl
2024-10-21 11:00:41 +02:00
d8c1605930
Merge pull request #15164 from machine424/quantile
...
feat: normalize "le" and "quantile" labels values upon ingestion
2024-10-19 21:13:03 +02:00
8bcb4d865d
feat: normalize "le" and "quantile" labels values upon ingestion
...
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-18 17:37:29 +02:00
421a3c22ea
scrape: provide a fallback format ( #15136 )
...
scrape: Remove implicit fallback to the Prometheus text format
Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config.
---------
Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>
Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-10-18 17:12:31 +02:00
18b81ad79d
feat: ProtobufParse.formatOpenMetricsFloat: improve float formatting by using strconv.AppendFloat instead of fmt.Sprint
...
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-18 15:19:55 +02:00
5ee698de2c
Apply review comments
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-17 12:55:45 +02:00
04b827dd77
Merge branch 'main' into nhcb-scrape-impl
2024-10-17 12:34:44 +02:00
d4b1f9eb33
Corrects the behaviour of binary opperators between histogram and float ( #14726 )
...
promql: corrects binary operators functioning for mixed sample with histogram and float
For invalid pairings of sample types, an annotation is added now.
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
---------
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-10-15 14:44:36 +02:00
d930648afb
Add doc string for NHCBParser
...
Very high level since we'll do a lot of optimizations so not
worth going into details at this time.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 14:35:11 +02:00
c13585665d
Merge branch 'main' into nhcb-scrape-impl
...
# Conflicts:
# promql/promqltest/test.go
# util/convertnhcb/convertnhcb.go
2024-10-14 14:26:11 +02:00
c3b8582cf4
[PERF] textparse: optimize OM `p.isCreatedSeries()` ( #15150 )
...
* refac: remove p.Metric usage
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* perf: use bytes instead of strings
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: comments
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-14 10:18:15 +01:00
a1700aab3a
Apply suggestions from code review
...
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-14 11:13:58 +02:00
0a40a09da5
Use const instead of -53
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 11:09:03 +02:00
032ca9ef96
[PERF] textparse: further optimzations for OM `CreatedTimestamps` ( #15097 )
...
* feat: Added more tests; some changes/optimizations when pair-programming with Bartek.
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: use an efficient replacement to p.Metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: reduce mem allocs + comments
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use single quotes
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: rename to seriesHash
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* refac: switch condition order
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: stronger checking
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* chore: fmt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: pass pointer of buf into seriesHash()
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-10 12:01:13 +01:00
9b5d7287bb
Use labels hash to determine change in metric like CT
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 15:16:46 +02:00
14f92319d9
Add basic benchmark cases for NHCB over OM
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 15:16:27 +02:00
530e9514b7
Fix case of keeping classic series
...
Make it more obvious that the code in if had side effect.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 13:08:54 +02:00
8dfa733596
Fix labels handling with dedupelabels tag
...
Use scratch builder.
Use hash compare instead of compare by label.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 12:29:59 +02:00
f3c8ed2947
minor fixes
...
Fix lint errors.
No need to pre-allocate exemplars.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 11:45:31 +02:00
fbbf10baad
Use the proper way to iterate labels
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 11:24:24 +02:00
7fccf1e6be
Disable CT handling and enable exemplar handling
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-09 11:08:08 +02:00
e931587bf8
Factor out label compare and store
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-08 10:12:25 +02:00
16f28be713
Fix CT handling
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-07 16:59:07 +02:00
6bebeaf41b
Fix not checking all labels before deciding to store NHCB
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-07 15:41:54 +02:00
2a3aa500e9
Make nhcb unit test pass with many exceptions marked as TODOs
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-07 14:27:23 +02:00
e0444d9384
Merge branch 'main' into nhcb-scrape-impl
2024-10-07 14:02:46 +02:00
e1a7008b6c
Add unit test nhcbparse_test.go
...
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-07 14:02:10 +02:00
f6e110d588
textparse: Refactored main testing utils for reusability; fixed proto Units. ( #15095 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 12:17:44 +01:00
9c4816df36
Rename bType to bName as Type returns the binary name and not the type name
...
See Parser.Type() function.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-07 11:31:43 +02:00
ed2e7dc258
Merge branch 'main' into nhcb-scrape-impl
...
# Conflicts:
# scrape/scrape.go
2024-10-07 11:23:44 +02:00
8d281c3491
textparse: Refactored benchmark ( #15083 )
...
* textparse: Refactored benchmark.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Kill sinks, tested, nothing is inlined.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 09:55:26 +01:00
ab64966e9d
fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" ( #15094 )
...
* fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()"
---------
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-06 16:35:29 +00:00
47aeca9663
feat: naive fixes and optimzations for `CreatedTimestamp` function ( #14965 )
...
* enhance: wip ct parse optimizations
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further work on optimization
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: further improvements and remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve optimizations and fix some CT parse errors
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check for LsetHash along with name
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup and documentation
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* enhance: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: improve comments and add cleaner functions
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: rename to resetCTParseValues
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post-merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: add all possible reserved suffixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* test: separate CT values for each metric
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-04 13:11:02 +01:00
f1c57a95ed
change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug ( #14738 )
...
* chore: revert TypeRequiresCT to private
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: init NewOpenMetricsParser with skipCT true
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: allow opt-in to OM CT ingestion
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use textparse interface to set om options
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: set skipOMSeries in test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* wip: add tests for OM CR parse
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: merge ct tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add cases for OM text
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check correct test cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use both scrape protocols in config
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: fix inputs and output tests for OM
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipOMSeries to skipOMCTSeries
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* fix: finish refac
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move setup code outside test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: verify _created lines create new metric in certain cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* manager: Fixed CT OMText conversion bug; Refactored tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2024-10-02 11:52:03 +01:00
98cd80b2e2
feat: add microbenchmarks for OM CT parsing ( #14933 )
...
* test: benchmark OM CT parsing
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move OM ct benchmark to promparse_test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: stricter comparison
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* feat: use richer OM test data
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move parse-ct test outside of inner loop
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: separate benchmarks for om and prom parsers
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: remove more unused code
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename to BenchmarkOMParseCreatedTimestamp
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-10-02 07:48:27 +01:00
71fd2d93a9
Merge branch 'main' into nhcb-scrape-impl
...
# Conflicts:
# config/config.go
# scrape/scrape.go
2024-09-25 13:43:57 +02:00
97ba2fc39d
use caps for NHCB
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-09-25 13:38:30 +02:00
f35c6649e4
don't blindly convert series with the classic histogram name suffixes if they are not actually histograms based on metadata
...
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-09-25 13:38:15 +02:00
George Krajcsovits