eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
fb3eb21230
enable gocritic, unconvert and unused linters
...
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2023-04-13 19:20:22 +00:00
36eee11434
refactor (package cmd): move from github.com/pkg/errors to 'errors' and 'fmt' packages ( #10733 )
...
Signed-off-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
Co-authored-by: Matthieu MOREL <mmorel-35@users.noreply.github.com>
2022-05-24 16:58:59 +10:00
c954cd9d1d
Move packages out of deprecated pkg directory
...
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
b7bdf6fab2
Fix imports formatting
...
According to
2829908806 (r58457095)
2021-11-02 19:52:34 +01:00
1a6c2283a3
Format Go source files using 'gofumpt -w -s -extra'
...
Part of #9557
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
97b0738895
add --max-block-duration in promtool create-blocks-from rules ( #9511 )
...
* support maxBlockDuration for promtool tsdb create-blocks-from rules
Fixes #9465
Signed-off-by: Will Tran <will@autonomic.ai>
* don't hardcode 2h as the default block size in rules test
Signed-off-by: Will Tran <will@autonomic.ai>
2021-10-21 23:28:37 +02:00
fd6c852567
promtool: backfill: allow configuring block duration ( #8919 )
...
* promtool: backfill: allow configuring block duration
When backfilling large amounts of data across long periods of time, it
may in certain circumstances be useful to use a longer block duration to
increase the efficiency and speed of the backfilling process. This patch
adds a flag --block-duration-power to allow a user to choose the power N
where the block duration is 2^(N+1)h.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
* promtool: use sub-tests in backfill testing
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
* backfill: add messages to tests for clarity
When someone new breaks a test, seeing "expected: false, got: true" is
really not useful. A nice message helps here.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
* backfill: test long block durations
A test that uses a long block duration to write bigger blocks is added.
The check to make sure all blocks are the default duration is removed.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2021-06-29 14:53:38 +05:30
ba76bceb6b
Merge pull request #8917 from stevekuznetsov/skuznets/silence-backfill
...
promtool: backfill: allow silencing output
2021-06-14 23:27:18 +02:00
b5f6f8fb36
Switched to go-kit/log
...
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-11 12:28:36 -04:00
ee771a2a66
promtool: backfill: allow silencing output
...
When using the backfill command to add data to an ephemeral/test
Prometheus instance, it is not important to see which data was added as
it is often generated ahead of time and mostly irrelevant to the
use-case. The current approach prints information about each block that
is written, but does so in a generally inefficient and costly manner.
This patch adds a `--quiet` flag that allows a user to opt out of this
behavior.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2021-06-10 15:31:16 -07:00
7369561305
Combine Appender.Add and AddFast into a single Append method. ( #8489 )
...
This moves the label lookup into TSDB, whilst still keeping the cached-ref optimisation for repeated Appends.
This makes the API easier to consume and implement. In particular this change is motivated by the scrape-time-aggregation work, which I don't think is possible to implement without it as it needs access to label values.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-18 17:37:00 +05:30
c4536fa28c
Increase block writer size for backfilling
...
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-02-17 15:45:41 +05:30
bc9f9ee3aa
Backfilling: fast-path for non-consecutive blocks ( #8324 )
...
* Backfilling: optimize for non-consecutive blocks
When you have missing data for > 2 hours, you spend a lot of time
re-reading the complete file. It is not optimal.
This introduces a fastpath for this scenario.
Next, we do parse the metric even when we know we will not use it, based
on its timestamp. This only computes the metric when we know its
timestamp is right.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-30 02:06:41 +01:00
53480c168d
Backfill: print created blocks only, add human-readable option
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-23 20:42:30 +01:00
82b5f1d8b1
Backfill: Use mmap to reuse parser code
...
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 23:48:31 +01:00
b317b6ab9c
Backfill from OpenMetrics format ( #8084 )
...
* get parser working
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* import file created
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Find min and max ts
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* make two passes over file and write to tsdb
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* print error messages
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix Max and Min initializer
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Start with unit tests
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* reset file read
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* align blocks to two hour range
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Add cleanup test
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove .ds_store
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add license to import_test
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix Circle CI error
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Refactor code
Move backfill from tsdb to promtool directory
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix gitignore
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Remove panic
Rename ContenType
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* adjust mint
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix return statement
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix go modules
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Added unit test for backfill
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix CI error
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix file handling
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Close DB
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Close directory
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Error Handling
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* inline err
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix command line flags
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add spaces before func
fix pointers
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Add defer'd calls
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* move openmetrics.go content to backfill
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* changed args to flags
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add tests for wrong OM files
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Added additional tests
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Add comment to warn of func reuse
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Make input required in main.go
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* defer blockwriter close
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix defer
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* defer
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Remove contentType
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove defer from backfilltest
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix defer remove in backfill_test
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* changes to fix CI errors
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix go.mod
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* change package name
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* assert->require
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove todo
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix format
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix todo
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix createblock
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix tests
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix defer
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix return
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* check err for anon func
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* change comments
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* update comment
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix for the Flush Bug
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix formatting, comments, names
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Print Blocks
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* cleanup
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* refactor test to take care of multiple samples
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* refactor tests
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove om
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* I dont know what I fixed
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix tests
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fix tests, add test description, print blocks
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* commit after 5000 samples
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* reviews part 1
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Series Count
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix CI
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove extra func
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* make timestamp into sec
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Reviews 2
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Add Todo
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Fixes
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fixes reviews
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* =0
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove backfill.om
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add global err var, remove stuff
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* change var name
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* sampleLimit pass as parameter
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Add test when number of samples greater than batch size
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Change name of batchsize
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* revert export
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* nits
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* remove
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add comment, remove newline,consistent err
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Print Blocks
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* Modify comments
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* db.Querier
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* add sanity check , get maxt and mint
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* ci error
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* comment change
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* nits
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* NoError
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
* fix
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-11-26 10:37:06 +05:30
Bryan Boreham