prometheus

Commit Graph

Author	SHA1	Message	Date
Callum Styan	8fd73b1d28	Add Exemplar Remote Write support (#8296 ) * Write exemplars to the WAL and send them over remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update example for exemplars, print data in a more obvious format. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add metrics for remote write of exemplars. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix incorrect slices passed to send in remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * We need to unregister the new metrics. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments Signed-off-by: Callum Styan <callumstyan@gmail.com> * Order of exemplar append vs write exemplar to WAL needs to change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Several fixes to prevent sending uninitialized or incorrect samples with an exemplar. Fix dropping exemplar for missing series. Add tests for queue_manager sending exemplars Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Store both samples and exemplars in the same timeseries buffer to remove the alloc when building final request, keep sub-slices in separate buffers for re-use Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Condense sample/exemplar delivery tests to parameterized sub-tests Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Rename test methods for clarity now that they also handle exemplars Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Rename counter variable. Fix instances where metrics were not updated correctly Signed-off-by: Martin Disibio <mdisibio@gmail.com> * Add exemplars to LoadWAL benchmark Signed-off-by: Callum Styan <callumstyan@gmail.com> * last exemplars timestamp metric needs to convert value to seconds with ms precision Signed-off-by: Callum Styan <callumstyan@gmail.com> * Process exemplar records in a separate go routine when loading the WAL. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments related to clarifying comments and variable names. Also refactor sample/exemplar to enqueue prompb types. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Regenerate types proto with comments, update protoc version again. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Put remote write of exemplars behind a feature flag. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some of Ganesh's review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Move exemplar remote write feature flag to a config file field. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address Bartek's review comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Don't allocate exemplar buffers in queue_manager if we're not going to send exemplars over remote write. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add ValidateExemplar function, validate exemplars when appending to head and log them all to WAL before adding them to exemplar storage. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address more reivew comments from Ganesh. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Add exemplar total label length check. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address a few last review comments Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Martin Disibio <mdisibio@gmail.com>	4 years ago
Julien Pivotto	75e505babb	Upgrade to golang 1.16 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Ben Kochie	000d7edec9	Remove inline Go vendor dir Remove the inline Go vendoring per Prometheus 2020 Dev Summit[0]. [0]: https://docs.google.com/document/d/1yuaPKLDvhJNXMF1ubsOOm5kE2_6dvCxHowBQIDs0KdU/edit Signed-off-by: Ben Kochie <superq@gmail.com> Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	e208afcc95	web: Remove APIv2 (#7935 ) * web: Remove APIv2 Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	6f296594a8	Update go deps + use exclude directive (#7624 ) Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	4 years ago
Julien Pivotto	ef63d8d16d	Update vendors Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Simon Pasquier	81c4248081	*: bump gRPC and protobuf dependencies (#5367 ) The goal is to remove almost all references to the golang.org/x/net/context package. github.com/gogo/protobuf => v1.2.1 google.golang.org/grpc => v1.19.1 github.com/grpc-ecosystem/grpc-gateway => v1.18.5 It also replaces github.com/cockroachdb/cmux by github.com/soheilhy/cmux because of [1] which fixes #3909 incidentally. [1] https://github.com/grpc/grpc-go/issues/2636 Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Simon Pasquier	375ad1185c	: bump gRPC dependencies (#5075 ) : bump gRPC dependencies This change updates the gRPC dependencies to more recent versions: github.com/gogo/protobuf => v1.2.0 * github.com/grpc-ecosystem/grpc-gateway => v1.6.3 * google.golang.org/grpc => v1.17.0 In addition scripts/genproto.sh leverages Go modules information instead of hardcoding SHA1 commits. This ensures that the code is generated from the exact same sources. Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Run 'make proto' in CI Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Revert tabs -> spaces change Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Fix 'make proto' step Signed-off-by: Simon Pasquier <spasquie@redhat.com> * 'go get' grpc/protobuf dependencies Signed-off-by: Simon Pasquier <spasquie@redhat.com> * Prepopulate cache with go mod download Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Bartek Plotka	c3434a39b7	proto: Allow reproducible proto generation. Current way of setting prometheus project to generate repo is tedious. If you just go install packages described in README.md it ends up with those (non buildable code) ``` ../../../prompb/remote.pb.go:61:34: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:109:33: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:164:34: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:214:27: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:284:33: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ``` The issue is that the guide presented in proto folder is asking to install some GO packages. It does not mention from what version and there is strong mismatch between those used to generate proto and those that are vendored and used to run the code later... The particular outdated packages in vendor are: `github.com/gogo/protobuf` and `google.golang.org/grpc`. Just updating vendor would be not enough, as it would only tmp fix. I propose adding script that will parse version from vendor.json and install correct version. Changes: - Added script that uses packages in exactly the same version the Go code is build against later. - Regenerated proto with those pinned deps. - Updated REAMDE.md - Added `make proto`Current way of setting prometheus project to generate code from proto files is tedious. If you just go install packages described in README.md, it ends up with this (non buildable code) ``` ../../../prompb/remote.pb.go:61:34: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:109:33: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:164:34: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:214:27: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ../../../prompb/remote.pb.go:284:33: undefined: "github.com/prometheus/prometheus/vendor/github.com/gogo/protobuf/proto".InternalMessageInfo ``` The issue is that the guide presented in proto folder is asking to install some GO packages. It does not mention from what version and there is strong mismatch between those used to generate proto and those that are vendored and used to run the code later... The particular outdated packages in vendor are: `github.com/gogo/protobuf` and `google.golang.org/grpc`. Just updating vendor would be not enough, as it would only tmp fix. I propose adding script that will parse version from vendor.json and install correct version. Changes: - Added script that uses packages in exactly the same version the Go code is build against later. - Regenerated proto with those pinned deps. - Updated REAMDE.md - Added `make proto` Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	6 years ago
Krasi Georgiev	99e717ad18	revert ot using the gogofast plugin and regenerate grpc server	7 years ago
Krasi Georgiev	a1229465b2	genproto.sh - update protoc to 3.5, gogofast is now gofast	7 years ago
Goutham Veeramachaneni	3de10e3b44	Add CleanTombstones API endpoint Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	7 years ago
Krasi Georgiev	5d8f93a22a	now using only github.com/gogo/protobuf bumped all grpc-gateway packages to v1.2.2 updated and run the denproto.sh script	7 years ago
Fabian Reinartz	ccf9e62972	*: add admin grpc API	7 years ago

14 Commits (ba76bceb6b09eaa4fa1f9744ac164d684e348886)