e01c5cefac
Previously, prometheus_notifications_errors_total was incremented by one whenever a batch of alerts was affected by an error during sending to a specific alertmanager. However, the corresponding metric prometheus_notifications_sent_total, counting all alerts that were sent (including those where the sent ended in error), is incremented by the batch size, i.e. the number of alerts. Therefore, the ratio used in the mixin for the PrometheusErrorSendingAlertsToSomeAlertmanagers alert is inconsistent. This commit changes the increment of prometheus_notifications_errors_total to the number of alerts that were sent in the attempt that ended in an error. It also adjusts the metrics help string accordingly and makes the wording in the alert in the mixin more precise. Signed-off-by: beorn7 <beorn@grafana.com> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| Makefile | ||
| README.md | ||
| alerts.jsonnet | ||
| alerts.libsonnet | ||
| config.libsonnet | ||
| dashboards.jsonnet | ||
| dashboards.libsonnet | ||
| jsonnetfile.json | ||
| mixin.libsonnet | ||
README.md
Prometheus Mixin
This is work in progress. We aim for it to become a good role model for alerts and dashboards eventually, but it is not quite there yet.
The Prometheus Mixin is a set of configurable, reusable, and extensible alerts and dashboards for Prometheus.
To use them, you need to have jsonnet (v0.13+) and jb installed. If you
have a working Go development environment, it's easiest to run the following:
$ go install github.com/google/go-jsonnet/cmd/jsonnet@latest
$ go install github.com/google/go-jsonnet/cmd/jsonnetfmt@latest
$ go install github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest
Note: The make targets lint and fmt need the jsonnetfmt binary, which is
available from v.0.16.0 in the Go implementation of jsonnet. If your jsonnet version is older than 0.16.0 you have to either upgrade or install the C++ version of
jsonnetfmt if you want to use make lint
or make fmt.
Next, install the dependencies by running the following command in this directory:
$ jb install
You can then build a prometheus_alerts.yaml with the alerts and a directory
dashboards_out with the Grafana dashboard JSON files:
$ make prometheus_alerts.yaml
$ make dashboards_out
For more advanced uses of mixins, see https://github.com/monitoring-mixins/docs.