Commit Graph

155 Commits (44aa7116e480ba7cbe430b5ee7c53b2cbbc51ab7)

Author SHA1 Message Date
Lantao Liu 70a0cdfa8e Add CRI log format support in fluentd. 2017-10-30 06:25:52 +00:00
Kubernetes Submit Queue 949ec719c3
Merge pull request #54635 from loburm/prom-to-sd
Automatic merge from submit-queue (batch tested with PRs 54635, 54250, 54657, 54696, 54700). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump version of prometheus-to-sd to 0.2.2.

Bump version of prometheus-to-sd to improve logging, add pod_name and
pod_namespace flags and remove deprecated flags.

Fixes #54583 

```release-note
NONE
```
2017-10-27 14:38:21 -07:00
Kubernetes Submit Queue fc8bfe2d89 Merge pull request #54395 from crassirostris/fluentd-gcp-rollback-host-networking
Automatic merge from submit-queue (batch tested with PRs 50776, 54395). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move fluentd-gcp out of host network

Since metadata proxy doesn't filter service account after all, make fluentd-gcp addon run in its own network

This will mitigate the problem with port collision

```release-note
[fluentd-gcp addon] Fluentd now runs in its own network, not in the host one.
```
2017-10-27 03:09:25 -07:00
Marian Lobur 5b62eb29d2 Bump version of prometheus-to-sd to 0.2.2.
Bump version of prometheus-to-sd to improve logging, add pod_name and
pod_namespace flags and remove deprecated flags.
2017-10-26 15:54:54 +02:00
Tim Allclair b18edfec7a
Update fluentd-gcp DaemonSet
- Use a dedicated service account to run the fluentd-gcp DS
- Update prometheus-to-sd from v0.1.3 to v0.2.1
- Use the certificates in the prometheus-to-sd image rather than mounting the host certs
2017-10-25 13:11:35 -07:00
Mik Vyatskov d30af4d8a0 Move fluentd-gcp out of host network 2017-10-23 12:02:54 +02:00
Kubernetes Submit Queue 74604d123d Merge pull request #52565 from crassirostris/sd-logging-update-plugin-0.6.7
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

[fluentd-gcp addon] Update Stackdriver plugin to version 0.6.7

A new gem among all fixes Java logging severity parsing and string timestamp parsing

Also sync the buffer size with the gem guidelines, making it 1M instead of 2M.

/cc @igorpeshansky
2017-09-20 16:09:31 -07:00
Mik Vyatskov e73eb86dda [fluentd-gcp addon] Remove audit logs from the fluentd configuration 2017-09-20 11:08:59 +02:00
Mik Vyatskov ae132a0900 [fluentd-gcp] Update Stackdriver plugin to version 0.6.7 2017-09-15 22:30:15 +02:00
Mik Vyatskov 42be772234 [fluentd-gcp addon] By default ingest audit logs in JSON format 2017-09-15 15:38:30 +02:00
Mik Vyatskov d8525f8bd1 [fluentd-gcp addon] Trim too long log entries due to Stackdriver limitation 2017-09-13 10:27:17 +02:00
Kubernetes Submit Queue 36b43013c6 Merge pull request #52335 from crassirostris/sd-logging-add-metric
Automatic merge from submit-queue (batch tested with PRs 51041, 52297, 52296, 52335, 52338)

[fluentd-gcp addon] Restore the metric for the number of read log entries

This metric, previously removed, will allow to monitor the number of log entries, that were read, but weren't sent by the output plugin because of liveness probe removing the data.
2017-09-12 11:10:15 -07:00
Mik Vyatskov 683fc23000 [fluentd-gcp addon] Restore the metric for the number of read log entries 2017-09-12 13:24:55 +02:00
Mik Vyatskov 5447a5a2b2 [fluentd-gcp addon] Update event-exporter to address metrics problem 2017-09-11 10:07:10 +02:00
Kubernetes Submit Queue 3b2e403a37 Merge pull request #51011 from xilabao/rbac-v1-in-yaml
Automatic merge from submit-queue (batch tested with PRs 50489, 51070, 51011, 51022, 51141)

update to rbac v1 in yaml file

**What this PR does / why we need it**:
ref to https://github.com/kubernetes/kubernetes/pull/49642
ref https://github.com/kubernetes/features/issues/2

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
cc @liggitt 

**Release note**:

```release-note
NONE
```
2017-08-23 19:54:28 -07:00
Marian Lobur d48e2d338d Add flags for prometheus-to-sd components.
Configure prometheus-to-sd-endpoint and prometheus-to-sd-prefix base on
the environment.
2017-08-22 08:30:08 +02:00
Chen Rong d23df051e1 update to rbac v1 in yaml file 2017-08-21 17:29:37 +08:00
Mik Vyatskov d64c8809af Update Stackdriver event exporter version 2017-08-11 12:11:25 +02:00
Mik Vyatskov 46f53daef0 Refactored the fluentd-es addon files, moved the fluentd configuration to ConfigMap 2017-08-04 20:05:15 +02:00
icereval 60872db873
log node-problem-detector 2017-07-16 16:46:25 -04:00
Kubernetes Submit Queue 455e44b616 Merge pull request #48812 from crassirostris/change-fluentd-monitoring
Automatic merge from submit-queue (batch tested with PRs 48812, 48276)

Change fluentd-gcp monitoring to use metrics exposed by SD plugin

Following https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/pull/135, make fluentd-gcp expose metrics in Prometheus registry and use them instead of counting records in the pipeline.

/cc @piosz @igorpeshansky

```release-note
Fluentd-gcp DaemonSet exposes different set of metrics.
```
2017-07-14 04:43:42 -07:00
Kubernetes Submit Queue b507254bb0 Merge pull request #46825 from zjj2wry/fix_doc_spell
Automatic merge from submit-queue

Update docs for user-guide

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-07-13 18:50:36 -07:00
Mik Vyatskov 125f759907 Change fluentd-gcp monitoring to use metrics exposed by SD plugin 2017-07-12 17:07:15 +02:00
Mik Vyatskov b11084e76c Bump event-exporter version 2017-07-10 17:32:40 +02:00
gmarek 3f57d8dba3 Add generic NoSchedule toleration to fluentd in gcp config as a quick-fix for #44445 2017-06-28 10:35:58 +02:00
Mik Vyatskov b6a0e442ce Fix fluentd-gcp configuration to facilitate JSON parsing 2017-06-27 16:16:00 +02:00
zhengjiajin 2ab0c65506 Update docs for user-guide 2017-06-27 12:21:49 +08:00
Kubernetes Submit Queue 93ef2f2f59 Merge pull request #47860 from crassirostris/fluentd-log-to-stdio
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)

Make fluentd log to stdio instead of a dedicated file

Lower verbosity also, to reduce volume of system logs exported to the backend.

Fix https://github.com/kubernetes/kubernetes/issues/43772

/cc @piosz
2017-06-26 14:03:04 -07:00
Piotr Szczesniak 77fd8a1898 Revert "Decrese fluentd cpu request" 2017-06-26 16:18:24 +02:00
Kubernetes Submit Queue 0cbd0ca189 Merge pull request #47915 from crassirostris/fix-event-exporter-noise
Automatic merge from submit-queue

Bump event-exporter version to reduce warnings noise

Fixes https://github.com/kubernetes/kubernetes/issues/47914
2017-06-23 03:45:20 -07:00
Kubernetes Submit Queue 045a6dca07 Merge pull request #47906 from gmarek/fluentd
Automatic merge from submit-queue

Decrese fluentd cpu request

Fix #47905

cc @piosz - this should fix your tests.
cc @dchen1107
2017-06-22 12:27:13 -07:00
Mik Vyatskov 3932622303 Bump event-exporter version to reduce warnings noise 2017-06-22 19:49:23 +02:00
gmarek 8427d5a274 Decrease fluentd cpu request 2017-06-22 18:30:21 +02:00
Jeff Grafton 4c7c865ff6 Update fluentd-gcp to 2.0.7 2017-06-21 14:08:12 -07:00
Jeff Grafton b43bb842ad Update event-exporter to v0.1.0-r2 and prometheus-to-sd to v0.1.2-r2 2017-06-21 14:08:09 -07:00
Mik Vyatskov 1d1c126783 Make fluentd log to stdio instead of a dedicated file 2017-06-21 20:14:43 +02:00
Jeff Grafton 641f8c1f29 Revert "Update fluentd-gcp to 2.0.6"
This reverts commit 0bcc271b28.
2017-06-15 11:46:17 -07:00
Jeff Grafton 0bcc271b28 Update fluentd-gcp to 2.0.6 2017-06-14 13:32:03 -07:00
Mik Vyatskov 1cc2235c17 Change port on which fluentd exposes its metrics 2017-06-13 08:15:34 +02:00
Kubernetes Submit Queue 695d438508 Merge pull request #46539 from crassirostris/fluentd-gcp-make-privileged
Automatic merge from submit-queue

Make fluentd-gcp run with host network

Fluentd-gcp should have access to instance's platform-dependent service account in order to work.

/cc @piosz
2017-06-12 10:13:21 -07:00
Kubernetes Submit Queue a03bb6fc5f Merge pull request #46787 from crassirostris/fluentd-gcp-update
Automatic merge from submit-queue

Update the fluentd-gcp image

Rolled back fluentd version to 0.12 to avoid performance problems and unnecessary noise in logs: https://github.com/kubernetes/contrib/pull/2625

Fixes https://github.com/kubernetes/kubernetes/issues/46990
2017-06-06 01:53:40 -07:00
Kubernetes Submit Queue 8df56da448 Merge pull request #46700 from crassirostris/add-event-exporter-deployment
Automatic merge from submit-queue

Add event exporter deployment to the fluentd-gcp addon

Introduce event exporter deployment to the fluentd-gcp addon so that by default if logging to Stackdriver is enabled, events will be available there also.

In this release, event exporter is a non-critical pod in BestEffort QoS class to avoid preempting actual workload in tightly loaded clusters. It will become critical in one of the future releases.


```release-note
Stackdriver cluster logging now deploys a new component to export Kubernetes events.
```
2017-06-06 00:00:49 -07:00
Mik Vyatskov 527206c36b Add event exporter deployment to the fluentd-gcp addon 2017-06-02 17:00:24 +02:00
Mik Vyatskov f41a61586d Update the fluentd-gcp image 2017-06-01 19:08:58 +02:00
Mik Vyatskov a4c3279a68 Update fluentd-gcp image version 2017-06-01 14:28:17 +02:00
Mik Vyatskov d3c809d900 Make fluentd-gcp run with host network 2017-05-26 22:32:28 +02:00
Mik Vyatskov a6ccc89541 Add version for fluentd-gcp config 2017-05-18 16:59:05 +02:00
Mik Vyatskov dcd3ce3bcb Export process start time metric in fluentd-gcp 2017-05-12 16:37:36 +02:00
gmarek 4d7d6b72b3 Add general NoExecute Toleration to fluentd in gcp configuration 2017-05-12 11:23:23 +02:00
Mik Vyatskov fb3e30729e Add metrics exporter to the fluentd-gcp deployment 2017-04-28 13:27:19 +02:00