Commit Graph

189 Commits (48a7048d9819ee516b3cfdf5c458b5815b640138)

Author SHA1 Message Date
Marek Siarkowicz 288dbd03e5 [fluentd-gcp addon] Fix passing location to event exporter 2018-03-05 15:05:35 +01:00
Bryan Moyles 84a86cffce Update to use Stackdriver Agent image.
Prometheus is enabled by default.
2018-02-26 14:05:33 -05:00
Kubernetes Submit Queue a85f7d9fff
Merge pull request #58090 from serathius/pass-location-to-event-exporter
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass location parameter to event exporter.

**What this PR does / why we need it**:
This PR makes event-exporter export cluster location together with events.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-23 23:15:43 -08:00
Kubernetes Submit Queue 5b0383ed9a
Merge pull request #60126 from serathius/update-event-exporter
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[fluentd-gcp addon] Update event-exporter

Update to new version of event-exporter which includes bugfix for metrics
https://github.com/GoogleCloudPlatform/k8s-stackdriver/releases/tag/event-exporter-v0.1.8

```release-note
[fluentd-gcp addon] Fixed bug with reporting metrics in event-exporter
```
2018-02-23 14:01:47 -08:00
Daniel Kłobuszewski aac6e3f739 Review #1 2018-02-22 09:59:16 +01:00
Daniel Kłobuszewski 6ef5d17701 readme update for fluentd-gcp-scaler 2018-02-21 15:50:23 +01:00
Marek Siarkowicz bbfcd681b5 Pass location parameter to event exporter.
Location passed based on ZONE from kube-env.
2018-02-21 12:54:29 +01:00
Marek Siarkowicz 79fabb0a26 [fluentd-gcp addon] Update event-exporter 2018-02-21 09:56:43 +01:00
Daniel Kłobuszewski a88ddac1e4 use prometheus-to-sd 0.2.4 and fluentd-gcp-image 2.0.16 2018-02-16 09:16:59 +01:00
Daniel Kłobuszewski 6db742fc10 fix fluentd-gcp-scaler to look at correct fluentd-gcp version 2018-02-15 16:15:41 +01:00
Kubernetes Submit Queue d3bacb914c
Merge pull request #59657 from x13n/manual-fluentd-gcp-scaler
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable scaling fluentd-gcp resources using ScalingPolicy.

See https://github.com/justinsb/scaler for more details about ScalingPolicy resource.

**What this PR does / why we need it**:
This is adding a way to override fluentd-gcp resources in a running cluster. The resources syncing for fluentd-gcp is decoupled from addon manager.

**Special notes for your reviewer**:

**Release note**:
```release-note
fluentd-gcp resources can be modified via a ScalingPolicy
```

cc @kawych @justinsb
2018-02-15 03:42:14 -08:00
Kubernetes Submit Queue bc9c6df31d
Merge pull request #59103 from Random-Liu/upload-container-runtime-log
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Upload container runtime log to sd/es.

I've verified this in my environment. My stackdriver has an extra `container-runtime` entry for node log, and it collects container runtime daemon log correctly.

@yujuhong @feiskyer @crassirostris @piosz 
@kubernetes/sig-node-pr-reviews @kubernetes/sig-instrumentation-pr-reviews 
Signed-off-by: Lantao Liu <lantaol@google.com>

**Release note**:

```release-note
Container runtime daemon (e.g. dockerd) logs in GCE cluster will be uploaded to stackdriver and elasticsearch with tag `container-runtime`
```
2018-02-14 03:33:21 -08:00
Lantao Liu 8d920d095c Upload container runtime log to sd/es.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-13 18:25:02 +00:00
Daniel Kłobuszewski 3b178ee45c Review #1 2018-02-13 10:23:58 +01:00
Kubernetes Submit Queue 7ef11bd964
Merge pull request #59237 from tanshanshan/addons1
Automatic merge from submit-queue (batch tested with PRs 59767, 56454, 59237, 59730, 55479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Change critical pods’ template to use priority

**What this PR does / why we need it**:
Change critical pods’ template to use priority
Thanks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref #57471

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-02-12 15:44:36 -08:00
Daniel Kłobuszewski 2eb24f9ae1 Enable scaling fluentd-gcp resources using ScalingPolicy.
See https://github.com/justinsb/scaler for more details about ScalingPolicy resource.
2018-02-09 14:33:33 +01:00
tanshanshan 95b2b94b1b Change critical pods’ template to use priority 2018-02-08 15:06:27 +08:00
Tim Hockin 3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Robert Bailey 7cd213ae82 Remove salt configuration from the fluentd-gcp configuration. 2018-01-22 21:06:09 -08:00
Ross Light 6831581f1c Bump fluentd-gcp version 2018-01-12 10:16:13 -08:00
Daniel Kłobuszewski dca74f17fd
Bump fluentd-gcp image used to 2.0.13 2018-01-08 14:54:26 +01:00
Daniel Kłobuszewski 2eded687be
Bump fluentd-gcp version 2018-01-03 11:46:13 +01:00
Tim Hockin e9dd8a68f6 Revert k8s.gcr.io vanity domain
This reverts commit eba5b6092a.

Fixes https://github.com/kubernetes/kubernetes/issues/57526
2017-12-22 14:36:16 -08:00
Tim Hockin eba5b6092a Use k8s.gcr.io vanity domain for container images 2017-12-18 09:18:34 -08:00
Daniel Kłobuszewski d2cbc37c05
Bump fluentd-gcp version 2017-12-07 14:23:05 +01:00
Rohit Agarwal ad05928c6e Add wildcard tolerations to kube-proxy.
fluend-gcp already has these tolerations. kube-proxy when it runs as a
static pod gets wildcard `NoExecute` toleration (all static pods get
that). So, added the same toleration to kube-proxy when it runs as a
daemonset. Also added wildcard `NoSchedule` toleration to kube-proxy.
2017-11-29 12:36:58 -08:00
Marian Lobur 3d7a694aed Fix typo in component name of prometheus-to-sd config. 2017-11-28 09:02:08 +01:00
Kubernetes Submit Queue f85649c6cd
Merge pull request #55950 from crassirostris/fluentd-gcp-setting-resources-fix
Automatic merge from submit-queue (batch tested with PRs 56207, 55950). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix setting resources in fluentd-gcp plugin

Currently if some of the variables are not set, scripts prints error, which is not critical, since the function is executed in a separate process, but it leads to the wrong resulting values

```release-note
NONE
```

/cc @piosz @x13n 
/assign @roberthbailey @mikedanese 
Could you please approve?
2017-11-22 05:21:25 -08:00
Mik Vyatskov e9322b929c Fix setting resources in fluentd-gcp plugin
Signed-off-by: Mik Vyatskov <vmik@google.com>
2017-11-22 12:40:50 +01:00
Lantao Liu 09b614abbb Do not add new field in fluentd CRI log format. 2017-11-22 01:25:24 +00:00
Lantao Liu 2bc0532eb3 Add fluentd support. 2017-11-21 00:36:01 +00:00
Kubernetes Submit Queue b2125f5aa8
Merge pull request #55509 from tallclair/psp-addons
Automatic merge from submit-queue (batch tested with PRs 54602, 54877, 55243, 55509, 55128). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

PodSecurityPolicies for addons

**What this PR does / why we need it**:

1. Colocate addon PodSecurityPolicy config with the addons (in a `podsecuritypolicies` subdirectory). 
2. Add policies for addons that are currently missing policies (not in the default GCE suite)
3. Remove HostPath SSL certs from several heapster deployments, so that heapster doesn't require a special PSP

**Which issue(s) this PR fixes**:
#43538

**Release note**:
```release-note
- Add PodSecurityPolicies for cluster addons
- Remove SSL cert HostPath volumes from heapster addons
```
2017-11-14 03:03:30 -08:00
Tim Allclair a1513161b3
Reorganize addon PodSecurityPolicies 2017-11-10 13:57:32 -08:00
Lantao Liu 53d7494b9e Fix CRI fluentd config.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-11-10 20:55:56 +00:00
Lantao Liu 70a0cdfa8e Add CRI log format support in fluentd. 2017-10-30 06:25:52 +00:00
Kubernetes Submit Queue 949ec719c3
Merge pull request #54635 from loburm/prom-to-sd
Automatic merge from submit-queue (batch tested with PRs 54635, 54250, 54657, 54696, 54700). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump version of prometheus-to-sd to 0.2.2.

Bump version of prometheus-to-sd to improve logging, add pod_name and
pod_namespace flags and remove deprecated flags.

Fixes #54583 

```release-note
NONE
```
2017-10-27 14:38:21 -07:00
Kubernetes Submit Queue fc8bfe2d89 Merge pull request #54395 from crassirostris/fluentd-gcp-rollback-host-networking
Automatic merge from submit-queue (batch tested with PRs 50776, 54395). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move fluentd-gcp out of host network

Since metadata proxy doesn't filter service account after all, make fluentd-gcp addon run in its own network

This will mitigate the problem with port collision

```release-note
[fluentd-gcp addon] Fluentd now runs in its own network, not in the host one.
```
2017-10-27 03:09:25 -07:00
Marian Lobur 5b62eb29d2 Bump version of prometheus-to-sd to 0.2.2.
Bump version of prometheus-to-sd to improve logging, add pod_name and
pod_namespace flags and remove deprecated flags.
2017-10-26 15:54:54 +02:00
Tim Allclair b18edfec7a
Update fluentd-gcp DaemonSet
- Use a dedicated service account to run the fluentd-gcp DS
- Update prometheus-to-sd from v0.1.3 to v0.2.1
- Use the certificates in the prometheus-to-sd image rather than mounting the host certs
2017-10-25 13:11:35 -07:00
Mik Vyatskov d30af4d8a0 Move fluentd-gcp out of host network 2017-10-23 12:02:54 +02:00
Kubernetes Submit Queue 74604d123d Merge pull request #52565 from crassirostris/sd-logging-update-plugin-0.6.7
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..

[fluentd-gcp addon] Update Stackdriver plugin to version 0.6.7

A new gem among all fixes Java logging severity parsing and string timestamp parsing

Also sync the buffer size with the gem guidelines, making it 1M instead of 2M.

/cc @igorpeshansky
2017-09-20 16:09:31 -07:00
Mik Vyatskov e73eb86dda [fluentd-gcp addon] Remove audit logs from the fluentd configuration 2017-09-20 11:08:59 +02:00
Mik Vyatskov ae132a0900 [fluentd-gcp] Update Stackdriver plugin to version 0.6.7 2017-09-15 22:30:15 +02:00
Mik Vyatskov 42be772234 [fluentd-gcp addon] By default ingest audit logs in JSON format 2017-09-15 15:38:30 +02:00
Mik Vyatskov d8525f8bd1 [fluentd-gcp addon] Trim too long log entries due to Stackdriver limitation 2017-09-13 10:27:17 +02:00
Kubernetes Submit Queue 36b43013c6 Merge pull request #52335 from crassirostris/sd-logging-add-metric
Automatic merge from submit-queue (batch tested with PRs 51041, 52297, 52296, 52335, 52338)

[fluentd-gcp addon] Restore the metric for the number of read log entries

This metric, previously removed, will allow to monitor the number of log entries, that were read, but weren't sent by the output plugin because of liveness probe removing the data.
2017-09-12 11:10:15 -07:00
Mik Vyatskov 683fc23000 [fluentd-gcp addon] Restore the metric for the number of read log entries 2017-09-12 13:24:55 +02:00
Mik Vyatskov 5447a5a2b2 [fluentd-gcp addon] Update event-exporter to address metrics problem 2017-09-11 10:07:10 +02:00
Kubernetes Submit Queue 3b2e403a37 Merge pull request #51011 from xilabao/rbac-v1-in-yaml
Automatic merge from submit-queue (batch tested with PRs 50489, 51070, 51011, 51022, 51141)

update to rbac v1 in yaml file

**What this PR does / why we need it**:
ref to https://github.com/kubernetes/kubernetes/pull/49642
ref https://github.com/kubernetes/features/issues/2

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
cc @liggitt 

**Release note**:

```release-note
NONE
```
2017-08-23 19:54:28 -07:00
Marian Lobur d48e2d338d Add flags for prometheus-to-sd components.
Configure prometheus-to-sd-endpoint and prometheus-to-sd-prefix base on
the environment.
2017-08-22 08:30:08 +02:00