Commit Graph

108 Commits (a7b34d50fc5581d54948f9d2bcd67cb0403219bc)

Author SHA1 Message Date
Derek Menteer 7b4f45e2d5
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. (#15913)
Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy.
Disable xds check on bootstrap mode and change check to warn only.
2023-01-06 12:34:49 -06:00
Michael Wilkerson 1b28b89439
Enhancement: Consul Compatibility Checking (#15818)
* add functions for returning the max and min Envoy major versions
- added an UnsupportedEnvoyVersions list
- removed an unused error from TestDetermineSupportedProxyFeaturesFromString
- modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0`

* added a function for executing the envoy --version command
- added a new exec.go file to not be locked to unix system

* added envoy version check when using consul connect envoy

* added changelog entry

* added docs change
2022-12-20 09:58:19 -08:00
Chris S. Kim f7b7f5d4b6
Error out `consul connect envoy` if agent explicitly disabled grpc (#15794)
Co-authored-by: Paul Glass <pglass@hashicorp.com>
2022-12-19 14:37:27 -05:00
Derek Menteer 6fa8fa4fca
Fix issue with connect Envoy choosing incorrect TLS settings. (#15466)
This commit fixes a situation where the API TLS configuration
incorrectly influences the GRPC port TLS configuration for XDS.
2022-11-18 14:36:20 -06:00
Kyle Schochenmaier bf0f61a878
removes ioutil usage everywhere which was deprecated in go1.16 (#15297)
* update go version to 1.18 for api and sdk, go mod tidy
* removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward.
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2022-11-10 10:26:01 -06:00
Curt Bushko 161273a931
Update command/connect/envoy/bootstrap_tpl.go
Co-authored-by: Paul Glass <pglass@hashicorp.com>
2022-10-19 15:52:10 -04:00
Curt Bushko cd185c4c2c update prometheus template args 2022-10-19 15:44:19 -04:00
Derek Menteer f64771c707 Address PR comments. 2022-09-01 16:54:24 -05:00
Derek Menteer 1255a8a20d Add separate grpc_tls port.
To ease the transition for users, the original gRPC
port can still operate in a deprecated mode as either
plain-text or TLS mode. This behavior should be removed
in a future release whenever we no longer support this.

The resulting behavior from this commit is:
  `ports.grpc > 0 && ports.grpc_tls > 0` spawns both plain-text and tls ports.
  `ports.grpc > 0 && grpc.tls == undefined` spawns a single plain-text port.
  `ports.grpc > 0 && grpc.tls != undefined` spawns a single tls port (backwards compat mode).
2022-08-29 13:43:43 -05:00
Chris S. Kim 8ed49ea4d0
Update envoy metrics label extraction for peered clusters and listeners (#13818)
Now that peered upstreams can generate envoy resources (#13758), we need a way to disambiguate local from peered resources in our metrics. The key difference is that datacenter and partition will be replaced with peer, since in the context of peered resources partition is ambiguous (could refer to the partition in a remote cluster or one that exists locally). The partition and datacenter of the proxy will always be that of the source service.

Regexes were updated to make emitting datacenter and partition labels mutually exclusive with peer labels.

Listener filter names were updated to better match the existing regex.

Cluster names assigned to peered upstreams were updated to be synthesized from local peer name (it previously used the externally provided primary SNI, which contained the peer name from the other side of the peering). Integration tests were updated to assert for the new peer labels.
2022-07-25 13:49:00 -04:00
Kyle Havlovitz 407e858389 Fix syntax for bootstrap sds secret config 2022-07-06 09:53:40 -07:00
R.B. Boyer 31b95c747b
xds: modify rbac rules to use the XFCC header for peered L7 enforcement (#13629)
When the protocol is http-like, and an intention has a peered source
then the normal RBAC mTLS SAN field check is replaces with a joint combo
of:

    mTLS SAN field must be the service's local mesh gateway leaf cert
      AND
    the first XFCC header (from the MGW) must have a URI field that matches the original intention source

Also:

- Update the regex program limit to be much higher than the teeny
  defaults, since the RBAC regex constructions are more complicated now.

- Fix a few stray panics in xds generation.
2022-06-29 10:29:54 -05:00
Kyle Havlovitz 55109eb9f6 command: Add TLS support for envoy prometheus endpoint 2022-06-16 17:53:05 -07:00
Riddhi Shah 411edc876b
[OSS] Support merge-central-config option in node services list API (#13450)
Adds the merge-central-config query param option to the /catalog/node-services/:node-name API,
to get a service definition in the response that is merged with central defaults (proxy-defaults/service-defaults).

Updated the consul connect envoy command to use this option when
retrieving the proxy service details so as to render the bootstrap configuration correctly.
2022-06-15 08:30:31 -07:00
Riddhi Shah 7a039b46a2
[OSS] consul connect envoy command changes for agentless (#13361)
Changes the sourcing of the envoy bootstrap configuration
to not use agent APIs and instead use the catalog(server) API.
This is done by passing a node-name flag to the command,
(which can only be used with proxy-id).

Also fixes a bug where the golden envoy bootstrap config files
used for tests did not use the expected destination service name
in certain places for connect proxy kind.
2022-06-06 09:23:08 -07:00
Evan Culver 000d0621b4
connect: Add Envoy 1.22 to integration tests, remove Envoy 1.18 (#12805)
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2022-04-18 09:36:07 -07:00
R.B. Boyer b60d89e7ef bulk rewrite using this script
set -euo pipefail

    unset CDPATH

    cd "$(dirname "$0")"

    for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do
        echo "=== require: $f ==="
        sed -i '/require := require.New(t)/d' $f
        # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f
        # require.XXX(tblah) but not require.XXX(t, blah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f
        # require.XXX(rblah) but not require.XXX(r, blah)
        sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f
        gofmt -s -w $f
    done

    for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do
        echo "=== assert: $f ==="
        sed -i '/assert := assert.New(t)/d' $f
        # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f
        # assert.XXX(tblah) but not assert.XXX(t, blah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f
        # assert.XXX(rblah) but not assert.XXX(r, blah)
        sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f
        gofmt -s -w $f
    done
2022-01-20 10:46:23 -06:00
freddygv 6bbf109bdd Update golden files 2021-12-14 08:57:09 -07:00
freddygv 68424b318a Get partition label from upstream metrics 2021-12-14 08:40:47 -07:00
Evan Culver a0c754d44f
connect: update SNI label extraction to support new taxonomy for partitions (#11786) 2021-12-10 10:26:22 -08:00
R.B. Boyer 1e02460bd1
re-run gofmt on 1.17 (#11579)
This should let freshly recompiled golangci-lint binaries using Go 1.17
pass 'make lint'
2021-11-16 12:04:01 -06:00
Evan Culver be667e280f
connect: Remove envoy_version from bootstrap template (#11215) 2021-10-11 19:18:56 -07:00
Evan Culver 1709309cc7
regenerate more envoy golden files 2021-09-22 09:30:23 -07:00
freddygv e0a7900f52 Fixup api config and Envoy test 2021-09-15 17:25:21 -06:00
freddygv cecf5b18f8 Bring back entmeta args defaulting 2021-09-15 17:23:52 -06:00
freddygv 7ecbac9228 Ensure Envoy can subscribe to non-default partition 2021-09-15 17:23:48 -06:00
Freddy 8d83d27674
connect: update envoy supported versions to latest patch release
(#10961)

Relevant advisory: 
https://github.com/envoyproxy/envoy/security/advisories/GHSA-6g4j-5vrw-2m8h
2021-08-31 10:39:18 -06:00
Blake Covarrubias e41d6ee60f
cli: Use admin bind address in self_admin cluster (#10757)
Configure the self_admin cluster to use the admin bind address
provided when starting Envoy.

Fixes #10747
2021-08-09 17:10:32 -07:00
R.B. Boyer b0657973f2
add partition cli flag to all cli commands that have namespace flag (#10668) 2021-07-21 14:45:24 -05:00
Blake Covarrubias 6c462d399b
cli: Document pass-through option for `consul connect envoy` (#10666)
Update help text of `consul connect envoy` command to mention the
ability to provide pass-through options.
2021-07-21 10:43:10 -07:00
Daniel Nephin 4ad80ccee3 command/envoy: stop using the DebugConfig from Self endpoint
The DebugConfig in the self endpoint can change at any time. It's not a stable API.

With the previous change to rename GRPCPort to XDSPort this command would have broken.
This commit adds the XDSPort to a stable part of the XDS api, and changes the envoy command to read
this new field.

It includes support for the old API as well, in case a newer CLI is used with an older API, and
adds a test for both cases.
2021-07-13 12:31:49 -04:00
Daniel Nephin 4d741531b4 Update references to the main branch
The main branch is being renamed from master->main. This commit should
update all references to the main branch to the new name.

Co-Authored-By: Mike Morris <mikemorris@users.noreply.github.com>
2021-06-29 17:17:38 -04:00
Freddy 429f9d8bb8
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Daniel Nephin 2054402a53 envoy: improve comments 2021-06-01 11:35:32 -04:00
Daniel Nephin c9bc5f92b7 envoy: fix bootstrap deadlock caused by a full named pipe
Normally the named pipe would buffer up to 64k, but in some cases when a
soft limit is reached, they will start only buffering up to 4k.
In either case, we should not deadlock.

This commit changes the pipe-bootstrap command to first buffer all of
stdin into the process, before trying to write it to the named pipe.
This allows the process memory to act as the buffer, instead of the
named pipe.

Also changed the order of operations in `makeBootstrapPipe`. The new
test added in this PR showed that simply buffering in the process memory
was not enough to fix the issue. We also need to ensure that the
`pipe-bootstrap` process is started before we try to write to its
stdin. Otherwise the write will still block.

Also set stdout/stderr on the subprocess, so that any errors are visible
to the user.
2021-05-31 18:53:17 -04:00
Daniel Nephin e1b1ab7ef6 envoy: start timeout func after validation
This removes the need to check arg length in the timeout function.
2021-05-31 17:37:58 -04:00
R.B. Boyer 3b50a55533
connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231) 2021-05-12 14:06:06 -05:00
R.B. Boyer abc1dc0fe9
connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101)
The only thing that needed fixing up pertained to this section of the 1.18.x release notes:

> grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default.

For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default.

Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.
2021-04-29 15:22:03 -05:00
R.B. Boyer 71d45a3460
Support Incremental xDS mode (#9855)
This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged.

Union of all commit messages follows to give an overarching summary:

xds: exclusively support incremental xDS when using xDS v3

Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail.
Work around a strange older envoy behavior involving empty CDS responses over incremental xDS.
xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support

Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead.
Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client.
xds: pull out checkStreamACLs method in advance of a later commit

xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings

In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty.

This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10.

xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time
2021-04-29 13:54:05 -05:00
R.B. Boyer 5bcfe930c6
command: when generating envoy bootstrap configs to stdout do not mix informational logs into the json (#9980)
Fixes #9921
2021-04-07 14:22:52 -05:00
woz5999 39f448589d support env var expansion in envoy statsd urls
Fixes #8561
2021-03-18 18:57:28 -04:00
Nitya Dhanushkodi 15e8b13891
Add flags to consul connect envoy for metrics merging. (#9768)
Allows setting -prometheus-backend-port to configure the cluster
envoy_prometheus_bind_addr points to.

Allows setting -prometheus-scrape-path to configure which path
envoy_prometheus_bind_addr exposes metrics on.

-prometheus-backend-port is used by the consul-k8s metrics merging feature, to
configure envoy_prometheus_bind_addr to point to the merged metrics
endpoint that combines Envoy and service metrics so that one set of
annotations on a Pod can scrape metrics from the service and it's Envoy
sidecar.

-prometheus-scrape-path is used to allow configurability of the path
where prometheus metrics are exposed on envoy_prometheus_bind_addr.
2021-03-04 16:15:47 -06:00
R.B. Boyer 398b766532
xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658)
- Also add support for envoy 1.17.0
2021-02-26 16:23:15 -06:00
R.B. Boyer 3b6ffc447b
xds: remove deprecated usages of xDS (#9602)
Note that this does NOT upgrade to xDS v3. That will come in a future PR.

Additionally:

- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.

Fixes #8425
2021-02-22 15:00:15 -06:00
R.B. Boyer 6eeccc93ce
connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737) 2021-02-10 13:11:15 -06:00
R.B. Boyer b5f9a293c9
chore: regenerate envoy golden files (#9634) 2021-01-25 14:03:15 -06:00
Daniel Nephin b9e60c0775 testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
R.B. Boyer 7c7a3e5165
command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229)
Fixes #9215
2020-11-19 15:27:31 -06:00
Freddy fe728855ed
Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 16:37:19 -07:00
Mike Morris 6396042ba7
connect: switch the default gateway port from 443 to 8443 (#9116)
* test: update ingress gateway golden file to port 8443

* test: update Envoy flags_test to port 8443

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-11-06 20:47:29 -05:00