209 Commits (8af4ad178c4562ee824685173f8929711ed52573)

Author	SHA1	Message	Date
Joshua Timmons	55056be093	Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888)	1 year ago
Dan Stough	a043981cc6	Revert "fix(connect envoy): set initial_fetch_timeout to wait for initial xDS… (#17317)" (#17540) ... This reverts commit be7d2a4d84.	2 years ago
Jared Kirschner	b9c9d79778	Accept ap, datacenter, and namespace query params (#17525) ... This commit only contains the OSS PR (datacenter query param support). A separate enterprise PR adds support for ap and namespace query params. Resources in Consul can exists within scopes such as datacenters, cluster peers, admin partitions, and namespaces. You can refer to those resources from interfaces such as the CLI, HTTP API, DNS, and configuration files. Some scope levels have consistent naming: cluster peers are always referred to as "peer". Other scope levels use a short-hand in DNS lookups... - "ns" for namespace - "ap" for admin partition - "dc" for datacenter ...But use long-hand in CLI commands: - "namespace" for namespace - "partition" for admin partition - and "datacenter" However, HTTP API query parameters do not follow a consistent pattern, supporting short-hand for some scopes but long-hand for others: - "ns" for namespace - "partition" for admin partition - and "dc" for datacenter. This inconsistency is confusing, especially for users who have been exposed to providing scope names through another interface such as CLI or DNS queries. This commit improves UX by consistently supporting both short-hand and long-hand forms of the namespace, partition, and datacenter scopes in HTTP API query parameters.	2 years ago
Connor	0789661ce5	Rename hcp-metrics-collector to consul-telemetry-collector (#17327) ... * Rename hcp-metrics-collector to consul-telemetry-collector * Fix docs * Fix doc comment --------- Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>	2 years ago
Dan Stough	be7d2a4d84	fix(connect envoy): set initial_fetch_timeout to wait for initial xDS… (#17317) ... * fix(connect envoy): set initial_fetch_timeout to wait for initial xDS indefinitely --------- Co-authored-by: Kiril Angov <kiril.angov@gmail.com>	2 years ago
Freddy	7c3e9cd862	Hash namespace+proxy ID when creating socket path (#17204) ... UNIX domain socket paths are limited to 104-108 characters, depending on the OS. This limit was quite easy to exceed when testing the feature on Kubernetes, due to how proxy IDs encode the Pod ID eg: metrics-collector-59467bcb9b-fkkzl-hcp-metrics-collector-sidecar-proxy To ensure we stay under that character limit this commit makes a couple changes: - Use a b64 encoded SHA1 hash of the namespace + proxy ID to create a short and deterministic socket file name. - Add validation to proxy registrations and proxy-defaults to enforce a limit on the socket directory length.	2 years ago
Freddy	0fc4fc6429	Revert "[CC-4519] Include Consul NodeID in Envoy bootstrap metadata" (#17191)	2 years ago
Freddy	c5c35ec924	[CC-4519] Include Consul NodeID in Envoy bootstrap metadata (#17139) ... This is being added so that metrics sent to HCP can be augmented with the source node's ID. Opting not to add this to stats_tag out of caution, since it would increase the cardinality of metrics emitted by Envoy for all users. There is no functional impact to Envoy expected from this change.	2 years ago
Ronald	4c070c38e4	Copyright headers for command folder (#16705) ... * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files * copyright headers for agent folder * Copyright headers for command folder * fix merge conflicts	2 years ago
Ashvitha	f95ffe0355	Allow HCP metrics collection for Envoy proxies ... Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory where a unix socket will be created with the name `<namespace>_<proxy_id>.sock` to forward Envoy metrics. If set, this will configure: - In bootstrap configuration a local stats_sink and static cluster. These will forward metrics to a loopback listener sent over xDS. - A dynamic listener listening at the socket path that the previously defined static cluster is sending metrics to. - A dynamic cluster that will forward traffic received at this listener to the hcp-metrics-collector service. Reasons for having a static cluster pointing at a dynamic listener: - We want to secure the metrics stream using TLS, but the stats sink can only be defined in bootstrap config. With dynamic listeners/clusters we can use the proxy's leaf certificate issued by the Connect CA, which isn't available at bootstrap time. - We want to intelligently route to the HCP collector. Configuring its addreess at bootstrap time limits our flexibility routing-wise. More on this below. Reasons for defining the collector as an upstream in `proxycfg`: - The HCP collector will be deployed as a mesh service. - Certificate management is taken care of, as mentioned above. - Service discovery and routing logic is automatically taken care of, meaning that no code changes are required in the xds package. - Custom routing rules can be added for the collector using discovery chain config entries. Initially the collector is expected to be deployed to each admin partition, but in the future could be deployed centrally in the default partition. These config entries could even be managed by HCP itself.	2 years ago
Chris S. Kim	8daddff08d	Follow-up fixes to consul connect envoy command (#16530)	2 years ago
Michael Wilkerson	c517f07eca	modified unsupported envoy version error (#16518) ... - When an envoy version is out of a supported range, we now return the envoy version being used as `major.minor.x` to indicate that it is the minor version at most that is incompatible - When an envoy version is in the list of unsupported envoy versions we return back the envoy version in the error message as `major.minor.patch` as now the exact version matters.	2 years ago
Andrew Stucki	99cf421e7b	Add some fixes to allow for registering via consul connect envoy -gateway api (#16219) ... * Add some fixes to allow for registering via consul connect envoy -gateway api * Fix infinite recursion --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Nitya Dhanushkodi	b8b37c2357	refactor: remove troubleshoot module dependency on consul top level module (#16162) ... Ensure nothing in the troubleshoot go module depends on consul's top level module. This is so we can import troubleshoot into consul-k8s and not import all of consul. * turns troubleshoot into a go module [authored by @curtbushko] * gets the envoy protos into the troubleshoot module [authored by @curtbushko] * adds a new go module `envoyextensions` which has xdscommon and extensioncommon folders that both the xds package and the troubleshoot package can import * adds testing and linting for the new go modules * moves the unit tests in `troubleshoot/validateupstream` that depend on proxycfg/xds into the xds package, with a comment describing why those tests cannot be in the troubleshoot package * fixes all the imports everywhere as a result of these changes Co-authored-by: Curt Bushko <cbushko@gmail.com>	2 years ago
Kyle Havlovitz	edef99011c	command: Fix logger not initializing properly in envoy command (#16148)	2 years ago
Kyle Havlovitz	d53c331a37	Add a flag for enabling debug logs to the `connect envoy` command (#15988) ... * Add a flag for enabling debug logs to the `connect envoy` command * Update website/content/commands/connect/envoy.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> * Add changelog note * Add debug log note to envoy proxy doc page * Update website/content/docs/connect/proxies/envoy.mdx Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com> * Wording tweak in envoy bootstrap section --------- Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>	2 years ago
Ashwin Venkatesh	a1e2a4f8d6	Add support for envoy readiness flags (#16015) ... * Add support for envoy readiness flags - add flags 'envoy-ready-bind-port` and `envoy-ready-bind-addr` on consul connect envoy to create a ready listener on that address.	2 years ago
Chris S. Kim	e4a268e33e	Warn if ACL is enabled but no token is provided to Envoy (#15967)	2 years ago
Dan Stough	6d2880e894	feat: add access logs to dataplane bootstrap rpc (#15951)	2 years ago
Dan Stough	88b9420a1a	[OSS] feat: add access log config to consul envoy connect (#15946)	2 years ago
Derek Menteer	7b4f45e2d5	Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. (#15913) ... Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. Disable xds check on bootstrap mode and change check to warn only.	2 years ago
Michael Wilkerson	1b28b89439	Enhancement: Consul Compatibility Checking (#15818) ... * add functions for returning the max and min Envoy major versions - added an UnsupportedEnvoyVersions list - removed an unused error from TestDetermineSupportedProxyFeaturesFromString - modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0` * added a function for executing the envoy --version command - added a new exec.go file to not be locked to unix system * added envoy version check when using consul connect envoy * added changelog entry * added docs change	2 years ago
Chris S. Kim	f7b7f5d4b6	Error out `consul connect envoy` if agent explicitly disabled grpc (#15794) ... Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
Semir Patel	bafa5c7156	Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700)	2 years ago
Derek Menteer	6fa8fa4fca	Fix issue with connect Envoy choosing incorrect TLS settings. (#15466) ... This commit fixes a situation where the API TLS configuration incorrectly influences the GRPC port TLS configuration for XDS.	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Chris S. Kim	bde57c0dd0	Regenerate files according to 1.19.2 formatter	2 years ago
Iryna Shustava	2a25669b13	cli/sdk: Allow redirection to a different consul dns port (#15050)	2 years ago
Curt Bushko	161273a931	Update command/connect/envoy/bootstrap_tpl.go ... Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
Curt Bushko	cd185c4c2c	update prometheus template args	2 years ago
Iryna Shustava	4bc4ef135c	cli: Add -node-name flag to redirect-traffic command (#14933)	2 years ago
Derek Menteer	f64771c707	Address PR comments.	2 years ago
Derek Menteer	1255a8a20d	Add separate grpc_tls port. ... To ease the transition for users, the original gRPC port can still operate in a deprecated mode as either plain-text or TLS mode. This behavior should be removed in a future release whenever we no longer support this. The resulting behavior from this commit is: `ports.grpc > 0 && ports.grpc_tls > 0` spawns both plain-text and tls ports. `ports.grpc > 0 && grpc.tls == undefined` spawns a single plain-text port. `ports.grpc > 0 && grpc.tls != undefined` spawns a single tls port (backwards compat mode).	2 years ago
Jared Kirschner	22511ec491	Allow uppercase in proxy launch -sidecar-for arg ... Previously, when launching a sidecar proxy with one of the following commands: - consul connect envoy -sidecar-for=... - consul connect proxy -sidecar-for=... ... the -sidecar-for argument could only contain lowercase letters, even if the service was registered with some uppercase letters. Now, the -sidecar-for argument is treated as case-insensitive.	2 years ago
Chris S. Kim	8ed49ea4d0	Update envoy metrics label extraction for peered clusters and listeners (#13818) ... Now that peered upstreams can generate envoy resources (#13758), we need a way to disambiguate local from peered resources in our metrics. The key difference is that datacenter and partition will be replaced with peer, since in the context of peered resources partition is ambiguous (could refer to the partition in a remote cluster or one that exists locally). The partition and datacenter of the proxy will always be that of the source service. Regexes were updated to make emitting datacenter and partition labels mutually exclusive with peer labels. Listener filter names were updated to better match the existing regex. Cluster names assigned to peered upstreams were updated to be synthesized from local peer name (it previously used the externally provided primary SNI, which contained the peer name from the other side of the peering). Integration tests were updated to assert for the new peer labels.	2 years ago
Kyle Havlovitz	407e858389	Fix syntax for bootstrap sds secret config	2 years ago
R.B. Boyer	31b95c747b	xds: modify rbac rules to use the XFCC header for peered L7 enforcement (#13629) ... When the protocol is http-like, and an intention has a peered source then the normal RBAC mTLS SAN field check is replaces with a joint combo of: mTLS SAN field must be the service's local mesh gateway leaf cert AND the first XFCC header (from the MGW) must have a URI field that matches the original intention source Also: - Update the regex program limit to be much higher than the teeny defaults, since the RBAC regex constructions are more complicated now. - Fix a few stray panics in xds generation.	2 years ago
Kyle Havlovitz	55109eb9f6	command: Add TLS support for envoy prometheus endpoint	2 years ago
Riddhi Shah	411edc876b	[OSS] Support merge-central-config option in node services list API (#13450) ... Adds the merge-central-config query param option to the /catalog/node-services/:node-name API, to get a service definition in the response that is merged with central defaults (proxy-defaults/service-defaults). Updated the consul connect envoy command to use this option when retrieving the proxy service details so as to render the bootstrap configuration correctly.	2 years ago
Riddhi Shah	7a039b46a2	[OSS] consul connect envoy command changes for agentless (#13361) ... Changes the sourcing of the envoy bootstrap configuration to not use agent APIs and instead use the catalog(server) API. This is done by passing a node-name flag to the command, (which can only be used with proxy-id). Also fixes a bug where the golden envoy bootstrap config files used for tests did not use the expected destination service name in certain places for connect proxy kind.	3 years ago
Evan Culver	000d0621b4	connect: Add Envoy 1.22 to integration tests, remove Envoy 1.18 (#12805) ... Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
R.B. Boyer	d06183ba7f	syncing changes back from enterprise (#12701)	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script ... set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
freddygv	6bbf109bdd	Update golden files	3 years ago
freddygv	68424b318a	Get partition label from upstream metrics	3 years ago
Evan Culver	a0c754d44f	connect: update SNI label extraction to support new taxonomy for partitions (#11786)	3 years ago
R.B. Boyer	1e02460bd1	re-run gofmt on 1.17 (#11579) ... This should let freshly recompiled golangci-lint binaries using Go 1.17 pass 'make lint'	3 years ago
freddygv	5bc4aa49bd	Fix test	3 years ago
freddygv	4c9c1b52ce	Support partitions in connect expose cmd	3 years ago
freddygv	a6d985040f	Fixup shared oss/ent tests	3 years ago