2100 Commits (2293ccfecafaba5a5efcd3cd7a9a39799fa853a1)

Author	SHA1	Message	Date
Freddy	645e406ca0	Rename CatalogDestinationsOnly (#10397) ... CatalogDestinationsOnly is a passthrough that would enable dialing addresses outside of Consul's catalog. However, when this flag is set to true only _connect_ endpoints for services can be dialed. This flag is being renamed to signal that non-Connect endpoints can't be dialed by transparent proxies when the value is set to true.	4 years ago
Freddy	168073c4dc	Add flag for transparent proxies to dial individual instances (#10329)	4 years ago
Dhia Ayachi	3717ab0991	generate a single debug file for a long duration capture (#10279) ... * debug: remove the CLI check for debug_enabled The API allows collecting profiles even debug_enabled=false as long as ACLs are enabled. Remove this check from the CLI so that users do not need to set debug_enabled=true for no reason. Also: - fix the API client to return errors on non-200 status codes for debug endpoints - improve the failure messages when pprof data can not be collected Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com> * remove parallel test runs parallel runs create a race condition that fail the debug tests * snapshot the timestamp at the beginning of the capture - timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures - capture append to the file if it already exist * Revert "snapshot the timestamp at the beginning of the capture" This reverts commit c2d03346 * Refactor captureDynamic to extract capture logic for each item in a different func * snapshot the timestamp at the beginning of the capture - timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures - capture append to the file if it already exist * Revert "snapshot the timestamp at the beginning of the capture" This reverts commit c2d03346 * Refactor captureDynamic to extract capture logic for each item in a different func * extract wait group outside the go routine to avoid a race condition * capture pprof in a separate go routine * perform a single capture for pprof data for the whole duration * add missing vendor dependency * add a change log and fix documentation to reflect the change * create function for timestamp dir creation and simplify error handling * use error groups and ticker to simplify interval capture loop * Logs, profile and traces are captured for the full duration. Metrics, Heap and Go routines are captured every interval * refactor Logs capture routine and add log capture specific test * improve error reporting when log test fail * change test duration to 1s * make time parsing in log line more robust * refactor log time format in a const * test on log line empty the earliest possible and return Co-authored-by: Freddy <freddygv@users.noreply.github.com> * rename function to captureShortLived * more specific changelog Co-authored-by: Paul Banks <banks@banksco.de> * update documentation to reflect current implementation * add test for behavior when invalid param is passed to the command * fix argument line in test * a more detailed description of the new behaviour Co-authored-by: Paul Banks <banks@banksco.de> * print success right after the capture is done * remove an unnecessary error check Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> * upgraded github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57 => v0.0.0-20210601050228-01bbb1931b22 Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Co-authored-by: Paul Banks <banks@banksco.de>	4 years ago
Dhia Ayachi	c14439045f	fix monitor to only start the monitor in json format when requested (#10358) ... * fix monitor to only start the monitor in json format when requested * add release notes * add test to validate json format when requested	4 years ago
Daniel Nephin	f4a6ab97dc	Merge pull request #10338 from hashicorp/dnephin/fix-logging-indent ... agent: remove leading whitespace from agent log lines	4 years ago
Daniel Nephin	07a1b7d081	Merge pull request #10324 from hashicorp/dnephin/fix-envoy-bootstrap-exec ... envoy: fix deadlock when input is larger than named pipe buffer size	4 years ago
Dhia Ayachi	db23df862c	debug: remove the CLI check for debug_enabled (#10273) ... * debug: remove the CLI check for debug_enabled The API allows collecting profiles even debug_enabled=false as long as ACLs are enabled. Remove this check from the CLI so that users do not need to set debug_enabled=true for no reason. Also: - fix the API client to return errors on non-200 status codes for debug endpoints - improve the failure messages when pprof data can not be collected Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com> * remove parallel test runs parallel runs create a race condition that fail the debug tests * Add changelog Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>	4 years ago
Daniel Nephin	510d81ac4d	Merge pull request #10253 from hashicorp/dnephin/fix-test-flake ... redirecttraffic: fix a flaky test	4 years ago
Iryna Shustava	47d8f050d2	Save exposed ports in agent's store and expose them via API (#10173) ... * Save exposed HTTP or GRPC ports to the agent's store * Add those the health checks API so we can retrieve them from the API * Change redirect-traffic command to also exclude those ports from inbound traffic redirection when expose.checks is set to true.	4 years ago
R.B. Boyer	88a8656e13	connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231)	4 years ago
Mark Anderson	42ff449d4f	Merge pull request #9981 from hashicorp/ma/uds_upstreams ... Unix Domain Socket support for upstreams and downstreams	4 years ago
Daniel Nephin	c1d1be2a4b	Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields ... config-entry: remove Kind and Name field from Mesh config entry	4 years ago
Luke Kysow	c816e29ef7	Give descriptive error if auth method not found (#10163) ... * Give descriptive error if auth method not found Previously during a `consul login -method=blah`, if the auth method was not found, the error returned would be "ACL not found". This is potentially confusing because there may be many different ACLs involved in a login: the ACL of the Consul client, perhaps the binding rule or the auth method. Now the error will be "auth method blah not found", which is much easier to debug.	4 years ago
R.B. Boyer	145a83e436	connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101) ... The only thing that needed fixing up pertained to this section of the 1.18.x release notes: > grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default. For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default. Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.	4 years ago
R.B. Boyer	6a39b47448	Support Incremental xDS mode (#9855) ... This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged. Union of all commit messages follows to give an overarching summary: xds: exclusively support incremental xDS when using xDS v3 Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail. Work around a strange older envoy behavior involving empty CDS responses over incremental xDS. xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead. Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client. xds: pull out checkStreamACLs method in advance of a later commit xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty. This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10. xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time	4 years ago
Iryna Shustava	f383452e87	Implement traffic redirection exclusion based on proxy config and user-provided values (#10134) ... * Use proxy outbound port from TransparentProxyConfig if provided * If -proxy-id is provided to the redirect-traffic command, exclude any listener ports from inbound traffic redirection. This includes envoy_prometheus_bind_addr, envoy_stats_bind_addr, and the ListenerPort from the Expose configuration. * Allow users to provide additional inbound and outbound ports, outbound CIDRs and additional user IDs to be excluded from traffic redirection. This affects both the traffic-redirect command and the iptables SDK package.	4 years ago
Freddy	c652580b5b	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
Paul Banks	54018da75f	Fix panic bug in snapshot inspect (#10091) ... * Fix panic bug in snapshot inspect * Add changelog entry * Update .changelog/10091.txt * Undo bad GitHub UI merge * Undo bad GitHub UI merge	4 years ago
Paul Banks	74c040498a	CLI: Allow snapshot inspect to work on internal raft snapshots directly. (#10089) ... * CLI: Add support for reading internal raft snapshots to snapshot inspect * Add snapshot inspect test for raw state files * Add changelog entry * Update .changelog/10089.txt	4 years ago
R.B. Boyer	4db8b78854	connect: update centralized upstreams representation in service-defaults (#10015)	4 years ago
freddygv	e1808af729	Fixup tests	4 years ago
freddygv	7cb3f32672	Convert new tproxy structs in api module into ptrs ... This way we avoid serializing these when empty. Otherwise users of the latest version of the api submodule cannot interact with older versions of Consul, because a new api client would send keys that the older Consul doesn't recognize yet.	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
Iryna Shustava	5755c97bc7	cli: Add new `consul connect redirect-traffic` command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910) ... * Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled. * Add new iptables package for applying traffic redirection rules with iptables.	4 years ago
Freddy	a02245b75a	Merge pull request #9976 from hashicorp/centralized-upstream-fixups	4 years ago
freddygv	ab752c1c86	Avoid sending zero-value upstream defaults from api	4 years ago
R.B. Boyer	5bcfe930c6	command: when generating envoy bootstrap configs to stdout do not mix informational logs into the json (#9980) ... Fixes #9921	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
Hans Hasselberg	53e9c134af	introduce certopts (#9606) ... * introduce cert opts * it should be using the same signer * lint and omit serial	4 years ago
woz5999	39f448589d	support env var expansion in envoy statsd urls ... Fixes #8561	4 years ago
Freddy	8207b832df	Add TransparentProxy option to proxy definitions	4 years ago
Freddy	c664938bae	Add per-upstream configuration to service-defaults	4 years ago
freddygv	8b46d8dcbb	Restore old Envoy prefix on escape hatches ... This is done because after removing ID and NodeName from ServiceConfigRequest we will no longer know whether a request coming in is for a Consul client earlier than v1.10.	4 years ago
Kyle Havlovitz	1e87c7183a	Merge pull request #9672 from hashicorp/ca-force-skip-xc ... connect/ca: Allow ForceWithoutCrossSigning for all providers	4 years ago
freddygv	6fd30d0384	Add TransparentProxy opt to proxy definition	4 years ago
freddygv	e3dc2a49df	Turn Limits and PassiveHealthChecks into pointers	4 years ago
Kyle Havlovitz	d62565f368	Merge pull request #9792 from dzeban/kv-import-prefix ... command/kv: Add prefix option to kv import command	4 years ago
freddygv	87cde19b4c	Create new types for service-defaults upstream cfg	4 years ago
Nitya Dhanushkodi	15e8b13891	Add flags to consul connect envoy for metrics merging. (#9768) ... Allows setting -prometheus-backend-port to configure the cluster envoy_prometheus_bind_addr points to. Allows setting -prometheus-scrape-path to configure which path envoy_prometheus_bind_addr exposes metrics on. -prometheus-backend-port is used by the consul-k8s metrics merging feature, to configure envoy_prometheus_bind_addr to point to the merged metrics endpoint that combines Envoy and service metrics so that one set of annotations on a Pod can scrape metrics from the service and it's Envoy sidecar. -prometheus-scrape-path is used to allow configurability of the path where prometheus metrics are exposed on envoy_prometheus_bind_addr.	4 years ago
R.B. Boyer	398b766532	xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658) ... - Also add support for envoy 1.17.0	4 years ago
R.B. Boyer	3b6ffc447b	xds: remove deprecated usages of xDS (#9602) ... Note that this does NOT upgrade to xDS v3. That will come in a future PR. Additionally: - Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated. - Shuffled some agent/xds imports in advance of a later xDS v3 upgrade. - Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR. Fixes #8425	4 years ago
Alex Dzyoba	098fd1797b	command/kv: Add prefix option to kv import command ... Currently when data is imported via `consul kv import` it overwrites keys under the root key. Since `consul kv export` can retrieve data for the given prefix, i.e. part of the KV tree, importing it under root may be not what users want. To mirror prefix behavior from export this PR adds prefix feature to the import command that adds prefix to all keys that are imported.	4 years ago
R.B. Boyer	6eeccc93ce	connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737)	4 years ago
Kyle Havlovitz	7dac583863	connect/ca: Allow ForceWithoutCrossSigning for all providers ... This allows setting ForceWithoutCrossSigning when reconfiguring the CA for any provider, in order to forcibly move to a new root in cases where the old provider isn't reachable or able to cross-sign for whatever reason.	4 years ago
Daniel Nephin	1dcafa51a4	config: make config.TestLoad_FullConfig use config.Load ... This commit makes a number of changes that should make TestLoad_FullConfig easier to work with, and make the test more like real world scenarios. * use separate files in testdata/ dir to store the config source. Separate files are much easier to edit because editors can syntax highlight json/hcl, and it makes strings easier to find. Previously trying to find strings would match strings used in other tests. * use the exported config.Load interface instead of internal NewBuilder and BuildAndValidate. * remove the tail config overrides, which are only necessary with nonZero works.	4 years ago
Daniel Nephin	32d36d0dd4	config: replace calls to config.NewBuilder with config.Load ... This is another incremental change to reduce config loading to a single small interface. All calls to NewBuilder can be replaced with Load.	4 years ago
Daniel Nephin	97a577502d	config: improve the interface of Load ... This commit reduces the interface to Load() a bit, in preparation for unexporting NewBuilder and having everything call Load. The three arguments are reduced to a single argument by moving the other two into the options struct. The three return values are reduced to two by moving the RuntimeConfig and Warnings into a LoadResult struct.	4 years ago
Hans Hasselberg	444cdeb8fb	Add flags to support CA generation for Connect (#9585)	4 years ago
Daniel Nephin	3d39359bcb	agent: move deprecated AddServiceFromSource to a test file ... The method is only used in tests, and only exists for legacy calls. There was one other package which used this method in tests. Export the AddServiceRequest and a couple of its fields so the new function can be used in those tests.	4 years ago
Daniel Nephin	6757231b82	agent: rename AddService->AddServiceFromSource ... In preparation for extracting a single AddService func that accepts a request struct.	4 years ago