2296 Commits (30112288c893a43e545ff7d446f05be15c037093)

Author	SHA1	Message	Date
Dhia Ayachi	fe8b3dfccf	add partition flag to catalog commands (#10949) ... * add partition flag to catalog commands * add missing files	3 years ago
R.B. Boyer	097e1645e3	agent: ensure that most agent behavior correctly respects partition configuration (#10880)	3 years ago
Daniel Nephin	797ee061e4	debug: use human readable dates for filenames ... The unix timestamps that were used make the debug data a little bit more difficult to consume. By using human readable dates we can easily see when the profile data was collected. This commit also improves the test coverage. Two test cases are removed and the assertions from those cases are moved to TestDebugCommand. Now TestDebugCommand is able to validate the contents of all files. This change reduces the test runtime of the command/debug package by almost 50%. It also makes much more strict assertions about the contents by using gotest.tools/v3/fs.	3 years ago
Daniel Nephin	2f8d0e12cf	debug: small cleanup ... Use the new WriteJsonFile function to write index.json Remove .String() from time.local() since that is done by %s Remove an unused field.	3 years ago
Daniel Nephin	4359e38114	debug: restore cancel on SigInt ... Some previous changes broke interrupting the debug on SigInterupt. This change restores the original behaviour by passing a context to requests. Since a new API client function was required to pass the context, I had it also return an io.ReadCloser, so that output can be streamed to files instead of fully buffering in process memory.	3 years ago
Daniel Nephin	31bcd80528	debug: improve a couple of the test cases ... Use gotest.tools/v3/fs to make better assertions about the files Remove the TestAgent from TestDebugCommand_Prepare_ValidateTiming, since we can test that validation without making any API calls.	3 years ago
Daniel Nephin	bbf6a94c9a	debug: rename cluster target to members ... The API is called members. Using the same name as the API should help describe the contents of the file.	3 years ago
Daniel Nephin	251026e374	debug: remove unused	3 years ago
Daniel Nephin	70c2cdc8f1	cli: remove a test case for updating a legacy token ... Legacy tokens are no longer accepted, so we don't need to test their upgrade path.	3 years ago
Mark Anderson	d3cebbd32c	Fixup to support unix domain socket via command line (#10758) ... Missed the need to add support for unix domain socket config via api/command line. This is a variant of the problems described in it is easy to drop one. Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Blake Covarrubias	1ee8655bfc	cli: Fix broken KV import on Windows (#10820) ... Consul 1.10 (PR #9792) introduced the ability to specify a prefix when importing KV's. This however introduced a regression on Windows systems which breaks `kv import`. The key name is joined with specified`-prefix` using `filepath.Join()` which uses a forward slash (/) to delimit values on Unix-based systems, and a backslash (\) to delimit values on Windows – the latter of which is incompatible with Consul KV paths. This commit replaces filepath.Join() with path.Join() which uses a forward slash as the delimiter, providing consistent key join behavior across supported operating systems. Fixes #10583	3 years ago
Blake Covarrubias	e41d6ee60f	cli: Use admin bind address in self_admin cluster (#10757) ... Configure the self_admin cluster to use the admin bind address provided when starting Envoy. Fixes #10747	3 years ago
Blake Covarrubias	6a68bfc5e1	cli: Test API access using /status/leader in consul watch (#10795) ... Replace call to /agent/self with /status/leader to verify agent reachability before initializing a watch. This endpoint is not guarded by ACLs, and as such can be queried by any API client regardless of their permissions. Fixes #9353	3 years ago
Daniel Nephin	9dd6d26d05	acl: remove rule == nil checks	3 years ago
Evan Culver	24db06f503	Fix maint test	3 years ago
Daniel Nephin	beea1c2218	http: emit indented JSON in the metrics stream endpoint ... To remove the need to decode and re-encode in the CLI	3 years ago
Daniel Nephin	c3149ec0fd	debug: use the new metrics stream in debug command	3 years ago
Dhia Ayachi	de124d0aa1	add http flag for admin partition (#10683)	3 years ago
R.B. Boyer	b0657973f2	add partition cli flag to all cli commands that have namespace flag (#10668)	3 years ago
Blake Covarrubias	6c462d399b	cli: Document pass-through option for `consul connect envoy` (#10666) ... Update help text of `consul connect envoy` command to mention the ability to provide pass-through options.	3 years ago
Evan Culver	0527dcff57	acls: Show `AuthMethodNamespace` when reading/listing ACL token meta (#10598)	3 years ago
Daniel Nephin	74fb650b6b	Merge pull request #10588 from hashicorp/dnephin/config-fix-ports-grpc ... config: rename `ports.grpc` to `ports.xds`	3 years ago
Daniel Nephin	233d03dbbd	Apply suggestions from code review ... Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
Daniel Nephin	4ad80ccee3	command/envoy: stop using the DebugConfig from Self endpoint ... The DebugConfig in the self endpoint can change at any time. It's not a stable API. With the previous change to rename GRPCPort to XDSPort this command would have broken. This commit adds the XDSPort to a stable part of the XDS api, and changes the envoy command to read this new field. It includes support for the old API as well, in case a newer CLI is used with an older API, and adds a test for both cases.	3 years ago
Iryna Shustava	95305881ce	cli/sdk: Allow applying redirect-traffic rules in a provided Linux namespace (#10564)	3 years ago
Daniel Nephin	895bf9adec	config: update GRPCPort and addr in runtime config	3 years ago
Evan Culver	13bd86527b	Add support for returning ACL secret IDs for accessors with acl:write (#10546)	3 years ago
R.B. Boyer	c94b8c6a39	config: add agent config flag for enterprise clients to indicate they wish to join a particular partition (#10572)	3 years ago
Daniel Nephin	2c4f22a9f0	Merge pull request #10552 from hashicorp/dnephin/ca-remove-rotation-period ... ca: remove unused RotationPeriod field	3 years ago
jkirschner-hashicorp	5f73de6fbc	Merge pull request #10560 from jkirschner-hashicorp/change-sane-to-reasonable ... Replace use of 'sane' where appropriate	3 years ago
Daniel Nephin	3a045cca8d	ca: remove unused RotationPeriod field ... This field was never used. Since it is persisted as part of a map[string]interface{} it is pretty easy to remove it.	3 years ago
Jared Kirschner	bd536151e1	Replace use of 'sane' where appropriate ... HashiCorp voice, style, and language guidelines recommend avoiding ableist language unless its reference to ability is accurate in a particular use.	3 years ago
Daniel Nephin	690dc41c55	Merge pull request #10515 from hashicorp/dnephin/fix-arm32-atomic-aligment ... Fix panic on 32-bit platforms	3 years ago
Daniel Nephin	4d741531b4	Update references to the main branch ... The main branch is being renamed from master->main. This commit should update all references to the main branch to the new name. Co-Authored-By: Mike Morris <mikemorris@users.noreply.github.com>	3 years ago
Daniel Nephin	f34d3543b1	testing: fix a test for 32-bit ... The hcl decoding apparently uses strconv.ParseInt, which fails to parse a 64bit int. Since hcl v1 is basically EOl, it seems unlikely we'll fix this in hcl. Since this test is only about loading values from config files, the extra large number doesn't seem important. Trim a few zeros from the numbers so that they parse properly on 32bit platforms. Also skip a slow test when -short is used.	3 years ago
Kyle Nusbaum	07cec75be2	command/agent: change io.Discard to ioutil.Discard	3 years ago
Freddy	ffb13f35f1	Rename CatalogDestinationsOnly (#10397) ... CatalogDestinationsOnly is a passthrough that would enable dialing addresses outside of Consul's catalog. However, when this flag is set to true only _connect_ endpoints for services can be dialed. This flag is being renamed to signal that non-Connect endpoints can't be dialed by transparent proxies when the value is set to true.	4 years ago
Freddy	429f9d8bb8	Add flag for transparent proxies to dial individual instances (#10329)	4 years ago
Dhia Ayachi	005ad9e46d	generate a single debug file for a long duration capture (#10279) ... * debug: remove the CLI check for debug_enabled The API allows collecting profiles even debug_enabled=false as long as ACLs are enabled. Remove this check from the CLI so that users do not need to set debug_enabled=true for no reason. Also: - fix the API client to return errors on non-200 status codes for debug endpoints - improve the failure messages when pprof data can not be collected Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com> * remove parallel test runs parallel runs create a race condition that fail the debug tests * snapshot the timestamp at the beginning of the capture - timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures - capture append to the file if it already exist * Revert "snapshot the timestamp at the beginning of the capture" This reverts commit c2d03346 * Refactor captureDynamic to extract capture logic for each item in a different func * snapshot the timestamp at the beginning of the capture - timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures - capture append to the file if it already exist * Revert "snapshot the timestamp at the beginning of the capture" This reverts commit c2d03346 * Refactor captureDynamic to extract capture logic for each item in a different func * extract wait group outside the go routine to avoid a race condition * capture pprof in a separate go routine * perform a single capture for pprof data for the whole duration * add missing vendor dependency * add a change log and fix documentation to reflect the change * create function for timestamp dir creation and simplify error handling * use error groups and ticker to simplify interval capture loop * Logs, profile and traces are captured for the full duration. Metrics, Heap and Go routines are captured every interval * refactor Logs capture routine and add log capture specific test * improve error reporting when log test fail * change test duration to 1s * make time parsing in log line more robust * refactor log time format in a const * test on log line empty the earliest possible and return Co-authored-by: Freddy <freddygv@users.noreply.github.com> * rename function to captureShortLived * more specific changelog Co-authored-by: Paul Banks <banks@banksco.de> * update documentation to reflect current implementation * add test for behavior when invalid param is passed to the command * fix argument line in test * a more detailed description of the new behaviour Co-authored-by: Paul Banks <banks@banksco.de> * print success right after the capture is done * remove an unnecessary error check Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> * upgraded github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57 => v0.0.0-20210601050228-01bbb1931b22 Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com> Co-authored-by: Paul Banks <banks@banksco.de>	4 years ago
Dhia Ayachi	dda3e68791	fix monitor to only start the monitor in json format when requested (#10358) ... * fix monitor to only start the monitor in json format when requested * add release notes * add test to validate json format when requested	4 years ago
Daniel Nephin	cec8bc88a9	cmd: remove unnecessary GatedUi ... The intent of this struct was to prevent non-json output to stdout. With the previous cleanup, this can now be done by simply changing the stdout stream to io.Discard. This is just one example of why passing around io.Writers for the streams is better than the UI interface.	4 years ago
Daniel Nephin	2261a469e3	cmd: move agent running message to logs ... Previously this line was mixed up with logging, which made the output quite ugly. Use the logger to output this message, instead of printing directly to stdout. This has the advantage that the message will be visible when json logs are enabled.	4 years ago
Daniel Nephin	b4b85bd83a	agent: fix agent logging ... Remove the leading whitespace on every log line. This was causing problems for a customer because their logging system was interpretting the logs as a single multi-line log.	4 years ago
Daniel Nephin	2fc988d51d	cmd: introduce a shim to expose Stdout/Stderr writers ... This will allow commands to do the right thing, and write to the proper output stream.	4 years ago
Daniel Nephin	e573641995	cmd: remove unnecessary args to agent.New ... The version args are static and passed in from the caller. Instead read the static values in New. The shutdownCh was never closed, so did nothing. Remove it as a field and an arg.	4 years ago
Daniel Nephin	eb4f8b17e9	Merge pull request #10324 from hashicorp/dnephin/fix-envoy-bootstrap-exec ... envoy: fix deadlock when input is larger than named pipe buffer size	4 years ago
Dhia Ayachi	15dddc9edb	make tests use a dummy node_name to avoid environment related failures (#10262) ... * fix tests to use a dummy nodeName and not fail when hostname is not a valid nodeName * remove conditional testing * add test when node name is invalid	4 years ago
Daniel Nephin	2054402a53	envoy: improve comments	4 years ago
Daniel Nephin	c9bc5f92b7	envoy: fix bootstrap deadlock caused by a full named pipe ... Normally the named pipe would buffer up to 64k, but in some cases when a soft limit is reached, they will start only buffering up to 4k. In either case, we should not deadlock. This commit changes the pipe-bootstrap command to first buffer all of stdin into the process, before trying to write it to the named pipe. This allows the process memory to act as the buffer, instead of the named pipe. Also changed the order of operations in `makeBootstrapPipe`. The new test added in this PR showed that simply buffering in the process memory was not enough to fix the issue. We also need to ensure that the `pipe-bootstrap` process is started before we try to write to its stdin. Otherwise the write will still block. Also set stdout/stderr on the subprocess, so that any errors are visible to the user.	4 years ago
Daniel Nephin	e1b1ab7ef6	envoy: start timeout func after validation ... This removes the need to check arg length in the timeout function.	4 years ago
Dhia Ayachi	4c7f5f31c7	debug: remove the CLI check for debug_enabled (#10273) ... * debug: remove the CLI check for debug_enabled The API allows collecting profiles even debug_enabled=false as long as ACLs are enabled. Remove this check from the CLI so that users do not need to set debug_enabled=true for no reason. Also: - fix the API client to return errors on non-200 status codes for debug endpoints - improve the failure messages when pprof data can not be collected Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com> * remove parallel test runs parallel runs create a race condition that fail the debug tests * Add changelog Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>	4 years ago
Daniel Nephin	5a7059f45c	redirecttraffic: fix a flaky test ... https://app.circleci.com/pipelines/github/hashicorp/consul-enterprise/6408/workflows/e90b1140-daa2-458f-8197-d1821e3693e3/jobs/94694/tests#failed-test-0 The ExcludeInboundPorts can be in a different order sometimes, although I'm not sure how. Also removes t.Parallel. We should only need to use t.Parallel in large packages with many slow tests. In this case there is only a single slow test so we don't get any benefit. Also add struct field names to the testcases list, so that it is easier to see what each value is doing, and to make it easier to add new fields in the future.	4 years ago
Iryna Shustava	d7d44f6ae7	Save exposed ports in agent's store and expose them via API (#10173) ... * Save exposed HTTP or GRPC ports to the agent's store * Add those the health checks API so we can retrieve them from the API * Change redirect-traffic command to also exclude those ports from inbound traffic redirection when expose.checks is set to true.	4 years ago
R.B. Boyer	3b50a55533	connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231)	4 years ago
Daniel Nephin	347f3d2128	Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields ... config-entry: remove Kind and Name field from Mesh config entry	4 years ago
Mark Anderson	06f0f79218	Continue working through proxy and agent ... Rework/listeners, rename makeListener Refactor, tests pass Signed-off-by: Mark Anderson <manderson@hashicorp.com>	4 years ago
Luke Kysow	8d6cbe7281	Give descriptive error if auth method not found (#10163) ... * Give descriptive error if auth method not found Previously during a `consul login -method=blah`, if the auth method was not found, the error returned would be "ACL not found". This is potentially confusing because there may be many different ACLs involved in a login: the ACL of the Consul client, perhaps the binding rule or the auth method. Now the error will be "auth method blah not found", which is much easier to debug.	4 years ago
Daniel Nephin	a07a58a873	config-entry: use custom MarshalJSON for mesh type ... So that the Kind field is added to the JSON object.	4 years ago
Daniel Nephin	62efaaab21	config-entry: remove Kind and Name field from Mesh config entry ... No config entry needs a Kind field. It is only used to determine the Go type to target. As we introduce new config entries (like this one) we can remove the kind field and have the GetKind method return the single supported value. In this case (similar to proxy-defaults) the Name field is also unnecessary. We always use the same value. So we can omit the name field entirely.	4 years ago
R.B. Boyer	abc1dc0fe9	connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101) ... The only thing that needed fixing up pertained to this section of the 1.18.x release notes: > grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default. For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default. Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.	4 years ago
R.B. Boyer	71d45a3460	Support Incremental xDS mode (#9855) ... This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged. Union of all commit messages follows to give an overarching summary: xds: exclusively support incremental xDS when using xDS v3 Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail. Work around a strange older envoy behavior involving empty CDS responses over incremental xDS. xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead. Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client. xds: pull out checkStreamACLs method in advance of a later commit xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty. This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10. xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time	4 years ago
Iryna Shustava	8dffb89131	Implement traffic redirection exclusion based on proxy config and user-provided values (#10134) ... * Use proxy outbound port from TransparentProxyConfig if provided * If -proxy-id is provided to the redirect-traffic command, exclude any listener ports from inbound traffic redirection. This includes envoy_prometheus_bind_addr, envoy_stats_bind_addr, and the ListenerPort from the Expose configuration. * Allow users to provide additional inbound and outbound ports, outbound CIDRs and additional user IDs to be excluded from traffic redirection. This affects both the traffic-redirect command and the iptables SDK package.	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
Paul Banks	c501468d78	Fix panic bug in snapshot inspect (#10091) ... * Fix panic bug in snapshot inspect * Add changelog entry * Update .changelog/10091.txt * Undo bad GitHub UI merge * Undo bad GitHub UI merge	4 years ago
Paul Banks	d717d2cdc4	CLI: Allow snapshot inspect to work on internal raft snapshots directly. (#10089) ... * CLI: Add support for reading internal raft snapshots to snapshot inspect * Add snapshot inspect test for raw state files * Add changelog entry * Update .changelog/10089.txt	4 years ago
R.B. Boyer	4db8b78854	connect: update centralized upstreams representation in service-defaults (#10015)	4 years ago
freddygv	e1808af729	Fixup tests	4 years ago
freddygv	7cb3f32672	Convert new tproxy structs in api module into ptrs ... This way we avoid serializing these when empty. Otherwise users of the latest version of the api submodule cannot interact with older versions of Consul, because a new api client would send keys that the older Consul doesn't recognize yet.	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
Iryna Shustava	5755c97bc7	cli: Add new `consul connect redirect-traffic` command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910) ... * Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled. * Add new iptables package for applying traffic redirection rules with iptables.	4 years ago
Freddy	a02245b75a	Merge pull request #9976 from hashicorp/centralized-upstream-fixups	4 years ago
freddygv	ab752c1c86	Avoid sending zero-value upstream defaults from api	4 years ago
R.B. Boyer	5bcfe930c6	command: when generating envoy bootstrap configs to stdout do not mix informational logs into the json (#9980) ... Fixes #9921	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
Hans Hasselberg	53e9c134af	introduce certopts (#9606) ... * introduce cert opts * it should be using the same signer * lint and omit serial	4 years ago
woz5999	39f448589d	support env var expansion in envoy statsd urls ... Fixes #8561	4 years ago
Freddy	8207b832df	Add TransparentProxy option to proxy definitions	4 years ago
Freddy	c664938bae	Add per-upstream configuration to service-defaults	4 years ago
freddygv	8b46d8dcbb	Restore old Envoy prefix on escape hatches ... This is done because after removing ID and NodeName from ServiceConfigRequest we will no longer know whether a request coming in is for a Consul client earlier than v1.10.	4 years ago
Kyle Havlovitz	1e87c7183a	Merge pull request #9672 from hashicorp/ca-force-skip-xc ... connect/ca: Allow ForceWithoutCrossSigning for all providers	4 years ago
freddygv	6fd30d0384	Add TransparentProxy opt to proxy definition	4 years ago
freddygv	e3dc2a49df	Turn Limits and PassiveHealthChecks into pointers	4 years ago
Kyle Havlovitz	d62565f368	Merge pull request #9792 from dzeban/kv-import-prefix ... command/kv: Add prefix option to kv import command	4 years ago
freddygv	87cde19b4c	Create new types for service-defaults upstream cfg	4 years ago
Nitya Dhanushkodi	15e8b13891	Add flags to consul connect envoy for metrics merging. (#9768) ... Allows setting -prometheus-backend-port to configure the cluster envoy_prometheus_bind_addr points to. Allows setting -prometheus-scrape-path to configure which path envoy_prometheus_bind_addr exposes metrics on. -prometheus-backend-port is used by the consul-k8s metrics merging feature, to configure envoy_prometheus_bind_addr to point to the merged metrics endpoint that combines Envoy and service metrics so that one set of annotations on a Pod can scrape metrics from the service and it's Envoy sidecar. -prometheus-scrape-path is used to allow configurability of the path where prometheus metrics are exposed on envoy_prometheus_bind_addr.	4 years ago
R.B. Boyer	398b766532	xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658) ... - Also add support for envoy 1.17.0	4 years ago
R.B. Boyer	3b6ffc447b	xds: remove deprecated usages of xDS (#9602) ... Note that this does NOT upgrade to xDS v3. That will come in a future PR. Additionally: - Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated. - Shuffled some agent/xds imports in advance of a later xDS v3 upgrade. - Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR. Fixes #8425	4 years ago
Alex Dzyoba	098fd1797b	command/kv: Add prefix option to kv import command ... Currently when data is imported via `consul kv import` it overwrites keys under the root key. Since `consul kv export` can retrieve data for the given prefix, i.e. part of the KV tree, importing it under root may be not what users want. To mirror prefix behavior from export this PR adds prefix feature to the import command that adds prefix to all keys that are imported.	4 years ago
R.B. Boyer	6eeccc93ce	connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737)	4 years ago
Kyle Havlovitz	7dac583863	connect/ca: Allow ForceWithoutCrossSigning for all providers ... This allows setting ForceWithoutCrossSigning when reconfiguring the CA for any provider, in order to forcibly move to a new root in cases where the old provider isn't reachable or able to cross-sign for whatever reason.	4 years ago
Daniel Nephin	1dcafa51a4	config: make config.TestLoad_FullConfig use config.Load ... This commit makes a number of changes that should make TestLoad_FullConfig easier to work with, and make the test more like real world scenarios. * use separate files in testdata/ dir to store the config source. Separate files are much easier to edit because editors can syntax highlight json/hcl, and it makes strings easier to find. Previously trying to find strings would match strings used in other tests. * use the exported config.Load interface instead of internal NewBuilder and BuildAndValidate. * remove the tail config overrides, which are only necessary with nonZero works.	4 years ago
Daniel Nephin	32d36d0dd4	config: replace calls to config.NewBuilder with config.Load ... This is another incremental change to reduce config loading to a single small interface. All calls to NewBuilder can be replaced with Load.	4 years ago
Daniel Nephin	97a577502d	config: improve the interface of Load ... This commit reduces the interface to Load() a bit, in preparation for unexporting NewBuilder and having everything call Load. The three arguments are reduced to a single argument by moving the other two into the options struct. The three return values are reduced to two by moving the RuntimeConfig and Warnings into a LoadResult struct.	4 years ago
Hans Hasselberg	444cdeb8fb	Add flags to support CA generation for Connect (#9585)	4 years ago
Daniel Nephin	3d39359bcb	agent: move deprecated AddServiceFromSource to a test file ... The method is only used in tests, and only exists for legacy calls. There was one other package which used this method in tests. Export the AddServiceRequest and a couple of its fields so the new function can be used in those tests.	4 years ago
Daniel Nephin	6757231b82	agent: rename AddService->AddServiceFromSource ... In preparation for extracting a single AddService func that accepts a request struct.	4 years ago
R.B. Boyer	b5f9a293c9	chore: regenerate envoy golden files (#9634)	4 years ago
Matt Keeler	9215fc4e9d	Fix flaky CLI Intention Listing Test	4 years ago
Michael Hofer	3c3b67288d	cli: Add consul intention list command (based on PR #6825) (#9468) ... This PR is based on the previous work by @snuggie12 in PR #6825. It adds the command consul intention list to list all available intentions. The list functionality for intentions seems a bit overdue as it's just very handy. The web UI cannot list intentions outside of the default namespace, and using the API is sometimes not the friendliest option. ;) I cherry picked snuggie12's commits who did most of the heavy lifting (thanks again @snuggie12 for your great work!). The changes in the original commit mostly still worked on the current HEAD. On top of that I added support for namespaces and fixed the docs as they are managed differently today. Also the requested changes related to the "Connect" references in the original PRs have been addressed. Fixes #5652 Co-authored-by: Matt Hoey <mhoey05@jcu.edu>	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
R.B. Boyer	7c7a3e5165	command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229) ... Fixes #9215	4 years ago
Freddy	fe728855ed	Add DC and NS support for Envoy metrics (#9207) ... This PR updates the tags that we generate for Envoy stats. Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.	4 years ago
Mike Morris	7af643ac37	ci: update to Go 1.15.4 and alpine:3.12 (#9036) ... * ci: stop building darwin/386 binaries Go 1.15 drops support for 32-bit binaries on Darwin https://golang.org/doc/go1.15#darwin * tls: ConnectionState::NegotiatedProtocolIsMutual is deprecated in Go 1.15, this value is always true * correct error messages that changed slightly * Completely regenerate some TLS test data Co-authored-by: R.B. Boyer <rb@hashicorp.com>	4 years ago
Matt Keeler	7ef9b04f90	Add a CLI command for retrieving the autopilot configuration. (#9142)	4 years ago
Joel Watson	81fb937e4f	Merge pull request #9098 from hashicorp/watsonian/kv-size-breakdown ... Add detailed key size breakdown to snapshot inspect	4 years ago
Joel Watson	182333b645	Fix some minor wording issues	4 years ago
Joel Watson	94689b6ecd	Back out implicit -kvdetails functionality	4 years ago
Joel Watson	0d48559416	Length check is required here ... If one isn't included, then the nil check in the formatter never fails due to an empty slice being passed in, which causes the kv output to always get printed.	4 years ago
Joel Watson	48a358df1b	Remove trailing tabs from output	4 years ago
Joel Watson	491f2f0a57	Add more func comments	4 years ago
Joel Watson	99ec1febcc	Move kvDetails default logic	4 years ago
Joel Watson	ea804e57b4	Check for nil rather than length	4 years ago
Joel Watson	7aebd179de	Update tests for new flag names	4 years ago
Joel Watson	40a8681816	Allow omission of -kvdetails if another -kv* flag is set	4 years ago
Joel Watson	1ef259b093	Rename params to better reflect their purpose	4 years ago
Joel Watson	5ad0db73c8	Make docs for params clearer	4 years ago
Joel Watson	fbe8503258	Break KV portion of enchance into separate func	4 years ago
Joel Watson	5dd703a62a	Move KV stat gen to separate func	4 years ago
Matt Keeler	c048e86bb2	Switch to using the external autopilot module	4 years ago
Mike Morris	6396042ba7	connect: switch the default gateway port from 443 to 8443 (#9116) ... * test: update ingress gateway golden file to port 8443 * test: update Envoy flags_test to port 8443 Co-authored-by: R.B. Boyer <rb@hashicorp.com>	4 years ago
R.B. Boyer	8baf158ea8	Revert "Add namespace support for metrics (OSS) (#9117)" (#9124) ... This reverts commit 06b3b017d3.	4 years ago
Freddy	06b3b017d3	Add namespace support for metrics (OSS) (#9117)	4 years ago
Joel Watson	aabb537aed	Pull sorting into separate function	4 years ago
Joel Watson	3ee20d500a	Refactor to reduce how many vars are being passed around	4 years ago
Joel Watson	1a50aa023a	Fallback to alphabetic sorting if size is equal	4 years ago
Joel Watson	be2a212d75	Add tests for new snapshot inspect flags	4 years ago
Joel Watson	5f5b49955c	Update snapshot inspect formatter test	4 years ago
Joel Watson	eafc593d0f	Make key breakdown total size accurate	4 years ago
Joel Watson	4bf70dd857	Cleanup formatter	4 years ago
Joel Watson	208d6ebd95	Get JSON formatting working	4 years ago
Joel Watson	e7a88d4a30	This ended up not being used.	4 years ago
Joel Watson	262fb3a0c3	Add snapshot inspect filter param	4 years ago
Joel Watson	10f325dc81	Initial stab at snapshot inspect key breakdown	4 years ago
s-christoff	79ce24e9fc	cli: Add JSON and Pretty Print formatting for `consul snapshot inspect` (#9006)	4 years ago
R.B. Boyer	a2c50d3303	connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944) ... Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"	4 years ago
s-christoff	9bb348c6c7	Enhance the output of consul snapshot inspect (#8787)	4 years ago
Blake Covarrubias	4c83ad3e02	doc: Update acl-method command example (#8845) ... * Update acl-method command example * add tailing backtick	4 years ago
R.B. Boyer	9fbcb2e68d	command: remove conditional envoy bootstrap generation for versions <=1.10.0 since those are not supported (#8855)	4 years ago
Ryan Ooi	852896b068	add tailing backtick	4 years ago
Ryan Ooi	c80b048111	Update acl-method command example	4 years ago
R.B. Boyer	1b413b0444	connect: support defining intentions using layer 7 criteria (#8839) ... Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834) ... - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
Hans Hasselberg	ad4e189354	add -list-primary to `consul keyring` command (#8692) ... * add -list-primary * add docs * use builder * fix multiple actions	4 years ago
Juliano Martinez	5ce309a2f7	remove nodeName call when using -service ... - fix #8734	4 years ago
freddygv	7b9d1b41d5	Resolve conflicts against master	4 years ago
freddygv	768dbaa68d	Add session flag to cookie config	4 years ago
freddygv	eab90ea9fa	Revert EnvoyConfig nesting	4 years ago
Tim Arenz	a1fe711390	Add support for -ca-path option in the connect envoy command (#8606) ... * Add support for -ca-path option in the connect envoy command * Adding changelog entry	4 years ago
Daniel Nephin	6ca45e1a61	agent: add apiServers type for managing HTTP servers ... Remove Server field from HTTPServer. The field is no longer used.	4 years ago
freddygv	cf018cb2e0	Fixup stray LB infix refs	4 years ago
freddygv	f81fe6a1a1	Remove LB infix and move injection to xds	4 years ago
R.B. Boyer	119e945c3e	connect: all config entries pick up a meta field (#8596) ... Fixes #8595	4 years ago
freddygv	63f79e5f9b	Restructure structs and other PR comments	4 years ago
Daniel Nephin	e16375216d	config: use logging.Config in RuntimeConfig ... To add structure to RuntimeConfig, and remove the need to translate into a third type.	4 years ago
Daniel Nephin	33c401a16e	logging: Setup accept io.Writer instead of []io.Writer ... Also accept a non-pointer Config, since the config is not modified	4 years ago
Daniel Nephin	5d4df54296	agent: extract dependency creation from New ... With this change, Agent.New() accepts many of the dependencies instead of creating them in New. Accepting fully constructed dependencies from a constructor makes the type easier to test, and easier to change. There are still a number of dependencies created in Start() which can be addressed in a follow up.	4 years ago
Daniel Nephin	16217fe9b9	testing: use t.Cleanup in testutil.TempFile ... So that it has the same behaviour as TempDir. Also remove the now unnecessary 'defer os.Remove'	4 years ago
Daniel Nephin	d68edcecf4	testing: Remove all the defer os.Removeall ... Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage the removal	4 years ago
R.B. Boyer	397019d970	xds: revert setting set_node_on_first_message_only to true when generating envoy bootstrap config (#8440) ... When consul is restarted and an envoy that had already sent DiscoveryRequests to the previous consul process sends a request to the new process it doesn't respect the setting and never populates DiscoveryRequest.Node for the life of the new consul process due to this bug: https://github.com/envoyproxy/envoy/issues/9682 Fixes #8430	4 years ago
Daniel Nephin	0420d91cdd	Remove LogOutput from Agent ... Now that it is no longer used, we can remove this unnecessary field. This is a pre-step in cleanup up RuntimeConfig->Consul.Config, which is a pre-step to adding a gRPCHandler component to Server for streaming. Removing this field also allows us to remove one of the return values from logging.Setup.	4 years ago
R.B. Boyer	c599a2f5f4	xds: add support for envoy 1.15.0 and drop support for 1.11.x (#8424) ... Related changes: - hard-fail the xDS connection attempt if the envoy version is known to be too old to be supported - remove the RouterMatchSafeRegex proxy feature since all supported envoy versions have it - stop using --max-obj-name-len (due to: envoyproxy/envoy#11740)	4 years ago
Marc Billow	6960afdfe3	Simple tls cert create help text typo	4 years ago
Chris Piraino	7c4cc71131	Fix envoy bootstrap logic to not append multiple self_admin clusters (#8371) ... Previously, the envoy bootstrap config would blindly copy the self_admin cluster into the list of static clusters when configuring either ReadyBindAddr, PrometheusBindAddr, or StatsBindAddr. Since ingress gateways always configure the ReadyBindAddr property, users ran into this case much more often than previously.	4 years ago
Alvin Huang	dd9f5928fc	add v to version pretty formatter (#8341) ... * add v to version pretty formatter * remove v from json version output	4 years ago
Kit Patella	0fe7e987f0	command/kv: remove error case in put and edit error message for delete	4 years ago
Kit Patella	f530fe558f	command: fix cas put when index=0 and better errors in put and delete	4 years ago
Hans Hasselberg	496fb5fc5b	add support for envoy 1.14.4, 1.13.4, 1.12.6 (#8216)	4 years ago
Matt Keeler	fb9517bae8	Merge pull request #8268 from hashicorp/feature/improved-version-output ... Add Revision to version CLI output and add JSON support	4 years ago
R.B. Boyer	1eef096dfe	xds: version sniff envoy and switch regular expressions from 'regex' to 'safe_regex' on newer envoy versions (#8222) ... - cut down on extra node metadata transmission - split the golden file generation to compare all envoy version	4 years ago
Matt Keeler	680ac0c1c6	Add Revision to version CLI output and add JSON support ... Also add JSON format support	4 years ago
Chris Piraino	ea683ebb6c	cli: Output message on success when writing/deleting entries (#7806) ... This provides a user with a better experience, knowing that the command worked appropriately. The output of the write/delete CLI commands are not going to be used in a bash script, in fact previously a success provided no ouput, so we do not have to worry about spurious text being injected into bash pipelines.	4 years ago
R.B. Boyer	462f0f37ed	connect: various changes to make namespaces for intentions work more like for other subsystems (#8194) ... Highlights: - add new endpoint to query for intentions by exact match - using this endpoint from the CLI instead of the dump+filter approach - enforcing that OSS can only read/write intentions with a SourceNS or DestinationNS field of "default". - preexisting OSS intentions with now-invalid namespace fields will delete those intentions on initial election or for wildcard namespaces an attempt will be made to downgrade them to "default" unless one exists. - also allow the '-namespace' CLI arg on all of the intention subcommands - update lots of docs	4 years ago
Matt Keeler	e2cfa93f02	Don’t leak metrics go routines in tests (#8182)	4 years ago
freddygv	c791fbc79c	Update namespaces subject-verb agreement	4 years ago
Matt Keeler	d6e05482ab	Allow cancelling startup when performing auto-config (#8157) ... Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>	5 years ago
Matt Keeler	3dbbd2d37d	Implement Client Agent Auto Config ... There are a couple of things in here. First, just like auto encrypt, any Cluster.AutoConfig RPC will implicitly use the less secure RPC mechanism. This drastically modifies how the Consul Agent starts up and moves most of the responsibilities (other than signal handling) from the cli command and into the Agent.	5 years ago
Daniel Nephin	692a4a8fc8	Merge pull request #7762 from hashicorp/dnephin/warn-on-unknown-service-file ... config: warn if a config file is being skipped because of its file extension	5 years ago
Daniel Nephin	be29d6bf75	config: warn when a config file is skipped ... All commands which read config (agent, services, and validate) will now print warnings when one of the config files is skipped because it did not match an expected format. Also ensures that config validate prints all warnings.	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Matt Keeler	d3881dd754	ACL Node Identities (#7970) ... A Node Identity is very similar to a service identity. Its main targeted use is to allow creating tokens for use by Consul agents that will grant the necessary permissions for all the typical agent operations (node registration, coordinate updates, anti-entropy). Half of this commit is for golden file based tests of the acl token and role cli output. Another big updates was to refactor many of the tests in agent/consul/acl_endpoint_test.go to use the same style of tests and the same helpers. Besides being less boiler plate in the tests it also uses a common way of starting a test server with ACLs that should operate without any warnings regarding deprecated non-uuid master tokens etc.	5 years ago
Daniel Nephin	219790ca49	config: Make ConfigFormat not a pointer ... The nil value was never used. We can avoid a bunch of complications by making the field a string value instead of a pointer. This change is in preparation for fixing a silent config failure.	5 years ago
Daniel Nephin	77101eee82	config: rename Flags to BuilderOpts ... Flags is an overloaded term in this context. It generally is used to refer to command line flags. This struct, however, is a data object used as input to the construction. It happens to be partially populated by command line flags, but otherwise has very little to do with them. Renaming this struct should make the actual responsibility of this struct more obvious, and remove the possibility that it is confused with command line flags. This change is in preparation for adding additional fields to BuilderOpts.	5 years ago
Daniel Nephin	85e0338136	config: remove Args field from Flags ... This field was populated for one reason, to test that it was empty. Of all the callers, only a single one used this functionality. The rest constructed a `Flags{}` struct which did not set Args. I think this shows that the logic was in the wrong place. Only the agent command needs to care about validating the args. This commit removes the field, and moves the logic to the one caller that cares. Also fix some comments.	5 years ago
Hans Hasselberg	e62a43c6cf	Support envoy 1.14.2, 1.13.2, 1.12.4 (#8057)	5 years ago
Kyle Havlovitz	6fd3b25313	Fix a CLI test failure with namespaces in enterprise	5 years ago
Kyle Havlovitz	0c8966220f	Merge pull request #8040 from hashicorp/ingress/expose-cli ... Ingress expose CLI command	5 years ago
Kyle Havlovitz	e3a725c4e0	Always allow updating the exposed service and differentiate by namespace	5 years ago
Kyle Havlovitz	edab5588d8	Add -host flag to expose command	5 years ago
Kyle Havlovitz	5958328552	Allow multiple listeners per service via expose command	5 years ago
Daniel Nephin	75cbbe2702	config: add HookWeakDecodeFromSlice ... Currently opaque config blocks (config entries, and CA provider config) are modified by PatchSliceOfMaps, making it impossible for these opaque config sections to contain slices of maps. In order to fix this problem, any lazy-decoding of these blocks needs to support weak decoding of []map[string]interface{} to a struct type before PatchSliceOfMaps is replaces. This is necessary because these config blobs are persisted, and during an upgrade an older version of Consul could read one of the new configuration values, which would cause an error. To support the upgrade path, this commit first introduces the new hooks for weak decoding of []map[string]interface{} and uses them only in the lazy-decode paths. That way, in a future release, new style configuration will be supported by the older version of Consul. This decode hook has a number of advantages: 1. It no longer panics. It allows mapstructure to report the error 2. It no longer requires the user to declare which fields are slices of structs. It can deduce that information from the 'to' value. 3. It will make it possible to preserve opaque configuration, allowing for structured opaque config.	5 years ago
Kyle Havlovitz	acae044df4	Document the namespace format for expose CLI command	5 years ago
Kyle Havlovitz	b874c8ef0c	Add connect expose CLI command	5 years ago
R.B. Boyer	4d083103e2	fix some flaky snapshot tests (#8015)	5 years ago
R.B. Boyer	833211c14c	acl: allow auth methods created in the primary datacenter to optionally create global tokens (#7899)	5 years ago
R.B. Boyer	ffb9c7d6f7	acl: remove the deprecated `acl_enforce_version_8` option (#7991) ... Fixes #7292	5 years ago
Jono Sosulska	c554ba9e10	Replace whitelist/blacklist terminology with allowlist/denylist (#7971) ... * Replace whitelist/blacklist terminology with allowlist/denylist	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Daniel Nephin	6a2d7d77c0	config: use the new HookTranslateKeys instead of lib.TranslateKeys ... With the exception of CA provider config, which will be migrated at some later time.	5 years ago
Kyle Havlovitz	b14696e32a	Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924) ... * Standardize support for Tagged and BindAddresses in Ingress Gateways This updates the TaggedAddresses and BindAddresses behavior for Ingress to match Mesh/Terminating gateways. The `consul connect envoy` command now also allows passing an address without a port for tagged/bind addresses. * Update command/connect/envoy/envoy.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * PR comments * Check to see if address is an actual IP address * Update agent/xds/listeners.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * fix whitespace Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Daniel Nephin	c662f0f0de	Fix a number of problems found by staticcheck ... Some of these problems are minor (unused vars), but others are real bugs (ignored errors). Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago