656 Commits (e0a7900f52bf8d68206b2d870b557628c8df809f)

Author	SHA1	Message	Date
Daniel Nephin	62efaaab21	config-entry: remove Kind and Name field from Mesh config entry ... No config entry needs a Kind field. It is only used to determine the Go type to target. As we introduce new config entries (like this one) we can remove the kind field and have the GetKind method return the single supported value. In this case (similar to proxy-defaults) the Name field is also unnecessary. We always use the same value. So we can omit the name field entirely.	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
R.B. Boyer	4db8b78854	connect: update centralized upstreams representation in service-defaults (#10015)	4 years ago
freddygv	7cb3f32672	Convert new tproxy structs in api module into ptrs ... This way we avoid serializing these when empty. Otherwise users of the latest version of the api submodule cannot interact with older versions of Consul, because a new api client would send keys that the older Consul doesn't recognize yet.	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
freddygv	98ba582797	Fixup mesh gateway docs	4 years ago
Freddy	a02245b75a	Merge pull request #9976 from hashicorp/centralized-upstream-fixups	4 years ago
freddygv	d6db67ef86	Fixup test	4 years ago
Daniel Nephin	5d21d4d77f	Merge pull request #9925 from hashicorp/dnephin/update-memberlist ... Update memberlist to v0.2.3	4 years ago
freddygv	ab752c1c86	Avoid sending zero-value upstream defaults from api	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
Daniel Nephin	46279547ec	Update memberlist to v0.2.3 ... To pickup data race fixes	4 years ago
Seth Hoenig	cd1cd4febd	api: enable query options on agent endpoints ... This PR adds support for setting QueryOptions on a few agent API endpoints. Nomad needs to be able to set the Namespace field on these endpoints to: - query for services / checks in a namespace - deregister services / checks in a namespace - update TTL status on checks in a namespace	4 years ago
Freddy	8207b832df	Add TransparentProxy option to proxy definitions	4 years ago
Freddy	c664938bae	Add per-upstream configuration to service-defaults	4 years ago
Christopher Broglie	f0307c73e5	Add support for configuring TLS ServerName for health checks ... Some TLS servers require SNI, but the Golang HTTP client doesn't include it in the ClientHello when connecting to an IP address. This change adds a new TLSServerName field to health check definitions to optionally set it. This fixes #9473.	4 years ago
freddygv	6090cfcf68	PR comments	4 years ago
freddygv	8b46d8dcbb	Restore old Envoy prefix on escape hatches ... This is done because after removing ID and NodeName from ServiceConfigRequest we will no longer know whether a request coming in is for a Consul client earlier than v1.10.	4 years ago
freddygv	41b2ba1e58	Add omitempty across the board for UpstreamConfig	4 years ago
Matt Keeler	30903db442	AutopilotServerHealth now handles the 429 status code (#8599) ... AutopilotServerHealthy now handles the 429 status code Previously we would error out and not parse the response. Now either a 200 or 429 status code are considered expected statuses and will result in the method returning the reply allowing API consumers to not only see if the system is healthy or not but which server is unhealthy.	4 years ago
freddygv	23ffa3d3f3	And another test fix	4 years ago
Kyle Havlovitz	1e87c7183a	Merge pull request #9672 from hashicorp/ca-force-skip-xc ... connect/ca: Allow ForceWithoutCrossSigning for all providers	4 years ago
freddygv	6fd30d0384	Add TransparentProxy opt to proxy definition	4 years ago
freddygv	e3dc2a49df	Turn Limits and PassiveHealthChecks into pointers	4 years ago
freddygv	87cde19b4c	Create new types for service-defaults upstream cfg	4 years ago
Mark Anderson	b9d22f48cd	Add fields to the /acl/auth-methods endpoint. (#9741) ... * A GET of the /acl/auth-method/:name endpoint returns the fields MaxTokenTTL and TokenLocality, while a LIST (/acl/auth-methods) does not. The list command returns a filtered subset of the full set. This is somewhat deliberate, so that secrets aren't shown, but the TTL and Locality fields aren't (IMO) security critical, and it is useful for the front end to be able to show them. For consistency these changes mirror the 'omit empty' and string representation choices made for the GET call. This includes changes to the gRPC and API code in the client. The new output looks similar to this curl 'http://localhost:8500/v1/acl/auth-methods' | jq '.' { "MaxTokenTTL": "8m20s", "Name": "minikube-ttl-local2", "Type": "kubernetes", "Description": "minikube auth method", "TokenLocality": "local", "CreateIndex": 530, "ModifyIndex": 530, "Namespace": "default" } ] Signed-off-by: Mark Anderson <manderson@hashicorp.com> * Add changelog Signed-off-by: Mark Anderson <manderson@hashicorp.com>	4 years ago
Kyle Havlovitz	7dac583863	connect/ca: Allow ForceWithoutCrossSigning for all providers ... This allows setting ForceWithoutCrossSigning when reconfiguring the CA for any provider, in order to forcibly move to a new root in cases where the old provider isn't reachable or able to cross-sign for whatever reason.	4 years ago
Conor Mongey	7a368bd2b6	Move header methods from config to client	4 years ago
Conor Mongey	eb65e59741	Only override headers if they're set	4 years ago
Conor Mongey	f647569b84	Prefer http.Header over map[string]string to allow for multi-valued headers	4 years ago
Conor Mongey	cdc8cd7b0e	Allow setting arbitrary headers in API client	4 years ago
Daniel Nephin	5a7f4c0dea	Merge pull request #8609 from hashicorp/dnephin/add-query-options-to-ServiceRegister ... api: Add a context to ServiceRegisterOpts	4 years ago
Mike Morris	ab927d5480	Merge pull request #9270 from hashicorp/release/1.9.0 ... merge: release/1.9.0 back into 1.9.x	4 years ago
R.B. Boyer	7c7a3e5165	command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229) ... Fixes #9215	4 years ago
Matt Keeler	66fd23d67f	Refactor to call non-voting servers read replicas (#9191) ... Co-authored-by: Kit Patella <kit@jepsen.io>	4 years ago
Freddy	fe728855ed	Add DC and NS support for Envoy metrics (#9207) ... This PR updates the tags that we generate for Envoy stats. Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.	4 years ago
Matt Keeler	7f87adbcf4	Remove this constant as it is soon to be changing and we want to prevent backwards compat issues (#9193)	4 years ago
R.B. Boyer	61eac21f1a	agent: return the default ACL policy to callers as a header (#9101) ... Header is: X-Consul-Default-ACL-Policy=<allow|deny> This is of particular utility when fetching matching intentions, as the fallthrough for a request that doesn't match any intentions is to enforce using the default acl policy.	4 years ago
Matt Keeler	7ef9b04f90	Add a CLI command for retrieving the autopilot configuration. (#9142)	4 years ago
Matt Keeler	c048e86bb2	Switch to using the external autopilot module	4 years ago
R.B. Boyer	8baf158ea8	Revert "Add namespace support for metrics (OSS) (#9117)" (#9124) ... This reverts commit 06b3b017d3.	4 years ago
Freddy	06b3b017d3	Add namespace support for metrics (OSS) (#9117)	4 years ago
R.B. Boyer	e113dc0fe2	upstream some differences from enterprise (#8902)	4 years ago
Matt Keeler	38f5ddce2a	Add per-agent reconnect timeouts (#8781) ... This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it. This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.	4 years ago
R.B. Boyer	0c9177f6a5	api: unflake some intention-related api tests (#8857)	4 years ago
R.B. Boyer	1b413b0444	connect: support defining intentions using layer 7 criteria (#8839) ... Extend Consul’s intentions model to allow for request-based access control enforcement for HTTP-like protocols in addition to the existing connection-based enforcement for unspecified protocols (e.g. tcp).	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834) ... - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
R.B. Boyer	d2eb27e0a3	api: support GetMeta() and GetNamespace() on all config entry kinds (#8764) ... Fixes #8755 Since I was updating the interface, i also added the missing `GetNamespace()`. Depending upon how you look at it, this is a breaking change since it adds methods to the exported interface `api.ConfigEntry`. Given that you cannot define your own config entry kinds, and all of the machinery of the `api.Client` acts like a factory to construct the canned ones from the rest of the module, this feels like it's not a problematic change as it would only break someone who had reimplemented the `ConfigEntry` interface themselves for no apparent utility?	4 years ago
freddygv	7b9d1b41d5	Resolve conflicts against master	4 years ago
freddygv	768dbaa68d	Add session flag to cookie config	4 years ago
freddygv	eab90ea9fa	Revert EnvoyConfig nesting	4 years ago
Seth Hoenig	9fab3fe990	api: create fresh http client for unix sockets (#8602) ... Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	4 years ago
Freddy	c9c9e4face	Make LockDelay configurable in api locks (#8621)	4 years ago
Daniel Nephin	d083d4717b	api: Add a context to ServiceRegisterOpts ... So that operations can timeout instead of hanging forever	4 years ago
freddygv	eaa250cc80	Ensure resolver node with LB isn't considered default	4 years ago
freddygv	f81fe6a1a1	Remove LB infix and move injection to xds	4 years ago
R.B. Boyer	119e945c3e	connect: all config entries pick up a meta field (#8596) ... Fixes #8595	4 years ago
freddygv	63f79e5f9b	Restructure structs and other PR comments	4 years ago
freddygv	0236e169bb	Add documentation for resolver LB cfg	4 years ago
freddygv	ff56a64b08	Add LB policy to service-resolver	4 years ago
Jack	9e1c6727f9	Add http2 and grpc support to ingress gateways (#8458)	4 years ago
Matt Keeler	7c3914d89e	Add helpers to the API client to help with getting information from `AgentMember` tags (#8575) ... Lots of constants were added for various tags that would concern users and are not already parsed out. Additionally two methods on the AgentMember type were added to ask a member what its ACL Mode is and whether its a server or not.	4 years ago
Hans Hasselberg	a932aafc91	add primary keys to list keyring (#8522) ... During gossip encryption key rotation it would be nice to be able to see if all nodes are using the same key. This PR adds another field to the json response from `GET v1/operator/keyring` which lists the primary keys in use per dc. That way an operator can tell when a key was successfully setup as primary key. Based on https://github.com/hashicorp/serf/pull/611 to add primary key to list keyring output: ```json [ { "WAN": true, "Datacenter": "dc2", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 6, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6 }, "NumNodes": 6 }, { "WAN": false, "Datacenter": "dc2", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 8, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "NumNodes": 8 }, { "WAN": false, "Datacenter": "dc1", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 3, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "NumNodes": 8 } ] ``` I intentionally did not change the CLI output because I didn't find a good way of displaying this information. There are a couple of options that we could implement later: * add a flag to show the primary keys * add a flag to show json output Fixes #3393.	4 years ago
Daniel Nephin	d68edcecf4	testing: Remove all the defer os.Removeall ... Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage the removal	4 years ago
Mike Morris	48e7c07cf9	api: bump consul/api to v1.6.0 and consul/sdk to v0.6.0 (#8460) ... * api: bump consul/sdk dependency to v0.6.0 * api: bump dependency to v1.6.0	4 years ago
Daniel Nephin	ed8210fe4d	api: Use a Logger instead of an io.Writer in api.Watch ... So that we can pass around only a Logger, not a LogOutput	4 years ago
Mike Morris	85ef7ba943	api: restore Leader() and Peers() to avoid breaking function signatures (#8395) ... api: add TestAPI_StatusLeaderWithQueryOptions and TestAPI_StatusPeersWithQueryOptions api: make TestAPI_Status* error messages more verbose	4 years ago
spooner	c34b088583	Added QueryOptions for status api (#7818) ... * Added QueryOptions & Tests for status api	4 years ago
R.B. Boyer	e853368c23	gossip: Avoid issue where two unique leave events for the same node could lead to infinite rebroadcast storms (#8343) ... bump serf to v0.9.3 to include fix for https://github.com/hashicorp/serf/pull/606	4 years ago
Daniel Nephin	51efba2c7d	testutil: NewLogBuffer - buffer logs until a test fails ... Replaces #7559 Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case. Attaching log output from background goroutines also cause data races. If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race. The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long. This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful. All of the logs are printed from the test goroutine, so they should be associated with the correct test. Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly. Related: https://github.com/golang/go/issues/38458 (may be fixed in go1.15) https://github.com/golang/go/issues/38382#issuecomment-612940030	4 years ago
Freddy	e72af87918	Add api mod support for /catalog/gateway-services (#8278)	4 years ago
Seth Hoenig	95f46eb3ed	api/agent: enable setting SuccessBeforePassing and FailuresBeforeCritical in API (#7949) ... Fixes #7764 Until now these two fields could only be set through on-disk agent configuration. This change adds the fields to the agent API struct definition so that they can be set using the agent HTTP API.	4 years ago
R.B. Boyer	462f0f37ed	connect: various changes to make namespaces for intentions work more like for other subsystems (#8194) ... Highlights: - add new endpoint to query for intentions by exact match - using this endpoint from the CLI instead of the dump+filter approach - enforcing that OSS can only read/write intentions with a SourceNS or DestinationNS field of "default". - preexisting OSS intentions with now-invalid namespace fields will delete those intentions on initial election or for wildcard namespaces an attempt will be made to downgrade them to "default" unless one exists. - also allow the '-namespace' CLI arg on all of the intention subcommands - update lots of docs	4 years ago
freddygv	c791fbc79c	Update namespaces subject-verb agreement	4 years ago
s-christoff	818d00fda3	Add AgentMemberStatus const (#8110) ... * Add AgentMemberStatus const	4 years ago
Daniel Nephin	5afcf5c1bc	Merge pull request #8034 from hashicorp/dnephin/add-linter-staticcheck-4 ... ci: enable SA4006 staticcheck check and add ineffassign	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Daniel Nephin	cb050b280c	ci: enable SA4006 staticcheck check ... And fix the 'value not used' issues. Many of these are not bugs, but a few are tests not checking errors, and one appears to be a missed error in non-test code.	5 years ago
Matt Keeler	d3881dd754	ACL Node Identities (#7970) ... A Node Identity is very similar to a service identity. Its main targeted use is to allow creating tokens for use by Consul agents that will grant the necessary permissions for all the typical agent operations (node registration, coordinate updates, anti-entropy). Half of this commit is for golden file based tests of the acl token and role cli output. Another big updates was to refactor many of the tests in agent/consul/acl_endpoint_test.go to use the same style of tests and the same helpers. Besides being less boiler plate in the tests it also uses a common way of starting a test server with ACLs that should operate without any warnings regarding deprecated non-uuid master tokens etc.	5 years ago
Chris Piraino	91ab89dd48	Move ingress param to a new endpoint (#8081) ... In discussion with team, it was pointed out that query parameters tend to be filter mechanism, and that semantically the "/v1/health/connect" endpoint should return "all healthy connect-enabled endpoints (e.g. could be side car proxies or native instances) for this service so I can connect with mTLS". That does not fit an ingress gateway, so we remove the query parameter and add a new endpoint "/v1/health/ingress" that semantically means "all the healthy ingress gateway instances that I can connect to to access this connect-enabled service without mTLS"	5 years ago
Chris Piraino	496e683360	Merge pull request #8064 from hashicorp/ingress/health-query-param ... Add API query parameter ?ingress to allow users to find ingress gateways associated to a service	5 years ago
Chris Piraino	5e0cd7ede5	Remove unnecessary defer from api.health_test.go ... We do not need to deregister services because every test gets its own instance of the client agent and the tmp directories are all deleted at the end.	5 years ago
Daniel Nephin	08f1ed16b4	Merge pull request #7900 from hashicorp/dnephin/add-linter-staticcheck-2 ... intentions: fix a bug in Intention.SetHash	5 years ago
Chris Piraino	4837069fe0	api: update api module with health.Ingress() method	5 years ago
Daniel Nephin	caa692deea	ci: Enabled SA2002 staticcheck check ... And handle errors in the main test goroutine	5 years ago
Daniel Nephin	ce6cc094a1	intentions: fix a bug in Intention.SetHash ... Found using staticcheck. binary.Write does not accept int types without a size. The error from binary.Write was ignored, so we never saw this error. Casting the data to uint64 produces a correct hash. Also deprecate the Default{Addr,Port} fields, and prevent them from being encoded. These fields will always be empty and are not used. Removing these would break backwards compatibility, so they are left in place for now. Co-authored-by: Hans Hasselberg <me@hans.io>	5 years ago
R.B. Boyer	833211c14c	acl: allow auth methods created in the primary datacenter to optionally create global tokens (#7899)	5 years ago
R.B. Boyer	ffb9c7d6f7	acl: remove the deprecated `acl_enforce_version_8` option (#7991) ... Fixes #7292	5 years ago
Jono Sosulska	c554ba9e10	Replace whitelist/blacklist terminology with allowlist/denylist (#7971) ... * Replace whitelist/blacklist terminology with allowlist/denylist	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Daniel Nephin	8ced4300c8	Add alias struct tags for new decode hook	5 years ago
Daniel Nephin	600645b5f9	Add unconvert linter ... To find unnecessary type convertions	5 years ago
R.B. Boyer	1efafd7523	acl: add auth method for JWTs (#7846)	5 years ago
Kyle Havlovitz	136549205c	Merge pull request #7759 from hashicorp/ingress/tls-hosts ... Add TLS option for Ingress Gateway listeners	5 years ago
Chris Piraino	5105bf3d67	Require individual services in ingress entry to match protocols (#7774) ... We require any non-wildcard services to match the protocol defined in the listener on write, so that we can maintain a consistent experience through ingress gateways. This also helps guard against accidental misconfiguration by a user. - Update tests that require an updated protocol for ingress gateways	5 years ago
Chris Piraino	0c22eacca8	Add TLS field to ingress API structs ... - Adds test in api and command/config/write packages	5 years ago
Chris Piraino	d8517bd6fd	Better document wildcard specifier interactions	5 years ago
Chris Piraino	f40833d094	Allow Hosts field to be set on an ingress config entry ... - Validate that this cannot be set on a 'tcp' listener nor on a wildcard service. - Add Hosts field to api and test in consul config write CLI - xds: Configure envoy with user-provided hosts from ingress gateways	5 years ago
Chris Piraino	b73a13fc9e	Remove service_subset field from ingress config entry ... We decided that this was not a useful MVP feature, and just added unnecessary complexity	5 years ago
R.B. Boyer	a854e4d9c5	acl: oss plumbing to support auth method namespace rules in enterprise (#7794) ... This includes website docs updates.	5 years ago