84 Commits (0827c9422227720752d421eada1cf74c71685486)

Author	SHA1	Message	Date
freddygv	ed6076db26	Rename partition-exports to exported-services ... Using a name less tied to partitions gives us more flexibility to use this config entry in OSS for exports between datacenters/meshes.	3 years ago
Daniel Upton	d47b7311b8	Support Check-And-Set deletion of config entries (#11419) ... Implements #11372	3 years ago
freddygv	5c24ed61a8	Rename service-exports to partition-exports ... Existing config entries prefixed by service- are specific to individual services. Since this config entry applies to partitions it is being renamed. Additionally, the Partition label was changed to Name because using Partition at the top-level and in the enterprise meta was leading to the enterprise meta partition being dropped by msgpack.	3 years ago
Kyle Havlovitz	04cd2c983e	Add new service-exports config entry	3 years ago
Chris S. Kim	9bbfa048a2	Sync enterprise changes to oss (#10994) ... This commit updates OSS with files for enterprise-specific admin partitions feature work	3 years ago
R.B. Boyer	ee372a854a	acl: adding a new mesh resource	3 years ago
R.B. Boyer	e3835ac6a1	structs: prohibit config entries from referencing more than one partition at a time (#10478) ... affected kinds: service-defaults, ingress-gateway, terminating-gateway, service-intentions	3 years ago
R.B. Boyer	8344b7fe2e	structs: prevent service-defaults upstream configs from using wildcard names or namespaces (#10475)	3 years ago
Daniel Nephin	ba15f92a8a	structs: fix cache keys ... So that requests are cached properly, and the cache does not return the wrong data for a request.	4 years ago
R.B. Boyer	597448da47	server: ensure that central service config flattening properly resets the state each time (#10239) ... The prior solution to call reply.Reset() aged poorly since newer fields were added to the reply, but not added to Reset() leading serial blocking query loops on the server to blend replies. This could manifest as a service-defaults protocol change from default=>http not reverting back to default after the config entry reponsible was deleted.	4 years ago
R.B. Boyer	7e1d7803b8	agent: ensure we hash the non-deprecated upstream fields on ServiceConfigRequest (#10240)	4 years ago
Daniel Nephin	c8c85523e1	config-entries: add a test for the API client ... Also fixes a bug with listing kind=mesh config entries. ValidateConfigEntryKind was only being used by the List endpoint, and was yet another place where we have to enumerate all the kinds. This commit removes ValidateConfigEntryKind and uses MakeConfigEntry instead. This change removes the need to maintain two separate functions at the cost of creating an instance of the config entry which will be thrown away immediately.	4 years ago
Daniel Nephin	62efaaab21	config-entry: remove Kind and Name field from Mesh config entry ... No config entry needs a Kind field. It is only used to determine the Go type to target. As we introduce new config entries (like this one) we can remove the kind field and have the GetKind method return the single supported value. In this case (similar to proxy-defaults) the Name field is also unnecessary. We always use the same value. So we can omit the name field entirely.	4 years ago
Freddy	078c40425f	Rename "cluster" config entry to "mesh" (#10127) ... This config entry is being renamed primarily because in k8s the name cluster could be confusing given that the config entry applies across federated datacenters. Additionally, this config entry will only apply to Consul as a service mesh, so the more generic "cluster" name is not needed.	4 years ago
Freddy	f265dd635f	Disallow * as service-defaults name (#10069)	4 years ago
freddygv	a0f3591aee	Don't panic on nil UpstreamConfiguration.Clone()	4 years ago
R.B. Boyer	4db8b78854	connect: update centralized upstreams representation in service-defaults (#10015)	4 years ago
Freddy	8fc60a6ca6	Merge pull request #10000 from hashicorp/remove-upstream-cfg-validation ... Remove zero-value validation of upstream cfg structs	4 years ago
freddygv	d7c43049fa	Remove zero-value validation of upstream cfg structs ... The zero value of these flags was already being excluded in the xDS generation of circuit breaker/outlier detection config. See: makeThresholdsIfNeeded and ToOutlierDetection.	4 years ago
freddygv	7bd51ff536	Replace TransparentProxy bool with ProxyMode ... This PR replaces the original boolean used to configure transparent proxy mode. It was replaced with a string mode that can be set to: - "": Empty string is the default for when the setting should be defaulted from other configuration like config entries. - "direct": Direct mode is how applications originally opted into the mesh. Proxy listeners need to be dialed directly. - "transparent": Transparent mode enables configuring Envoy as a transparent proxy. Traffic must be captured and redirected to the inbound and outbound listeners. This PR also adds a struct for transparent proxy specific configuration. Initially this is not stored as a pointer. Will revisit that decision before GA.	4 years ago
freddygv	ab752c1c86	Avoid sending zero-value upstream defaults from api	4 years ago
freddygv	986bcccbea	Pass down upstream defaults to client proxies ... This is needed in case the client proxy is in TransparentProxy mode. Typically they won't have explicit configuration for every upstream, so this ensures the settings can be applied to all of them when generating xDS config.	4 years ago
R.B. Boyer	499fee73b3	connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973) ... This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.	4 years ago
freddygv	52bf00de8b	Split up normalizing from defaulting values for upstream cfg	4 years ago
freddygv	f4f45af6d0	Merge master and fix upstream config protocol defaulting	4 years ago
Freddy	8207b832df	Add TransparentProxy option to proxy definitions	4 years ago
Freddy	c664938bae	Add per-upstream configuration to service-defaults	4 years ago
freddygv	7df846aa24	Pass MeshGateway config in service config request ... ResolveServiceConfig is called by service manager before the proxy registration is in the catalog. Therefore we should pass proxy registration flags in the request rather than trying to fetch them from the state store (where they may not exist yet).	4 years ago
freddygv	8b46d8dcbb	Restore old Envoy prefix on escape hatches ... This is done because after removing ID and NodeName from ServiceConfigRequest we will no longer know whether a request coming in is for a Consul client earlier than v1.10.	4 years ago
freddygv	93c3c1780d	Only lowercase the protocol when normalizing	4 years ago
freddygv	41b2ba1e58	Add omitempty across the board for UpstreamConfig	4 years ago
freddygv	6fd30d0384	Add TransparentProxy opt to proxy definition	4 years ago
freddygv	306ef7d252	Restore old escape hatch alias	4 years ago
freddygv	e3dc2a49df	Turn Limits and PassiveHealthChecks into pointers	4 years ago
freddygv	acec711a6a	Update server-side config resolution and client-side merging	4 years ago
freddygv	1710ec87d2	finish moving UpstreamConfig and related fields to structs pkg	4 years ago
Daniel Nephin	b06b3dd8f8	state: move ConfigEntryKindName ... Previously this type was defined in structs, but unlike the other types in structs this type is not used by RPC requests. By moving it to state we can better indicate that this is not an API type, but part of the state implementation.	4 years ago
freddygv	87cde19b4c	Create new types for service-defaults upstream cfg	4 years ago
Daniel Nephin	d113f0e690	structs: Fix printing of IDs ... These types are used as values (not pointers) in other structs. Using a pointer receiver causes problems when the value is printed. fmt will not call the String method if it is passed a value and the String method has a pointer receiver. By using a value receiver the correct string is printed. Also remove some unused methods.	4 years ago
R.B. Boyer	58387fef0a	server: config entry replication now correctly uses namespaces in comparisons (#9024) ... Previously config entries sharing a kind & name but in different namespaces could occasionally cause "stuck states" in replication because the namespace fields were ignored during the differential comparison phase. Example: Two config entries written to the primary: kind=A,name=web,namespace=bar kind=A,name=web,namespace=foo Under the covers these both get saved to memdb, so they are sorted by all 3 components (kind,name,namespace) during natural iteration. This means that before the replication code does it's own incomplete sort, the underlying data IS sorted by namespace ascending (bar comes before foo). After one pass of replication the primary and secondary datacenters have the same set of config entries present. If "kind=A,name=web,namespace=bar" were to be deleted, then things get weird. Before replication the two sides look like: primary: [ kind=A,name=web,namespace=foo ] secondary: [ kind=A,name=web,namespace=bar kind=A,name=web,namespace=foo ] The differential comparison phase walks these two lists in sorted order and first compares "kind=A,name=web,namespace=foo" vs "kind=A,name=web,namespace=bar" and falsely determines they are the SAME and are thus cause an update of "kind=A,name=web,namespace=foo". Then it compares "<nothing>" with "kind=A,name=web,namespace=foo" and falsely determines that the latter should be DELETED. During reconciliation the deletes are processed before updates, and so for a brief moment in the secondary "kind=A,name=web,namespace=foo" is erroneously deleted and then immediately restored. Unfortunately after this replication phase the final state is identical to the initial state, so when it loops around again (rate limited) it repeats the same set of operations indefinitely.	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834) ... - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
R.B. Boyer	119e945c3e	connect: all config entries pick up a meta field (#8596) ... Fixes #8595	4 years ago
R.B. Boyer	d0f74cd1e8	connect: fix bug in preventing some namespaced config entry modifications (#8601) ... Whenever an upsert/deletion of a config entry happens, within the open state store transaction we speculatively test compile all discovery chains that may be affected by the pending modification to verify that the write would not create an erroneous scenario (such as splitting traffic to a subset that did not exist). If a single discovery chain evaluation references two config entries with the same kind and name in different namespaces then sometimes the upsert/deletion would be falsely rejected. It does not appear as though this bug would've let invalid writes through to the state store so the correction does not require a cleanup phase.	4 years ago
Freddy	10d6e9c458	Split up unused key validation for oss/ent (#8189) ... Split up unused key validation in config entry decode for oss/ent. This is needed so that we can return an informative error in OSS if namespaces are provided.	4 years ago
freddygv	c791fbc79c	Update namespaces subject-verb agreement	4 years ago
freddygv	044d027ff8	Remove break	4 years ago
freddygv	70810b0602	Let users know namespaces are ent only in config entry decode	4 years ago
Daniel Nephin	75cbbe2702	config: add HookWeakDecodeFromSlice ... Currently opaque config blocks (config entries, and CA provider config) are modified by PatchSliceOfMaps, making it impossible for these opaque config sections to contain slices of maps. In order to fix this problem, any lazy-decoding of these blocks needs to support weak decoding of []map[string]interface{} to a struct type before PatchSliceOfMaps is replaces. This is necessary because these config blobs are persisted, and during an upgrade an older version of Consul could read one of the new configuration values, which would cause an error. To support the upgrade path, this commit first introduces the new hooks for weak decoding of []map[string]interface{} and uses them only in the lazy-decode paths. That way, in a future release, new style configuration will be supported by the older version of Consul. This decode hook has a number of advantages: 1. It no longer panics. It allows mapstructure to report the error 2. It no longer requires the user to declare which fields are slices of structs. It can deduce that information from the 'to' value. 3. It will make it possible to preserve opaque configuration, allowing for structured opaque config.	5 years ago
Daniel Nephin	6a2d7d77c0	config: use the new HookTranslateKeys instead of lib.TranslateKeys ... With the exception of CA provider config, which will be migrated at some later time.	5 years ago
Daniel Nephin	8ced4300c8	Add alias struct tags for new decode hook	5 years ago