consul

Commit Graph

Author	SHA1	Message	Date
sarahalsmiller	e2a81aa8bd	xds: generate listeners directly from API gateway snapshot (#17398 ) * API Gateway XDS Primitives, endpoints and clusters (#17002) * XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Routes for API Gateway (#17158) * XDS primitive generation for endpoints and clusters Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * server_test * deleted extra file * add missing parents to test * checkpoint * delete extra file * httproute flattening code * linting issue * so close on this, calling for tonight * unit test passing * add in header manip to virtual host * upstream rebuild commented out * Use consistent upstream name whether or not we're rebuilding * Start working through route naming logic * Fix typos in test descriptions * Simplify route naming logic * Simplify RebuildHTTPRouteUpstream * Merge additional compiled discovery chains instead of overwriting * Use correct chain for flattened route, clean up + add TODOs * Remove empty conditional branch * Restore previous variable declaration Limit the scope of this PR * Clean up, improve TODO * add logging, clean up todos * clean up function --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * checkpoint, skeleton, tests not passing * checkpoint * endpoints xds cluster configuration * resources test fix * fix reversion in resources_test * checkpoint * Update agent/proxycfg/api_gateway.go Co-authored-by: John Maguire <john.maguire@hashicorp.com> * unit tests passing * gofmt * add deterministic sorting to appease the unit test gods * remove panic * Find ready upstream matching listener instead of first in list * Clean up, improve TODO * Modify getReadyUpstreams to filter upstreams by listener (#17410) Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners. * clean up todos, references to api gateway in listeners_ingress * merge in Nathan's fix * Update agent/consul/discoverychain/gateway.go * cleanup current todos, remove snapshot manipulation from generation code * Update agent/structs/config_entry_gateways.go Co-authored-by: Thomas Eckert <teckert@hashicorp.com> * Update agent/consul/discoverychain/gateway.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Update agent/consul/discoverychain/gateway.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Update agent/proxycfg/snapshot.go Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * clarified header comment for FlattenHTTPRoute, changed RebuildHTTPRouteUpstream to BuildHTTPRouteUpstream * simplify cert logic * Delete scratch * revert route related changes in listener PR * Update agent/consul/discoverychain/gateway.go * Update agent/proxycfg/snapshot.go * clean up uneeded extra lines in endpoints --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: John Maguire <john.maguire@hashicorp.com> Co-authored-by: Thomas Eckert <teckert@hashicorp.com>	2 years ago
Ronald	94ec4eb2f4	copyright headers for agent folder (#16704 ) * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files	2 years ago
John Maguire	c833464daf	Update normalization of route refs (#16789 ) * Use merge of enterprise meta's rather than new custom method * Add merge logic for tcp routes * Add changelog * Normalize certificate refs on gateways * Fix infinite call loop * Explicitly call enterprise meta	2 years ago
Melisa Griffin	606f8fbbab	Adds check to verify that the API Gateway is being created with at least one listener	2 years ago
Andrew Stucki	7f9ec78932	[API Gateway] Validate listener name is not empty (#16340 ) * [API Gateway] Validate listener name is not empty * Update docstrings and test	2 years ago
Andrew Stucki	15d2684ecc	Normalize all API Gateway references (#16316 )	2 years ago
Andrew Stucki	58801cc8aa	Add stricter validation and some normalization code for API Gateway ConfigEntries (#16304 ) * Add stricter validation and some normalization code for API Gateway ConfigEntries	2 years ago
Andrew Stucki	e4a992c581	Fix hostname alignment checks for HTTPRoutes (#16300 ) * Fix hostname alignment checks for HTTPRoutes	2 years ago
Andrew Stucki	318ba215ab	[API Gateway] Add integration test for conflicted TCP listeners (#16225 )	2 years ago
Andrew Stucki	0891b4554d	Clean-up Gateway Controller Binding Logic (#16214 ) * Fix detecting when a route doesn't bind to a gateway because it's already bound * Clean up status setting code * rework binding a bit * More cleanup * Flatten all files * Fix up docstrings	2 years ago
Nathan Coleman	72a73661c9	Implement APIGateway proxycfg snapshot (#16194 ) * Stub proxycfg handler for API gateway * Add Service Kind constants/handling for API Gateway * Begin stubbing for SDS * Add new Secret type to xDS order of operations * Continue stubbing of SDS * Iterate on proxycfg handler for API gateway * Handle BoundAPIGateway config entry subscription in proxycfg-glue * Add API gateway to config snapshot validation * Add API gateway to config snapshot clone, leaf, etc. * Subscribe to bound route + cert config entries on bound-api-gateway * Track routes + certs on API gateway config snapshot * Generate DeepCopy() for types used in watch.Map * Watch all active references on api-gateway, unwatch inactive * Track loading of initial bound-api-gateway config entry * Use proper proto package for SDS mapping * Use ResourceReference instead of ServiceName, collect resources * Fix typo, add + remove TODOs * Watch discovery chains for TCPRoute * Add TODO for updating gateway services for api-gateway * make proto * Regenerate deep-copy for proxycfg * Set datacenter on upstream ID from query source * Watch discovery chains for http-route service backends * Add ServiceName getter to HTTP+TCP Service structs * Clean up unwatched discovery chains on API Gateway * Implement watch for ingress leaf certificate * Collect upstreams on http-route + tcp-route updates * Remove unused GatewayServices update handler * Remove unnecessary gateway services logic for API Gateway * Remove outdate TODO * Use .ToIngress where appropriate, including TODO for cleaning up * Cancel before returning error * Remove GatewayServices subscription * Add godoc for handlerAPIGateway functions * Update terminology from Connect => Consul Service Mesh Consistent with terminology changes in https://github.com/hashicorp/consul/pull/12690 * Add missing TODO * Remove duplicate switch case * Rerun deep-copy generator * Use correct property on config snapshot * Remove unnecessary leaf cert watch * Clean up based on code review feedback * Note handler properties that are initialized but set elsewhere * Add TODO for moving helper func into structs pkg * Update generated DeepCopy code * gofmt * Generate DeepCopy() for API gateway listener types * Improve variable name * Regenerate DeepCopy() code * Fix linting issue * Temporarily remove the secret type from resource generation	2 years ago
Andrew Stucki	df03b45bbc	Add additional controller implementations (#16188 ) * Add additional controller implementations * remove additional interface * Fix comparison checks and mark unused contexts * Switch to time.Now().UTC() * Add a pointer helper for shadowing loop variables * Extract anonymous functions for readability * clean up logging * Add Type to the Condition proto * Update some comments and add additional space for readability * Address PR feedback * Fix up dirty checks and change to pointer receiver	2 years ago
Andrew Stucki	da99514ac8	Add a server-only method for updating ConfigEntry Statuses (#16053 ) * Add a server-only method for updating ConfigEntry Statuses * Address PR feedback * Regen proto	2 years ago
Thomas Eckert	7814471159	Match route and listener protocols when binding (#16057 ) * Add GatewayMeta for matching routes to listeners based on protocols * Add GetGatewayMeta * Apply suggestions from code review Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> * Make GatewayMeta private * Bound -> BoundGateway * Document gatewayMeta more * Simplify conditional * Parallelize tests and simplify bind conditional * gofmt * 💧 getGatewayMeta --------- Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>	2 years ago
Thomas Eckert	20146f2916	Implement BindRoutesToGateways (#15950 ) * Stub out bind code * Move into a new package and flesh out binding * Fill in the actual binding logic * Bind to all listeners if not specified * Move bind code up to gateways package * Fix resource type check * Add UpsertRoute to listeners * Add RemoveRoute to listener * Implement binding as associated functions * Pass in gateways to BindRouteToGateways * Add a bunch of tests * Fix hopping from one listener on a gateway to another * Remove parents from HTTPRoute * Apply suggestions from code review * Fix merge conflict * Unify binding into a single variadic function 🙌 @nathancoleman * Remove vestigial error * Add TODO on protocol check	2 years ago
Thomas Eckert	13da1a5285	Native API Gateway Config Entries (#15897 ) * Stub Config Entries for Consul Native API Gateway (#15644) * Add empty InlineCertificate struct and protobuf * apigateway stubs * Stub HTTPRoute in api pkg * Stub HTTPRoute in structs pkg * Simplify api.APIGatewayConfigEntry to be consistent w/ other entries * Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry * Add TCPRoute to MakeConfigEntry, return unique Kind * Stub BoundAPIGatewayConfigEntry in agent * Add RaftIndex to APIGatewayConfigEntry stub * Add new config entry kinds to validation allow-list * Add RaftIndex to other added config entry stubs * Update usage metrics assertions to include new cfg entries * Add Meta and acl.EnterpriseMeta to all new ConfigEntry types * Remove unnecessary Services field from added config entry types * Implement GetMeta(), GetEnterpriseMeta() for added config entry types * Add meta field to proto, name consistently w/ existing config entries * Format config_entry.proto * Add initial implementation of CanRead + CanWrite for new config entry types * Add unit tests for decoding of new config entry types * Add unit tests for parsing of new config entry types * Add unit tests for API Gateway config entry ACLs * Return typed PermissionDeniedError on BoundAPIGateway CanWrite * Add unit tests for added config entry ACLs * Add BoundAPIGateway type to AllConfigEntryKinds * Return proper kind from BoundAPIGateway * Add docstrings for new config entry types * Add missing config entry kinds to proto def * Update usagemetrics_oss_test.go * Use utility func for returning PermissionDeniedError * EventPublisher subscriptions for Consul Native API Gateway (#15757) * Create new event topics in subscribe proto * Add tests for PBSubscribe func * Make configs singular, add all configs to PBToStreamSubscribeRequest * Add snapshot methods * Add config_entry_events tests * Add config entry kind to topic for new configs * Add unit tests for snapshot methods * Start adding integration test * Test using the new controller code * Update agent/consul/state/config_entry_events.go * Check value of error * Add controller stubs for API Gateway (#15837) * update initial stub implementation * move files, clean up mutex references * Remove embed, use idiomatic names for constructors * Remove stray file introduced in merge * Add APIGateway validation (#15847) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * APIGateway InlineCertificate validation (#15856) * Add APIGateway validation * Add additional validations * Add protobuf definitions * Tabs to spaces * Add API structs * Move struct fields around a bit * Add validation for InlineCertificate * Fix ACL test * APIGateway BoundAPIGateway validation (#15858) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * Add validation for BoundAPIGateway * APIGateway TCPRoute validation (#15855) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Add TCPRoute normalization and validation * Add forgotten Status * Add some more field docs in api package * Fix test * Format imports * Rename snapshot test variable names * Add plumbing for Native API GW Subscriptions (#16003) Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
cskh	04bf24c8c1	feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614 ) * feat(ingress-gateway): support outlier detection of upstream service for ingress gateway * changelog Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>	2 years ago
boruszak	38b1a515f1	Mergimg	2 years ago
cskh	69f40df548	feat(ingress gateway: support configuring limits in ingress-gateway c… (#14749 ) * feat(ingress gateway: support configuring limits in ingress-gateway config entry - a new Defaults field with max_connections, max_pending_connections, max_requests is added to ingress gateway config entry - new field max_connections, max_pending_connections, max_requests in individual services to overwrite the value in Default - added unit test and integration test - updated doc Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Dhia Ayachi	1b779240ae	update gateway-services table with endpoints (#13217 ) * update gateway-services table with endpoints * fix failing test * remove unneeded config in test * rename "endpoint" to "destination" * more endpoint renaming to destination in tests * update isDestination based on service-defaults config entry creation * use a 3 state kind to be able to set the kind to unknown (when neither a service or a destination exist) * set unknown state to empty to avoid modifying alot of tests * fix logic to set the kind correctly on CRUD * fix failing tests * add missing tests and fix service delete * fix failing test * Apply suggestions from code review Co-authored-by: Dan Stough <dan.stough@hashicorp.com> * fix a bug with kind and add relevant test * fix compile error * fix failing tests * add kind to clone * fix failing tests * fix failing tests in catalog endpoint * fix service dump test * Apply suggestions from code review Co-authored-by: Dan Stough <dan.stough@hashicorp.com> * remove duplicate tests * rename consts and fix kind when no destination is defined in the service-defaults. * rename Kind to ServiceKind and change switch to use .(type) Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	3 years ago
DanStough	2e2c71d2f2	fix: multiple grpc/http2 services for ingress listeners	3 years ago
Mark Anderson	05eded4f1d	Manual Structs fixup Change things by hand that I couldn't figure out how to automate Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Kyle Havlovitz	f8efe9a208	Log a warning when a terminating gateway service has TLS but not SNI configured	3 years ago
R.B. Boyer	e79ce8ab03	xds: adding control of the mesh-wide min/max TLS versions and cipher suites from the mesh config entry (#12601 ) - `tls.incoming`: applies to the inbound mTLS targeting the public listener on `connect-proxy` and `terminating-gateway` envoy instances - `tls.outgoing`: applies to the outbound mTLS dialing upstreams from `connect-proxy` and `ingress-gateway` envoy instances Fixes #11966	3 years ago
Mark Anderson	676ea58bc4	Refactor config checks oss (#12550 ) Currently the config_entry.go subsystem delegates authorization decisions via the ConfigEntry interface CanRead and CanWrite code. Unfortunately this returns a true/false value and loses the details of the source. This is not helpful, especially since it the config subsystem can be more complex to understand, since it covers so many domains. This refactors CanRead/CanWrite to return a structured error message (PermissionDenied or the like) with more details about the reason for denial. Part of #12241 Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mike Morris	1b1a97e8f9	ingress: allow setting TLS min version and cipher suites in ingress gateway config entries (#11576 ) * xds: refactor ingress listener SDS configuration * xds: update resolveListenerSDS call args in listeners_test * ingress: add TLS min, max and cipher suites to GatewayTLSConfig * xds: implement envoyTLSVersions and envoyTLSCipherSuites * xds: merge TLS config * xds: configure TLS parameters with ingress TLS context from leaf * xds: nil check in resolveListenerTLSConfig validation * xds: nil check in makeTLSParameters* functions * changelog: add entry for TLS params on ingress config entries * xds: remove indirection for TLS params in TLSConfig structs * xds: return tlsContext, nil instead of ambiguous err Co-authored-by: Chris S. Kim <ckim@hashicorp.com> * xds: switch zero checks to types.TLSVersionUnspecified * ingress: add validation for ingress config entry TLS params * ingress: validate listener TLS config * xds: add basic ingress with TLS params tests * xds: add ingress listeners mixed TLS min version defaults precedence test * xds: add more explicit tests for ingress listeners inheriting gateway defaults * xds: add test for single TLS listener on gateway without TLS defaults * xds: regen golden files for TLSVersionInvalid zero value, add TLSVersionAuto listener test * types/tls: change TLSVersion to string * types/tls: update TLSCipherSuite to string type * types/tls: implement validation functions for TLSVersion and TLSCipherSuites, make some maps private * api: add TLS params to GatewayTLSConfig, add tests * api: add TLSMinVersion to ingress gateway config entry test JSON * xds: switch to Envoy TLS cipher suite encoding from types package * xds: fixup validation for TLSv1_3 min version with cipher suites * add some kitchen sink tests and add a missing struct tag * xds: check if mergedCfg.TLSVersion is in TLSVersionsWithConfigurableCipherSuites * xds: update connectTLSEnabled comment * xds: remove unsued resolveGatewayServiceTLSConfig function * xds: add makeCommonTLSContextFromLeafWithoutParams * types/tls: add LessThan comparator function for concrete values * types/tls: change tlsVersions validation map from string to TLSVersion keys * types/tls: remove unused envoyTLSCipherSuites * types/tls: enable chacha20 cipher suites for Consul agent * types/tls: remove insecure cipher suites from allowed config TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 are both explicitly listed as insecure and disabled in the Go source. Refs https://cs.opensource.google/go/go/+/refs/tags/go1.17.3:src/crypto/tls/cipher_suites.go;l=329-330 * types/tls: add ValidateConsulAgentCipherSuites function, make direct lookup map private * types/tls: return all unmatched cipher suites in validation errors * xds: check that Envoy API value matching TLS version is found when building TlsParameters * types/tls: check that value is found in map before appending to slice in MarshalEnvoyTLSCipherSuiteStrings * types/tls: cast to string rather than fmt.Printf in TLSCihperSuite.String() * xds: add TLSVersionUnspecified to list of configurable cipher suites * structs: update note about config entry warning * xds: remove TLS min version cipher suite unconfigurable test placeholder * types/tls: update tests to remove assumption about private map values Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
freddygv	e5b7c4713f	Accept partition for ingress services	3 years ago
Paul Banks	7b4cbe3143	Final readability tweaks from review	3 years ago
Paul Banks	07f81991df	Refactor SDS validation to make it more contained and readable	3 years ago
Paul Banks	4e39f03d5b	Add ingress-gateway config for SDS	3 years ago
Freddy	e18f3c1f6d	Update error texts (#11022 ) Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>	3 years ago
Paul Banks	b38e84df63	Include namespace and partition in error messages when validating ingress header manip	3 years ago
Paul Banks	f439dfc04f	Ingress gateway header manip plumbing	3 years ago
Paul Banks	46e4041283	Header manip and validation added for ingress-gateway entries	3 years ago
R.B. Boyer	ee372a854a	acl: adding a new mesh resource	3 years ago
R.B. Boyer	e3835ac6a1	structs: prohibit config entries from referencing more than one partition at a time (#10478 ) affected kinds: service-defaults, ingress-gateway, terminating-gateway, service-intentions	3 years ago
John Cowen	5892e75452	ui: Remove any trailing fullstop/period DNS characters from Gateways UI API (#9752 ) Previous to this commit, the API response would include Gateway Addresses in the form `domain.name.:8080`, which due to the addition of the port is probably not the expected response. This commit rightTrims any `.` characters from the end of the domain before formatting the address to include the port resulting in `domain.name:8080`	4 years ago
Daniel Nephin	d113f0e690	structs: Fix printing of IDs These types are used as values (not pointers) in other structs. Using a pointer receiver causes problems when the value is printed. fmt will not call the String method if it is passed a value and the String method has a pointer receiver. By using a value receiver the correct string is printed. Also remove some unused methods.	4 years ago
R.B. Boyer	119e945c3e	connect: all config entries pick up a meta field (#8596 ) Fixes #8595	4 years ago
Jack	9e1c6727f9	Add http2 and grpc support to ingress gateways (#8458 )	4 years ago
R.B. Boyer	e3cd4a8539	connect: use stronger validation that ingress gateways have compatible protocols defined for their upstreams (#8470 ) Fixes #8466 Since Consul 1.8.0 there was a bug in how ingress gateway protocol compatibility was enforced. At the point in time that an ingress-gateway config entry was modified the discovery chain for each upstream was checked to ensure the ingress gateway protocol matched. Unfortunately future modifications of other config entries were not validated against existing ingress-gateway definitions, such as: 1. create tcp ingress-gateway pointing to 'api' (ok) 2. create service-defaults for 'api' setting protocol=http (worked, but not ok) 3. create service-splitter or service-router for 'api' (worked, but caused an agent panic) If you were to do these in a different order, it would fail without a crash: 1. create service-defaults for 'api' setting protocol=http (ok) 2. create service-splitter or service-router for 'api' (ok) 3. create tcp ingress-gateway pointing to 'api' (fail with message about protocol mismatch) This PR introduces the missing validation. The two new behaviors are: 1. create tcp ingress-gateway pointing to 'api' (ok) 2. (NEW) create service-defaults for 'api' setting protocol=http ("ok" for back compat) 3. (NEW) create service-splitter or service-router for 'api' (fail with message about protocol mismatch) In consideration for any existing users that may be inadvertently be falling into item (2) above, that is now officiall a valid configuration to be in. For anyone falling into item (3) above while you cannot use the API to manufacture that scenario anymore, anyone that has old (now bad) data will still be able to have the agent use them just enough to generate a new agent/proxycfg error message rather than a panic. Unfortunately we just don't have enough information to properly fix the config entries.	4 years ago
Chris Piraino	b3db907bdf	Update gateway-services-nodes API endpoint to allow multiple addresses Previously, we were only returning a single ListenerPort for a single service. However, we actually allow a single service to be serviced over multiple ports, as well as allow users to define what hostnames they expect their services to be contacted over. When no hosts are defined, we return the default ingress domain for any configured DNS domain. To show this in the UI, we modify the gateway-services-nodes API to return a GatewayConfig.Addresses field, which is a list of addresses over which the specific service can be contacted.	4 years ago
freddygv	19e3954603	Move compound service names to use ServiceName type	5 years ago
Chris Piraino	6fa48c9512	Allow users to set hosts to the wildcard specifier when TLS is disabled (#8083 ) This allows easier demoing/testing of ingress gateways, while still preserving the validation we have for DNSSANs	5 years ago
Daniel Nephin	8ced4300c8	Add alias struct tags for new decode hook	5 years ago
R.B. Boyer	77f2e54618	create lib/stringslice package (#7934 )	5 years ago
Kyle Havlovitz	8d140ce9af	Disallow the blanket wildcard prefix from being used as custom host	5 years ago
Freddy	c32a4f1ece	Fix up enterprise compatibility for gateways (#7813 )	5 years ago
Chris Piraino	0c22eacca8	Add TLS field to ingress API structs - Adds test in api and command/config/write packages	5 years ago
Chris Piraino	0b9ba9660d	Validate hosts input in ingress gateway config entry We can only allow host names that are valid domain names because we put these hosts into a DNSSAN. In addition, we validate that the wildcard specifier '*' is only present as the leftmost label to allow for a wildcard DNSSAN and associated wildcard Host routing in the ingress gateway proxy.	5 years ago

1 2

63 Commits (67254a924485bec90071016666cf41e92bbffed0)