github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '1f25289048'
${ noResults }
consul/agent/proxycfg/testing_peering.go

380 lines
10 KiB
Raw Normal View History Unescape

Use embedded SpiffeID for peered upstreams
3 years ago
package proxycfg
import (
"github.com/mitchellh/go-testing-interface"
"github.com/hashicorp/consul/agent/structs"
Configure upstream TLS context with peer root certs (#13321) For mTLS to work between two proxies in peered clusters with different root CAs, proxies need to configure their outbound listener to use different root certificates for validation. Up until peering was introduced proxies would only ever use one set of root certificates to validate all mesh traffic, both inbound and outbound. Now an upstream proxy may have a leaf certificate signed by a CA that's different from the dialing proxy's. This PR makes changes to proxycfg and xds so that the upstream TLS validation uses different root certificates depending on which cluster is being dialed.
3 years ago
"github.com/hashicorp/consul/proto/pbpeering"
Use embedded SpiffeID for peered upstreams
3 years ago
)
func TestConfigSnapshotPeering(t testing.T) *ConfigSnapshot {
var (
paymentsUpstream = structs.Upstream{
DestinationName: "payments",
DestinationPeer: "cloud",
LocalBindPort: 9090,
}
paymentsUID = NewUpstreamID(&paymentsUpstream)
refundsUpstream = structs.Upstream{
DestinationName: "refunds",
DestinationPeer: "cloud",
LocalBindPort: 9090,
}
refundsUID = NewUpstreamID(&refundsUpstream)
)
peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) This removes unnecessary, vestigal remnants of discovery chains.
3 years ago
const peerTrustDomain = "1c053652-8512-4373-90cf-5a7f6263a994.consul"
Use embedded SpiffeID for peered upstreams
3 years ago
return TestConfigSnapshot(t, func(ns *structs.NodeService) {
ns.Proxy.Upstreams = structs.Upstreams{
paymentsUpstream,
refundsUpstream,
}
}, []UpdateEvent{
Configure upstream TLS context with peer root certs (#13321) For mTLS to work between two proxies in peered clusters with different root CAs, proxies need to configure their outbound listener to use different root certificates for validation. Up until peering was introduced proxies would only ever use one set of root certificates to validate all mesh traffic, both inbound and outbound. Now an upstream proxy may have a leaf certificate signed by a CA that's different from the dialing proxy's. This PR makes changes to proxycfg and xds so that the upstream TLS validation uses different root certificates depending on which cluster is being dialed.
3 years ago
{
CorrelationID: peerTrustBundleIDPrefix + "cloud",
Result: &pbpeering.TrustBundleReadResponse{
Bundle: TestPeerTrustBundles(t).Bundles[0],
},
},
Use embedded SpiffeID for peered upstreams
3 years ago
{
peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) This removes unnecessary, vestigal remnants of discovery chains.
3 years ago
CorrelationID: upstreamPeerWatchIDPrefix + paymentsUID.String(),
Use embedded SpiffeID for peered upstreams
3 years ago
Result: &structs.IndexedCheckServiceNodes{
Nodes: []structs.CheckServiceNode{
{
Node: &structs.Node{
Address: "85.252.102.31",
Datacenter: "cloud-dc",
},
Service: &structs.NodeService{
Service: "payments-sidecar-proxy",
Kind: structs.ServiceKindConnectProxy,
Port: 443,
xds: allow for peered upstreams to use tagged addresses that are hostnames (#13422) Mesh gateways can use hostnames in their tagged addresses (#7999). This is useful if you were to expose a mesh gateway using a cloud networking load balancer appliance that gives you a DNS name but no reliable static IPs. Envoy cannot accept hostnames via EDS and those must be configured using CDS. There was already logic when configuring gateways in other locations in the code, but given the illusions in play for peering the downstream of a peered service wasn't aware that it should be doing that. Also: - ensuring that we always try to use wan-like addresses to cross peer boundaries.
3 years ago
TaggedAddresses: map[string]structs.ServiceAddress{
structs.TaggedAddressLAN: {
Address: "85.252.102.31",
Port: 443,
},
structs.TaggedAddressWAN: {
Address: "123.us-east-1.elb.notaws.com",
Port: 8443,
},
},
Use embedded SpiffeID for peered upstreams
3 years ago
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) This removes unnecessary, vestigal remnants of discovery chains.
3 years ago
SNI: []string{
"payments.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/payments",
},
Protocol: "tcp",
Use embedded SpiffeID for peered upstreams
3 years ago
},
},
},
},
},
},
},
{
peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) This removes unnecessary, vestigal remnants of discovery chains.
3 years ago
CorrelationID: upstreamPeerWatchIDPrefix + refundsUID.String(),
Use embedded SpiffeID for peered upstreams
3 years ago
Result: &structs.IndexedCheckServiceNodes{
Nodes: []structs.CheckServiceNode{
{
Node: &structs.Node{
Address: "106.96.90.233",
Datacenter: "cloud-dc",
},
Service: &structs.NodeService{
Service: "refunds-sidecar-proxy",
Kind: structs.ServiceKindConnectProxy,
Port: 443,
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
peering: update how cross-peer upstreams and represented in proxycfg and rendered in xds (#13362) This removes unnecessary, vestigal remnants of discovery chains.
3 years ago
SNI: []string{
"refunds.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/refunds",
},
Protocol: "tcp",
Use embedded SpiffeID for peered upstreams
3 years ago
},
},
},
},
},
},
},
})
}
Make envoy resources for inferred peered upstreams (#13758) Peered upstreams has a separate loop in xds from discovery chain upstreams. This PR adds similar but slightly modified code to add filters for peered upstream listeners, clusters, and endpoints in the case of transparent proxy.
2 years ago
func TestConfigSnapshotPeeringTProxy(t testing.T) *ConfigSnapshot {
// Test two explicitly defined upstreams api-a and noEndpoints
// as well as one implicitly inferred upstream db.
var (
noEndpointsUpstream = structs.Upstream{
DestinationName: "no-endpoints",
DestinationPeer: "peer-a",
LocalBindPort: 1234,
}
noEndpoints = structs.PeeredServiceName{
ServiceName: structs.NewServiceName("no-endpoints", nil),
Peer: "peer-a",
}
apiAUpstream = structs.Upstream{
DestinationName: "api-a",
DestinationPeer: "peer-a",
LocalBindPort: 9090,
}
apiA = structs.PeeredServiceName{
ServiceName: structs.NewServiceName("api-a", nil),
Peer: "peer-a",
}
db = structs.PeeredServiceName{
ServiceName: structs.NewServiceName("db", nil),
Peer: "peer-a",
}
)
const peerTrustDomain = "1c053652-8512-4373-90cf-5a7f6263a994.consul"
return TestConfigSnapshot(t, func(ns *structs.NodeService) {
ns.Proxy.Mode = structs.ProxyModeTransparent
ns.Proxy.Upstreams = []structs.Upstream{
noEndpointsUpstream,
apiAUpstream,
}
}, []UpdateEvent{
{
CorrelationID: meshConfigEntryID,
Result: &structs.ConfigEntryResponse{
Entry: nil,
},
},
{
CorrelationID: peeredUpstreamsID,
Result: &structs.IndexedPeeredServiceList{
Services: []structs.PeeredServiceName{
apiA,
noEndpoints,
db, // implicitly added here
},
},
},
{
CorrelationID: peerTrustBundleIDPrefix + "peer-a",
Result: &pbpeering.TrustBundleReadResponse{
Bundle: TestPeerTrustBundles(t).Bundles[0],
},
},
{
CorrelationID: upstreamPeerWatchIDPrefix + NewUpstreamID(&noEndpointsUpstream).String(),
Result: &structs.IndexedCheckServiceNodes{
Nodes: []structs.CheckServiceNode{},
},
},
{
CorrelationID: upstreamPeerWatchIDPrefix + NewUpstreamID(&apiAUpstream).String(),
Result: &structs.IndexedCheckServiceNodes{
Nodes: structs.CheckServiceNodes{
{
Node: &structs.Node{
Node: "node1",
Address: "127.0.0.1",
PeerName: "peer-a",
},
Service: &structs.NodeService{
ID: "api-a-1",
Service: "api-a",
PeerName: "peer-a",
Address: "1.2.3.4",
TaggedAddresses: map[string]structs.ServiceAddress{
"virtual": {Address: "10.0.0.1"},
structs.TaggedAddressVirtualIP: {Address: "240.0.0.1"},
},
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
SNI: []string{
"api-a.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/api-a",
},
Protocol: "tcp",
},
},
},
},
},
},
},
{
CorrelationID: upstreamPeerWatchIDPrefix + NewUpstreamIDFromPeeredServiceName(db).String(),
Result: &structs.IndexedCheckServiceNodes{
Nodes: structs.CheckServiceNodes{
{
Node: &structs.Node{
Node: "node1",
Address: "127.0.0.1",
PeerName: "peer-a",
},
Service: &structs.NodeService{
ID: "db-1",
Service: "db",
PeerName: "peer-a",
Address: "2.3.4.5", // Expect no endpoint or listener for this address
TaggedAddresses: map[string]structs.ServiceAddress{
"virtual": {Address: "10.0.0.2"},
structs.TaggedAddressVirtualIP: {Address: "240.0.0.2"},
},
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
SNI: []string{
"db.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/db",
},
Protocol: "tcp",
},
},
},
},
},
},
},
})
}
Enable outbound peered requests to go through local mesh gateway (#14763)
2 years ago
func TestConfigSnapshotPeeringLocalMeshGateway(t testing.T) *ConfigSnapshot {
var (
paymentsUpstream = structs.Upstream{
DestinationName: "payments",
DestinationPeer: "cloud",
LocalBindPort: 9090,
Fix mesh gateway configuration with proxy-defaults (#15186) * Fix mesh gateway proxy-defaults not affecting upstreams. * Clarify distinction with upstream settings Top-level mesh gateway mode in proxy-defaults and service-defaults gets merged into NodeService.Proxy.MeshGateway, and only gets merged with the mode attached to an an upstream in proxycfg/xds. * Fix mgw mode usage for peered upstreams There were a couple issues with how mgw mode was being handled for peered upstreams. For starters, mesh gateway mode from proxy-defaults and the top-level of service-defaults gets stored in NodeService.Proxy.MeshGateway, but the upstream watch for peered data was only considering the mesh gateway config attached in NodeService.Proxy.Upstreams[i]. This means that applying a mesh gateway mode via global proxy-defaults or service-defaults on the downstream would not have an effect. Separately, transparent proxy watches for peered upstreams didn't consider mesh gateway mode at all. This commit addresses the first issue by ensuring that we overlay the upstream config for peered upstreams as we do for non-peered. The second issue is addressed by re-using setupWatchesForPeeredUpstream when handling transparent proxy updates. Note that for transparent proxies we do not yet support mesh gateway mode per upstream, so the NodeService.Proxy.MeshGateway mode is used. * Fix upstream mesh gateway mode handling in xds This commit ensures that when determining the mesh gateway mode for peered upstreams we consider the NodeService.Proxy.MeshGateway config as a baseline. In absense of this change, setting a mesh gateway mode via proxy-defaults or the top-level of service-defaults will not have an effect for peered upstreams. * Merge service/proxy defaults in cfg resolver Previously the mesh gateway mode for connect proxies would be merged at three points: 1. On servers, in ComputeResolvedServiceConfig. 2. On clients, in MergeServiceConfig. 3. On clients, in proxycfg/xds. The first merge returns a ServiceConfigResponse where there is a top-level MeshGateway config from proxy/service-defaults, along with per-upstream config. The second merge combines per-upstream config specified at the service instance with per-upstream config specified centrally. The third merge combines the NodeService.Proxy.MeshGateway config containing proxy/service-defaults data with the per-upstream mode. This third merge is easy to miss, which led to peered upstreams not considering the mesh gateway mode from proxy-defaults. This commit removes the third merge, and ensures that all mesh gateway config is available at the upstream. This way proxycfg/xds do not need to do additional overlays. * Ensure that proxy-defaults is considered in wc Upstream defaults become a synthetic Upstream definition under a wildcard key "*". Now that proxycfg/xds expect Upstream definitions to have the final MeshGateway values, this commit ensures that values from proxy-defaults/service-defaults are the default for this synthetic upstream. * Add changelog. Co-authored-by: freddygv <freddy@hashicorp.com>
2 years ago
MeshGateway: structs.MeshGatewayConfig{Mode: structs.MeshGatewayModeRemote},
Enable outbound peered requests to go through local mesh gateway (#14763)
2 years ago
}
paymentsUID = NewUpstreamID(&paymentsUpstream)
refundsUpstream = structs.Upstream{
DestinationName: "refunds",
DestinationPeer: "cloud",
LocalBindPort: 9090,
MeshGateway: structs.MeshGatewayConfig{Mode: structs.MeshGatewayModeLocal},
}
refundsUID = NewUpstreamID(&refundsUpstream)
)
const peerTrustDomain = "1c053652-8512-4373-90cf-5a7f6263a994.consul"
return TestConfigSnapshot(t, func(ns *structs.NodeService) {
ns.Proxy.Upstreams = structs.Upstreams{
paymentsUpstream,
refundsUpstream,
}
}, []UpdateEvent{
{
CorrelationID: peerTrustBundleIDPrefix + "cloud",
Result: &pbpeering.TrustBundleReadResponse{
Bundle: TestPeerTrustBundles(t).Bundles[0],
},
},
{
CorrelationID: upstreamPeerWatchIDPrefix + paymentsUID.String(),
Result: &structs.IndexedCheckServiceNodes{
Nodes: []structs.CheckServiceNode{
{
Node: &structs.Node{
Address: "85.252.102.31",
Datacenter: "cloud-dc",
},
Service: &structs.NodeService{
Service: "payments-sidecar-proxy",
Kind: structs.ServiceKindConnectProxy,
Port: 443,
TaggedAddresses: map[string]structs.ServiceAddress{
structs.TaggedAddressLAN: {
Address: "85.252.102.31",
Port: 443,
},
structs.TaggedAddressWAN: {
Address: "123.us-east-1.elb.notaws.com",
Port: 8443,
},
},
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
SNI: []string{
"payments.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/payments",
},
Protocol: "tcp",
},
},
},
},
},
},
},
{
CorrelationID: upstreamPeerWatchIDPrefix + refundsUID.String(),
Result: &structs.IndexedCheckServiceNodes{
Nodes: []structs.CheckServiceNode{
{
Node: &structs.Node{
Address: "106.96.90.233",
Datacenter: "cloud-dc",
},
Service: &structs.NodeService{
Service: "refunds-sidecar-proxy",
Kind: structs.ServiceKindConnectProxy,
Port: 443,
Connect: structs.ServiceConnect{
PeerMeta: &structs.PeeringServiceMeta{
SNI: []string{
"refunds.default.default.cloud.external." + peerTrustDomain,
},
SpiffeID: []string{
"spiffe://" + peerTrustDomain + "/ns/default/dc/cloud-dc/svc/refunds",
},
Protocol: "tcp",
},
},
},
},
},
},
},
{
CorrelationID: "mesh-gateway:dc1",
Result: &structs.IndexedCheckServiceNodes{
Nodes: structs.CheckServiceNodes{
structs.CheckServiceNode{
Node: &structs.Node{
ID: "mesh-gateway",
Node: "mesh-gateway",
Address: "10.0.0.1",
Datacenter: "dc1",
},
Service: &structs.NodeService{
Kind: structs.ServiceKindMeshGateway,
Service: "mesh-gateway",
Port: 1234,
TaggedAddresses: map[string]structs.ServiceAddress{
structs.TaggedAddressWAN: {Address: "172.100.0.14", Port: 8080},
},
EnterpriseMeta: *structs.DefaultEnterpriseMetaInDefaultPartition(),
},
},
},
},
},
})
}