diff --git a/.changelog/21871.txt b/.changelog/21871.txt new file mode 100644 index 0000000000..425ba2b5e2 --- /dev/null +++ b/.changelog/21871.txt @@ -0,0 +1,3 @@ +```release-note:bug +proxycfg: fix a bug where peered upstreams watches are canceled even when another target needs it. +``` diff --git a/.changelog/21908.txt b/.changelog/21908.txt new file mode 100644 index 0000000000..16668cf7eb --- /dev/null +++ b/.changelog/21908.txt @@ -0,0 +1,3 @@ +```release-note:security +Resolved issue where hcl would allow duplicates of the same key in acl policy configuration. +``` \ No newline at end of file diff --git a/.changelog/21909.txt b/.changelog/21909.txt new file mode 100644 index 0000000000..b49562f137 --- /dev/null +++ b/.changelog/21909.txt @@ -0,0 +1,3 @@ +```release-note:bug +state: ensure that identical manual virtual IP updates result in not bumping the modify indexes +``` diff --git a/.changelog/21930.txt b/.changelog/21930.txt new file mode 100644 index 0000000000..bfcf2748f0 --- /dev/null +++ b/.changelog/21930.txt @@ -0,0 +1,3 @@ +```release-note:security +api: Enforces strict content-type header validation to protect against XSS vulnerability. +``` \ No newline at end of file diff --git a/.changelog/21950.txt b/.changelog/21950.txt new file mode 100644 index 0000000000..e15f9d6c80 --- /dev/null +++ b/.changelog/21950.txt @@ -0,0 +1,3 @@ +```release-note:security +Removed ability to use bexpr to filter results without ACL read on endpoint +``` \ No newline at end of file diff --git a/.changelog/21951.txt b/.changelog/21951.txt new file mode 100644 index 0000000000..89796b0e37 --- /dev/null +++ b/.changelog/21951.txt @@ -0,0 +1,3 @@ +```release-note:security +Update `github.com/golang-jwt/jwt/v4` to v4.5.1 to address [GHSA-29wx-vh33-7x7r](https://github.com/golang-jwt/jwt/security/advisories/GHSA-29wx-vh33-7x7r). +``` \ No newline at end of file diff --git a/.changelog/22001.txt b/.changelog/22001.txt new file mode 100644 index 0000000000..04b211c9ed --- /dev/null +++ b/.changelog/22001.txt @@ -0,0 +1,3 @@ +```release-note:security +Update `golang.org/x/crypto` to v0.31.0 to address [GO-2024-3321](https://pkg.go.dev/vuln/GO-2024-3321). +``` \ No newline at end of file diff --git a/.changelog/22011.txt b/.changelog/22011.txt new file mode 100644 index 0000000000..cf4800dd08 --- /dev/null +++ b/.changelog/22011.txt @@ -0,0 +1,4 @@ +```release-note:security +Update `registry.access.redhat.com/ubi9-minimal` image to 9.5 to address [CVE-2024-3596](https://nvd.nist.gov/vuln/detail/CVE-2024-3596),[CVE-2024-2511](https://nvd.nist.gov/vuln/detail/CVE-2024-2511),[CVE-2024-26458](https://nvd.nist.gov/vuln/detail/CVE-2024-26458). +``` + diff --git a/.changelog/22021.txt b/.changelog/22021.txt new file mode 100644 index 0000000000..336f3c73e4 --- /dev/null +++ b/.changelog/22021.txt @@ -0,0 +1,3 @@ +```release-note:security +Update `golang.org/x/net` to v0.33.0 to address [GO-2024-3333](https://pkg.go.dev/vuln/GO-2024-3333). +``` \ No newline at end of file diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index d9af3f042a..3747a5f4dd 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,3 +1,5 @@ +* @hashicorp/consul-selfmanage-maintainers + # Techical Writer Review /website/content/docs/ @hashicorp/consul-docs @@ -6,36 +8,5 @@ # release configuration -/.release/ @hashicorp/release-engineering @hashicorp/github-consul-core -/.github/workflows/build.yml @hashicorp/release-engineering @hashicorp/github-consul-core - - -# Staff Engineer Review (protocol buffer definitions) -/proto-public/ @hashicorp/consul-core-staff -/proto/ @hashicorp/consul-core-staff - -# Staff Engineer Review (v1 architecture shared components) -/agent/cache/ @hashicorp/consul-core-staff -/agent/consul/fsm/ @hashicorp/consul-core-staff -/agent/consul/leader*.go @hashicorp/consul-core-staff -/agent/consul/server*.go @hashicorp/consul-core-staff -/agent/consul/state/ @hashicorp/consul-core-staff -/agent/consul/stream/ @hashicorp/consul-core-staff -/agent/submatview/ @hashicorp/consul-core-staff -/agent/blockingquery/ @hashicorp/consul-core-staff - -# Staff Engineer Review (raft/autopilot) -/agent/consul/autopilotevents/ @hashicorp/consul-core-staff -/agent/consul/autopilot*.go @hashicorp/consul-core-staff - -# Staff Engineer Review (v2 architecture shared components) -/internal/controller/ @hashicorp/consul-core-staff -/internal/resource/ @hashicorp/consul-core-staff -/internal/storage/ @hashicorp/consul-core-staff -/agent/consul/controller/ @hashicorp/consul-core-staff -/agent/grpc-external/services/resource/ @hashicorp/consul-core-staff - -# Staff Engineer Review (v1 security) -/acl/ @hashicorp/consul-core-staff -/agent/xds/rbac*.go @hashicorp/consul-core-staff -/agent/xds/jwt*.go @hashicorp/consul-core-staff +/.release/ @hashicorp/team-selfmanaged-releng @hashicorp/consul-selfmanage-maintainers +/.github/workflows/build.yml @hashicorp/team-selfmanaged-releng @hashicorp/consul-selfmanage-maintainers diff --git a/.release/security-scan.hcl b/.release/security-scan.hcl index 20c105f3b4..48325fbbd2 100644 --- a/.release/security-scan.hcl +++ b/.release/security-scan.hcl @@ -37,8 +37,8 @@ container { triage { suppress { vulnerabilities = [ - "CVE-2024-8096", # curl@8.9.1-r2, - "CVE-2024-9143", # openssl@3.3.2-r0, + "CVE-2024-4067", # libsolv@0:0.7.24-3.el9 + "CVE-2019-12900" # bzip2-libs@0:1.0.8-8.el9 ] paths = [ "internal/tools/proto-gen-rpc-glue/e2e/consul/*", @@ -79,6 +79,7 @@ binary { triage { suppress { vulnerabilities = [ + "GO-2022-0635", // github.com/aws/aws-sdk-go@v1.55.5 ] paths = [ "internal/tools/proto-gen-rpc-glue/e2e/consul/*", diff --git a/Dockerfile b/Dockerfile index dc617c5e04..037e437ae9 100644 --- a/Dockerfile +++ b/Dockerfile @@ -16,7 +16,7 @@ # Official docker image that includes binaries from releases.hashicorp.com. This # downloads the release from releases.hashicorp.com and therefore requires that # the release is published before building the Docker image. -FROM docker.mirror.hashicorp.services/alpine:3.20 as official +FROM docker.mirror.hashicorp.services/alpine:3.21 as official # This is the release of Consul to pull in. ARG VERSION @@ -29,6 +29,13 @@ LABEL org.opencontainers.image.authors="Consul Team " \ org.opencontainers.image.vendor="HashiCorp" \ org.opencontainers.image.title="consul" \ org.opencontainers.image.description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ + name="Consul" \ + maintainer="Consul Team " \ + vendor="HashiCorp" \ + release=${PRODUCT_REVISION} \ + revision=${PRODUCT_REVISION} \ + summary="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ + description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ version=${VERSION} # This is the location of the releases. @@ -112,7 +119,7 @@ CMD ["agent", "-dev", "-client", "0.0.0.0"] # Production docker image that uses CI built binaries. # Remember, this image cannot be built locally. -FROM docker.mirror.hashicorp.services/alpine:3.20 as default +FROM docker.mirror.hashicorp.services/alpine:3.21 as default ARG PRODUCT_VERSION ARG BIN_NAME @@ -137,6 +144,13 @@ LABEL org.opencontainers.image.authors="Consul Team " \ org.opencontainers.image.title="consul" \ org.opencontainers.image.description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ org.opencontainers.image.licenses="BSL-1.1" \ + name="Consul" \ + maintainer="Consul Team " \ + vendor="HashiCorp" \ + release=${PRODUCT_REVISION} \ + revision=${PRODUCT_REVISION} \ + summary="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ + description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ version=${PRODUCT_VERSION} COPY LICENSE /usr/share/doc/$PRODUCT_NAME/LICENSE.txt @@ -203,7 +217,7 @@ CMD ["agent", "-dev", "-client", "0.0.0.0"] # Red Hat UBI-based image # This target is used to build a Consul image for use on OpenShift. -FROM registry.access.redhat.com/ubi9-minimal:9.4 as ubi +FROM registry.access.redhat.com/ubi9-minimal:9.5 as ubi ARG PRODUCT_VERSION ARG PRODUCT_REVISION @@ -227,6 +241,13 @@ LABEL org.opencontainers.image.authors="Consul Team " \ org.opencontainers.image.title="consul" \ org.opencontainers.image.description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ org.opencontainers.image.licenses="BSL-1.1" \ + name="Consul" \ + maintainer="Consul Team " \ + vendor="HashiCorp" \ + release=${PRODUCT_REVISION} \ + revision=${PRODUCT_REVISION} \ + summary="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ + description="Consul is a datacenter runtime that provides service discovery, configuration, and orchestration." \ version=${PRODUCT_VERSION} COPY LICENSE /usr/share/doc/$PRODUCT_NAME/LICENSE.txt diff --git a/acl/acl.go b/acl/acl.go index 753db01516..035aa06db3 100644 --- a/acl/acl.go +++ b/acl/acl.go @@ -22,6 +22,9 @@ type Config struct { // WildcardName is the string that represents a request to authorize a wildcard permission WildcardName string + //by default errors, but in certain instances we want to make sure to maintain backwards compatabilty + WarnOnDuplicateKey bool + // embedded enterprise configuration EnterpriseConfig } diff --git a/acl/policy.go b/acl/policy.go index 86c9e83cfc..54eb4e6587 100644 --- a/acl/policy.go +++ b/acl/policy.go @@ -310,8 +310,8 @@ func (pr *PolicyRules) Validate(conf *Config) error { return nil } -func parse(rules string, conf *Config, meta *EnterprisePolicyMeta) (*Policy, error) { - p, err := decodeRules(rules, conf, meta) +func parse(rules string, warnOnDuplicateKey bool, conf *Config, meta *EnterprisePolicyMeta) (*Policy, error) { + p, err := decodeRules(rules, warnOnDuplicateKey, conf, meta) if err != nil { return nil, err } @@ -338,7 +338,11 @@ func NewPolicyFromSource(rules string, conf *Config, meta *EnterprisePolicyMeta) var policy *Policy var err error - policy, err = parse(rules, conf, meta) + warnOnDuplicateKey := false + if conf != nil { + warnOnDuplicateKey = conf.WarnOnDuplicateKey + } + policy, err = parse(rules, warnOnDuplicateKey, conf, meta) return policy, err } diff --git a/acl/policy_ce.go b/acl/policy_ce.go index fe139ef7ab..457563f048 100644 --- a/acl/policy_ce.go +++ b/acl/policy_ce.go @@ -7,8 +7,9 @@ package acl import ( "fmt" - + "github.com/hashicorp/go-hclog" "github.com/hashicorp/hcl" + "strings" ) // EnterprisePolicyMeta stub @@ -30,12 +31,28 @@ func (r *EnterprisePolicyRules) Validate(*Config) error { return nil } -func decodeRules(rules string, _ *Config, _ *EnterprisePolicyMeta) (*Policy, error) { +func decodeRules(rules string, warnOnDuplicateKey bool, _ *Config, _ *EnterprisePolicyMeta) (*Policy, error) { p := &Policy{} - if err := hcl.Decode(p, rules); err != nil { + err := hcl.DecodeErrorOnDuplicates(p, rules) + + if errIsDuplicateKey(err) && warnOnDuplicateKey { + //because the snapshot saves the unparsed rules we have to assume some snapshots exist that shouldn't fail, but + // have duplicates + if err := hcl.Decode(p, rules); err != nil { + hclog.Default().Warn("Warning- Duplicate key in ACL Policy ignored", "errorMessage", err.Error()) + return nil, fmt.Errorf("Failed to parse ACL rules: %v", err) + } + } else if err != nil { return nil, fmt.Errorf("Failed to parse ACL rules: %v", err) } return p, nil } + +func errIsDuplicateKey(err error) bool { + if err == nil { + return false + } + return strings.Contains(err.Error(), "was already set. Each argument can only be defined once") +} diff --git a/acl/policy_test.go b/acl/policy_test.go index 2ce0b32892..e09ae535e1 100644 --- a/acl/policy_test.go +++ b/acl/policy_test.go @@ -342,6 +342,12 @@ func TestPolicySourceParse(t *testing.T) { RulesJSON: `{ "acl": "list" }`, // there is no list policy but this helps to exercise another check in isPolicyValid Err: "Invalid acl policy", }, + { + Name: "Bad Policy - Duplicate ACL Key", + Rules: `acl="read" + acl="write"`, + Err: "Failed to parse ACL rules: The argument \"acl\" at", + }, { Name: "Bad Policy - Agent", Rules: `agent "foo" { policy = "nope" }`, diff --git a/agent/agent_endpoint.go b/agent/agent_endpoint.go index 996212c97e..0a4402ffcb 100644 --- a/agent/agent_endpoint.go +++ b/agent/agent_endpoint.go @@ -380,16 +380,14 @@ func (s *HTTPHandlers) AgentServices(resp http.ResponseWriter, req *http.Request return nil, err } - raw, err := filter.Execute(agentSvcs) - if err != nil { - return nil, err - } - agentSvcs = raw.(map[string]*api.AgentService) - - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure total (and the filter-by-acls header we set below) - // do not include results that would be filtered out even if the user did have - // permission. + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. total := len(agentSvcs) if err := s.agent.filterServicesWithAuthorizer(authz, agentSvcs); err != nil { return nil, err @@ -407,6 +405,12 @@ func (s *HTTPHandlers) AgentServices(resp http.ResponseWriter, req *http.Request setResultsFilteredByACLs(resp, total != len(agentSvcs)) } + raw, err := filter.Execute(agentSvcs) + if err != nil { + return nil, err + } + agentSvcs = raw.(map[string]*api.AgentService) + return agentSvcs, nil } @@ -540,16 +544,14 @@ func (s *HTTPHandlers) AgentChecks(resp http.ResponseWriter, req *http.Request) } } - raw, err := filter.Execute(agentChecks) - if err != nil { - return nil, err - } - agentChecks = raw.(map[types.CheckID]*structs.HealthCheck) - - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure total (and the filter-by-acls header we set below) - // do not include results that would be filtered out even if the user did have - // permission. + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. total := len(agentChecks) if err := s.agent.filterChecksWithAuthorizer(authz, agentChecks); err != nil { return nil, err @@ -567,6 +569,12 @@ func (s *HTTPHandlers) AgentChecks(resp http.ResponseWriter, req *http.Request) setResultsFilteredByACLs(resp, total != len(agentChecks)) } + raw, err := filter.Execute(agentChecks) + if err != nil { + return nil, err + } + agentChecks = raw.(map[types.CheckID]*structs.HealthCheck) + return agentChecks, nil } @@ -623,21 +631,14 @@ func (s *HTTPHandlers) AgentMembers(resp http.ResponseWriter, req *http.Request) } } - // filter the members by parsed filter expression - var filterExpression string - s.parseFilter(req, &filterExpression) - if filterExpression != "" { - filter, err := bexpr.CreateFilter(filterExpression, nil, members) - if err != nil { - return nil, err - } - raw, err := filter.Execute(members) - if err != nil { - return nil, err - } - members = raw.([]serf.Member) - } - + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. total := len(members) if err := s.agent.filterMembers(token, &members); err != nil { return nil, err @@ -655,6 +656,21 @@ func (s *HTTPHandlers) AgentMembers(resp http.ResponseWriter, req *http.Request) setResultsFilteredByACLs(resp, total != len(members)) } + // filter the members by parsed filter expression + var filterExpression string + s.parseFilter(req, &filterExpression) + if filterExpression != "" { + filter, err := bexpr.CreateFilter(filterExpression, nil, members) + if err != nil { + return nil, err + } + raw, err := filter.Execute(members) + if err != nil { + return nil, err + } + members = raw.([]serf.Member) + } + return members, nil } diff --git a/agent/agent_endpoint_test.go b/agent/agent_endpoint_test.go index 69551d7c36..451c412b4b 100644 --- a/agent/agent_endpoint_test.go +++ b/agent/agent_endpoint_test.go @@ -433,6 +433,60 @@ func TestAgent_Services_ACLFilter(t *testing.T) { require.Len(t, val, 2) require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) }) + + // ensure ACL filtering occurs before bexpr filtering. + const bexprMatchingUserTokenPermissions = "Service matches `web.*`" + const bexprNotMatchingUserTokenPermissions = "Service matches `api.*`" + + tokenWithWebRead := testCreateToken(t, a, ` + service "web" { + policy = "read" + } + `) + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/services?filter="+url.QueryEscape(bexprMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithWebRead) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + var val map[string]*api.AgentService + err := dec.Decode(&val) + if err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 1) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/services?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithWebRead) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + var val map[string]*api.AgentService + err := dec.Decode(&val) + if err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/services?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + var val map[string]*api.AgentService + err := dec.Decode(&val) + if err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) } func TestAgent_Service(t *testing.T) { @@ -1432,6 +1486,57 @@ func TestAgent_Checks_ACLFilter(t *testing.T) { require.Len(t, val, 2) require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) }) + + // ensure ACL filtering occurs before bexpr filtering. + const bexprMatchingUserTokenPermissions = "ServiceName matches `web.*`" + const bexprNotMatchingUserTokenPermissions = "ServiceName matches `api.*`" + + tokenWithWebRead := testCreateToken(t, a, ` + service "web" { + policy = "read" + } + `) + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/checks?filter="+url.QueryEscape(bexprMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithWebRead) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make(map[types.CheckID]*structs.HealthCheck) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 1) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/checks?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithWebRead) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make(map[types.CheckID]*structs.HealthCheck) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/checks?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make(map[types.CheckID]*structs.HealthCheck) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) } func TestAgent_Self(t *testing.T) { @@ -2110,6 +2215,57 @@ func TestAgent_Members_ACLFilter(t *testing.T) { require.Len(t, val, 2) require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) }) + + // ensure ACL filtering occurs before bexpr filtering. + bexprMatchingUserTokenPermissions := fmt.Sprintf("Name matches `%s.*`", b.Config.NodeName) + bexprNotMatchingUserTokenPermissions := fmt.Sprintf("Name matches `%s.*`", a.Config.NodeName) + + tokenWithReadOnMemberB := testCreateToken(t, a, fmt.Sprintf(` + node "%s" { + policy = "read" + } + `, b.Config.NodeName)) + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/members?filter="+url.QueryEscape(bexprMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithReadOnMemberB) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make([]serf.Member, 0) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 1) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/members?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + req.Header.Add("X-Consul-Token", tokenWithReadOnMemberB) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make([]serf.Member, 0) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.NotEmpty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req, _ := http.NewRequest("GET", "/v1/agent/members?filter="+url.QueryEscape(bexprNotMatchingUserTokenPermissions), nil) + resp := httptest.NewRecorder() + a.srv.h.ServeHTTP(resp, req) + dec := json.NewDecoder(resp.Body) + val := make([]serf.Member, 0) + if err := dec.Decode(&val); err != nil { + t.Fatalf("Err: %v", err) + } + require.Len(t, val, 0) + require.Empty(t, resp.Header().Get("X-Consul-Results-Filtered-By-ACLs")) + }) } func TestAgent_Join(t *testing.T) { diff --git a/agent/catalog_endpoint_test.go b/agent/catalog_endpoint_test.go index c06efc748c..af1f2d36e3 100644 --- a/agent/catalog_endpoint_test.go +++ b/agent/catalog_endpoint_test.go @@ -122,7 +122,7 @@ func TestCatalogDeregister(t *testing.T) { a := NewTestAgent(t, "") defer a.Shutdown() - // Register node + // Deregister node args := &structs.DeregisterRequest{Node: "foo"} req, _ := http.NewRequest("PUT", "/v1/catalog/deregister", jsonReader(args)) obj, err := a.srv.CatalogDeregister(nil, req) diff --git a/agent/consul/acl_endpoint_test.go b/agent/consul/acl_endpoint_test.go index 4ed159a8aa..2b00aae6d5 100644 --- a/agent/consul/acl_endpoint_test.go +++ b/agent/consul/acl_endpoint_test.go @@ -2169,6 +2169,22 @@ func TestACLEndpoint_PolicySet(t *testing.T) { require.Error(t, err) }) + t.Run("Key Dup", func(t *testing.T) { + req := structs.ACLPolicySetRequest{ + Datacenter: "dc1", + Policy: structs.ACLPolicy{ + Description: "foobar", + Name: "baz2", + Rules: "service \"\" { policy = \"read\" policy = \"write\" }", + }, + WriteRequest: structs.WriteRequest{Token: TestDefaultInitialManagementToken}, + } + resp := structs.ACLPolicy{} + + err := aclEp.PolicySet(&req, &resp) + require.Error(t, err) + }) + t.Run("Update it", func(t *testing.T) { req := structs.ACLPolicySetRequest{ Datacenter: "dc1", diff --git a/agent/consul/catalog_endpoint.go b/agent/consul/catalog_endpoint.go index 36426afe25..09c23d90c5 100644 --- a/agent/consul/catalog_endpoint.go +++ b/agent/consul/catalog_endpoint.go @@ -533,19 +533,24 @@ func (c *Catalog) ListNodes(args *structs.DCSpecificRequest, reply *structs.Inde return nil } + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := c.srv.filterACL(args.Token, reply); err != nil { + return err + } + raw, err := filter.Execute(reply.Nodes) if err != nil { return err } reply.Nodes = raw.(structs.Nodes) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := c.srv.filterACL(args.Token, reply); err != nil { - return err - } - return c.srv.sortNodesByDistanceFrom(args.Source, reply.Nodes) }) } @@ -607,14 +612,25 @@ func (c *Catalog) ListServices(args *structs.DCSpecificRequest, reply *structs.I return nil } - raw, err := filter.Execute(serviceNodes) + // need to temporarily create an IndexedServiceNode so that the ACL filter can be applied + // to the service nodes and then re-use those same node to run the filter expression. + idxServiceNodeReply := &structs.IndexedServiceNodes{ + ServiceNodes: serviceNodes, + QueryMeta: reply.QueryMeta, + } + + // enforce ACLs + c.srv.filterACLWithAuthorizer(authz, idxServiceNodeReply) + + // run the filter expression + raw, err := filter.Execute(idxServiceNodeReply.ServiceNodes) if err != nil { return err } + // convert the result back to the original type reply.Services = servicesTagsByName(raw.(structs.ServiceNodes)) - - c.srv.filterACLWithAuthorizer(authz, reply) + reply.QueryMeta = idxServiceNodeReply.QueryMeta return nil }) @@ -813,6 +829,18 @@ func (c *Catalog) ServiceNodes(args *structs.ServiceSpecificRequest, reply *stru reply.ServiceNodes = filtered } + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := c.srv.filterACL(args.Token, reply); err != nil { + return err + } + // This is safe to do even when the filter is nil - its just a no-op then raw, err := filter.Execute(reply.ServiceNodes) if err != nil { @@ -820,13 +848,6 @@ func (c *Catalog) ServiceNodes(args *structs.ServiceSpecificRequest, reply *stru } reply.ServiceNodes = raw.(structs.ServiceNodes) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := c.srv.filterACL(args.Token, reply); err != nil { - return err - } - return c.srv.sortNodesByDistanceFrom(args.Source, reply.ServiceNodes) }) @@ -904,6 +925,18 @@ func (c *Catalog) NodeServices(args *structs.NodeSpecificRequest, reply *structs } reply.Index, reply.NodeServices = index, services + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := c.srv.filterACL(args.Token, reply); err != nil { + return err + } + if reply.NodeServices != nil { raw, err := filter.Execute(reply.NodeServices.Services) if err != nil { @@ -912,13 +945,6 @@ func (c *Catalog) NodeServices(args *structs.NodeSpecificRequest, reply *structs reply.NodeServices.Services = raw.(map[string]*structs.NodeService) } - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := c.srv.filterACL(args.Token, reply); err != nil { - return err - } - return nil }) } @@ -1009,21 +1035,26 @@ func (c *Catalog) NodeServiceList(args *structs.NodeSpecificRequest, reply *stru if mergedServices != nil { reply.NodeServices = *mergedServices - - raw, err := filter.Execute(reply.NodeServices.Services) - if err != nil { - return err - } - reply.NodeServices.Services = raw.([]*structs.NodeService) } - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. if err := c.srv.filterACL(args.Token, reply); err != nil { return err } + raw, err := filter.Execute(reply.NodeServices.Services) + if err != nil { + return err + } + reply.NodeServices.Services = raw.([]*structs.NodeService) + return nil }) } diff --git a/agent/consul/catalog_endpoint_test.go b/agent/consul/catalog_endpoint_test.go index 18827dc981..1f3311b0cd 100644 --- a/agent/consul/catalog_endpoint_test.go +++ b/agent/consul/catalog_endpoint_test.go @@ -984,6 +984,63 @@ func TestCatalog_RPC_Filter(t *testing.T) { require.Equal(t, "baz", out.Nodes[0].Node) }) + t.Run("ListServices", func(t *testing.T) { + args := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "redis", + QueryOptions: structs.QueryOptions{Filter: "ServiceMeta.version == 1"}, + } + + out := new(structs.IndexedServices) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &args, &out)) + require.Len(t, out.Services, 2) + require.Len(t, out.Services["redis"], 1) + require.Len(t, out.Services["web"], 2) + + args.Filter = "ServiceMeta.version == 2" + out = new(structs.IndexedServices) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &args, &out)) + require.Len(t, out.Services, 4) + require.Len(t, out.Services["redis"], 1) + require.Len(t, out.Services["web"], 2) + require.Len(t, out.Services["critical"], 1) + require.Len(t, out.Services["warning"], 1) + }) + + t.Run("NodeServices", func(t *testing.T) { + args := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: "baz", + QueryOptions: structs.QueryOptions{Filter: "Service == web"}, + } + + out := new(structs.IndexedNodeServices) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &args, &out)) + require.Len(t, out.NodeServices.Services, 2) + + args.Filter = "Service == web and Meta.version == 2" + out = new(structs.IndexedNodeServices) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &args, &out)) + require.Len(t, out.NodeServices.Services, 1) + }) + + t.Run("NodeServiceList", func(t *testing.T) { + args := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: "baz", + QueryOptions: structs.QueryOptions{Filter: "Service == web"}, + } + + out := new(structs.IndexedNodeServiceList) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &args, &out)) + require.Len(t, out.NodeServices.Services, 2) + + args.Filter = "Service == web and Meta.version == 2" + out = new(structs.IndexedNodeServiceList) + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &args, &out)) + require.Len(t, out.NodeServices.Services, 1) + }) + t.Run("ServiceNodes", func(t *testing.T) { args := structs.ServiceSpecificRequest{ Datacenter: "dc1", @@ -1006,22 +1063,6 @@ func TestCatalog_RPC_Filter(t *testing.T) { require.Equal(t, "foo", out.ServiceNodes[0].Node) }) - t.Run("NodeServices", func(t *testing.T) { - args := structs.NodeSpecificRequest{ - Datacenter: "dc1", - Node: "baz", - QueryOptions: structs.QueryOptions{Filter: "Service == web"}, - } - - out := new(structs.IndexedNodeServices) - require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &args, &out)) - require.Len(t, out.NodeServices.Services, 2) - - args.Filter = "Service == web and Meta.version == 2" - out = new(structs.IndexedNodeServices) - require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &args, &out)) - require.Len(t, out.NodeServices.Services, 1) - }) } func TestCatalog_ListNodes_StaleRead(t *testing.T) { @@ -1332,6 +1373,7 @@ func TestCatalog_ListNodes_ACLFilter(t *testing.T) { Datacenter: "dc1", } + readToken := token("read") t.Run("deny", func(t *testing.T) { args.Token = token("deny") @@ -1348,7 +1390,7 @@ func TestCatalog_ListNodes_ACLFilter(t *testing.T) { }) t.Run("allow", func(t *testing.T) { - args.Token = token("read") + args.Token = readToken var reply structs.IndexedNodes if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListNodes", &args, &reply); err != nil { @@ -1361,6 +1403,67 @@ func TestCatalog_ListNodes_ACLFilter(t *testing.T) { t.Fatal("ResultsFilteredByACLs should not true") } }) + + // Register additional node + regArgs := &structs.RegisterRequest{ + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.1", + WriteRequest: structs.WriteRequest{ + Token: "root", + }, + } + + var out struct{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.Register", regArgs, &out)) + + bexprMatchingUserTokenPermissions := fmt.Sprintf("Node matches `%s.*`", s1.config.NodeName) + const bexpNotMatchingUserTokenPermissions = "Node matches `node-deny.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + var reply structs.IndexedNodes + args = structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: readToken, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodes{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.ListNodes", &args, &reply)) + require.Equal(t, 1, len(reply.Nodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + var reply structs.IndexedNodes + args = structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: readToken, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodes{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.ListNodes", &args, &reply)) + require.Empty(t, reply.Nodes) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + var reply structs.IndexedNodes + args = structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodes{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.ListNodes", &args, &reply)) + require.Empty(t, reply.Nodes) + require.False(t, reply.ResultsFilteredByACLs) + }) } func Benchmark_Catalog_ListNodes(t *testing.B) { @@ -2758,6 +2861,14 @@ service "foo" { node_prefix "" { policy = "read" } + +node "node-deny" { + policy = "deny" +} + +service "service-deny" { + policy = "deny" +} ` token = createToken(t, codec, rules) @@ -2915,23 +3026,76 @@ func TestCatalog_ListServices_FilterACL(t *testing.T) { defer codec.Close() testrpc.WaitForTestAgent(t, srv.RPC, "dc1", testrpc.WithToken("root")) - opt := structs.DCSpecificRequest{ - Datacenter: "dc1", - QueryOptions: structs.QueryOptions{Token: token}, - } - reply := structs.IndexedServices{} - if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &opt, &reply); err != nil { - t.Fatalf("err: %s", err) - } - if _, ok := reply.Services["foo"]; !ok { - t.Fatalf("bad: %#v", reply.Services) - } - if _, ok := reply.Services["bar"]; ok { - t.Fatalf("bad: %#v", reply.Services) - } - if !reply.QueryMeta.ResultsFilteredByACLs { - t.Fatal("ResultsFilteredByACLs should be true") - } + t.Run("request with user token without filter param sets ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{Token: token}, + } + reply := structs.IndexedServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + if _, ok := reply.Services["foo"]; !ok { + t.Fatalf("bad: %#v", reply.Services) + } + if _, ok := reply.Services["bar"]; ok { + t.Fatalf("bad: %#v", reply.Services) + } + if !reply.QueryMeta.ResultsFilteredByACLs { + t.Fatal("ResultsFilteredByACLs should be true") + } + }) + + const bexprMatchingUserTokenPermissions = "ServiceName matches `f.*`" + const bexpNotMatchingUserTokenPermissions = "ServiceName matches `b.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ListServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.Services)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestCatalog_ServiceNodes_FilterACL(t *testing.T) { @@ -2982,11 +3146,80 @@ func TestCatalog_ServiceNodes_FilterACL(t *testing.T) { } require.True(t, reply.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + bexprMatchingUserTokenPermissions := fmt.Sprintf("Node matches `%s.*`", srv.config.NodeName) + const bexpNotMatchingUserTokenPermissions = "Node matches `node-deny.*`" + + // Register a service of the same name on the denied node + regArg := structs.RegisterRequest{ + Datacenter: "dc1", + Node: "node-deny", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "foo", + Service: "foo", + }, + Check: &structs.HealthCheck{ + CheckID: "service:foo", + Name: "service:foo", + ServiceID: "foo", + Status: api.HealthPassing, + }, + WriteRequest: structs.WriteRequest{Token: "root"}, + } + if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", ®Arg, nil); err != nil { + t.Fatalf("err: %s", err) + } + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt = structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedServiceNodes{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ServiceNodes", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.ServiceNodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt = structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedServiceNodes{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ServiceNodes", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.ServiceNodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + opt = structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedServiceNodes{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.ServiceNodes", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.ServiceNodes)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestCatalog_NodeServices_ACL(t *testing.T) { @@ -3075,6 +3308,139 @@ func TestCatalog_NodeServices_FilterACL(t *testing.T) { svc, ok := reply.NodeServices.Services["foo"] require.True(t, ok) require.Equal(t, "foo", svc.ID) + + const bexprMatchingUserTokenPermissions = "Service matches `f.*`" + const bexpNotMatchingUserTokenPermissions = "Service matches `b.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.NodeServices.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.NodeServices.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServices{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServices", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Nil(t, reply.NodeServices) + require.False(t, reply.ResultsFilteredByACLs) + }) +} + +func TestCatalog_NodeServicesList_FilterACL(t *testing.T) { + if testing.Short() { + t.Skip("too slow for testing.Short") + } + + t.Parallel() + dir, token, srv, codec := testACLFilterServer(t) + defer os.RemoveAll(dir) + defer srv.Shutdown() + defer codec.Close() + testrpc.WaitForTestAgent(t, srv.RPC, "dc1", testrpc.WithToken("root")) + + opt := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{Token: token}, + } + + var reply structs.IndexedNodeServiceList + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &opt, &reply)) + + require.NotNil(t, reply.NodeServices) + require.Len(t, reply.NodeServices.Services, 1) + + const bexprMatchingUserTokenPermissions = "Service matches `f.*`" + const bexpNotMatchingUserTokenPermissions = "Service matches `b.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServiceList{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.NodeServices.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServiceList{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.NodeServices.Services)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply = structs.IndexedNodeServiceList{} + if err := msgpackrpc.CallWithCodec(codec, "Catalog.NodeServiceList", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Empty(t, reply.NodeServices.Services) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestCatalog_GatewayServices_TerminatingGateway(t *testing.T) { diff --git a/agent/consul/config_endpoint.go b/agent/consul/config_endpoint.go index 96906dac68..a0d78cc6d2 100644 --- a/agent/consul/config_endpoint.go +++ b/agent/consul/config_endpoint.go @@ -9,7 +9,6 @@ import ( "time" metrics "github.com/armon/go-metrics" - "github.com/armon/go-metrics/prometheus" hashstructure_v2 "github.com/mitchellh/hashstructure/v2" "github.com/hashicorp/go-bexpr" @@ -22,33 +21,6 @@ import ( "github.com/hashicorp/consul/agent/structs" ) -var ConfigSummaries = []prometheus.SummaryDefinition{ - { - Name: []string{"config_entry", "apply"}, - Help: "", - }, - { - Name: []string{"config_entry", "get"}, - Help: "", - }, - { - Name: []string{"config_entry", "list"}, - Help: "", - }, - { - Name: []string{"config_entry", "listAll"}, - Help: "", - }, - { - Name: []string{"config_entry", "delete"}, - Help: "", - }, - { - Name: []string{"config_entry", "resolve_service_config"}, - Help: "", - }, -} - // The ConfigEntry endpoint is used to query centralized config information type ConfigEntry struct { srv *Server @@ -280,7 +252,14 @@ func (c *ConfigEntry) List(args *structs.ConfigEntryQuery, reply *structs.Indexe return err } - // Filter the entries returned by ACL permissions. + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. filteredEntries := make([]structs.ConfigEntry, 0, len(entries)) for _, entry := range entries { if err := entry.CanRead(authz); err != nil { diff --git a/agent/consul/config_endpoint_test.go b/agent/consul/config_endpoint_test.go index 49a10dce21..cf503ee525 100644 --- a/agent/consul/config_endpoint_test.go +++ b/agent/consul/config_endpoint_test.go @@ -783,7 +783,7 @@ service "foo" { } operator = "read" ` - id := createToken(t, codec, rules) + token := createToken(t, codec, rules) // Create some dummy service/proxy configs to be looked up. state := s1.fsm.State() @@ -804,7 +804,7 @@ operator = "read" args := structs.ConfigEntryQuery{ Kind: structs.ServiceDefaults, Datacenter: s1.config.Datacenter, - QueryOptions: structs.QueryOptions{Token: id}, + QueryOptions: structs.QueryOptions{Token: token}, } var out structs.IndexedConfigEntries err := msgpackrpc.CallWithCodec(codec, "ConfigEntry.List", &args, &out) @@ -828,6 +828,58 @@ operator = "read" require.Equal(t, structs.ProxyConfigGlobal, proxyConf.Name) require.Equal(t, structs.ProxyDefaults, proxyConf.Kind) require.False(t, out.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be false") + + // ensure ACL filtering occurs before bexpr filtering. + const bexprMatchingUserTokenPermissions = "Name matches `f.*`" + const bexprNotMatchingUserTokenPermissions = "Name matches `db.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + args = structs.ConfigEntryQuery{ + Kind: structs.ServiceDefaults, + Datacenter: s1.config.Datacenter, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + var reply structs.IndexedConfigEntries + err = msgpackrpc.CallWithCodec(codec, "ConfigEntry.List", &args, &reply) + require.NoError(t, err) + require.Equal(t, 1, len(reply.Entries)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + args = structs.ConfigEntryQuery{ + Kind: structs.ServiceDefaults, + Datacenter: s1.config.Datacenter, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + var reply structs.IndexedConfigEntries + err = msgpackrpc.CallWithCodec(codec, "ConfigEntry.List", &args, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.Entries)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + args = structs.ConfigEntryQuery{ + Kind: structs.ServiceDefaults, + Datacenter: s1.config.Datacenter, + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + var reply structs.IndexedConfigEntries + err = msgpackrpc.CallWithCodec(codec, "ConfigEntry.List", &args, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.Entries)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestConfigEntry_ListAll_ACLDeny(t *testing.T) { diff --git a/agent/consul/health_endpoint.go b/agent/consul/health_endpoint.go index 6f00ec4b08..4ded41bcce 100644 --- a/agent/consul/health_endpoint.go +++ b/agent/consul/health_endpoint.go @@ -63,19 +63,24 @@ func (h *Health) ChecksInState(args *structs.ChecksInStateRequest, } reply.Index, reply.HealthChecks = index, checks + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := h.srv.filterACL(args.Token, reply); err != nil { + return err + } + raw, err := filter.Execute(reply.HealthChecks) if err != nil { return err } reply.HealthChecks = raw.(structs.HealthChecks) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := h.srv.filterACL(args.Token, reply); err != nil { - return err - } - return h.srv.sortNodesByDistanceFrom(args.Source, reply.HealthChecks) }) } @@ -111,19 +116,24 @@ func (h *Health) NodeChecks(args *structs.NodeSpecificRequest, } reply.Index, reply.HealthChecks = index, checks + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := h.srv.filterACL(args.Token, reply); err != nil { + return err + } + raw, err := filter.Execute(reply.HealthChecks) if err != nil { return err } reply.HealthChecks = raw.(structs.HealthChecks) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := h.srv.filterACL(args.Token, reply); err != nil { - return err - } - return nil }) } @@ -303,6 +313,18 @@ func (h *Health) ServiceNodes(args *structs.ServiceSpecificRequest, reply *struc thisReply.Nodes = nodeMetaFilter(arg.NodeMetaFilters, thisReply.Nodes) } + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := h.srv.filterACL(arg.Token, &thisReply); err != nil { + return err + } + raw, err := filter.Execute(thisReply.Nodes) if err != nil { return err @@ -310,13 +332,6 @@ func (h *Health) ServiceNodes(args *structs.ServiceSpecificRequest, reply *struc filteredNodes := raw.(structs.CheckServiceNodes) thisReply.Nodes = filteredNodes.Filter(structs.CheckServiceNodeFilterOptions{FilterType: arg.HealthFilterType}) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := h.srv.filterACL(arg.Token, &thisReply); err != nil { - return err - } - if err := h.srv.sortNodesByDistanceFrom(arg.Source, thisReply.Nodes); err != nil { return err } diff --git a/agent/consul/health_endpoint_test.go b/agent/consul/health_endpoint_test.go index 07f23cc2e0..fef8d285a6 100644 --- a/agent/consul/health_endpoint_test.go +++ b/agent/consul/health_endpoint_test.go @@ -1527,11 +1527,62 @@ func TestHealth_NodeChecks_FilterACL(t *testing.T) { require.True(t, found, "bad: %#v", reply.HealthChecks) require.True(t, reply.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + const bexprMatchingUserTokenPermissions = "ServiceName matches `f.*`" + const bexprNotMatchingUserTokenPermissions = "ServiceName matches `b.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + + if err := msgpackrpc.CallWithCodec(codec, "Health.NodeChecks", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + + if err := msgpackrpc.CallWithCodec(codec, "Health.NodeChecks", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + opt := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: srv.config.NodeName, + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + + if err := msgpackrpc.CallWithCodec(codec, "Health.NodeChecks", &opt, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.HealthChecks)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestHealth_ServiceChecks_FilterACL(t *testing.T) { @@ -1571,11 +1622,77 @@ func TestHealth_ServiceChecks_FilterACL(t *testing.T) { require.Empty(t, reply.HealthChecks) require.True(t, reply.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + // Register a service of the same name on the denied node + regArg := structs.RegisterRequest{ + Datacenter: "dc1", + Node: "node-deny", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "foo", + Service: "foo", + }, + Check: &structs.HealthCheck{ + CheckID: "service:foo", + Name: "service:foo", + ServiceID: "foo", + Status: api.HealthPassing, + }, + WriteRequest: structs.WriteRequest{Token: "root"}, + } + if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", ®Arg, nil); err != nil { + t.Fatalf("err: %s", err) + } + const bexprMatchingUserTokenPermissions = "ServiceName matches `f.*`" + const bexprNotMatchingUserTokenPermissions = "Node matches `node-deny.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceChecks", &opt, &reply) + require.NoError(t, err) + + require.Equal(t, 1, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceChecks", &opt, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceChecks", &opt, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.HealthChecks)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestHealth_ServiceNodes_FilterACL(t *testing.T) { @@ -1607,11 +1724,77 @@ func TestHealth_ServiceNodes_FilterACL(t *testing.T) { require.Empty(t, reply.Nodes) require.True(t, reply.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + // Register a service of the same name on the denied node + regArg := structs.RegisterRequest{ + Datacenter: "dc1", + Node: "node-deny", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "foo", + Service: "foo", + }, + Check: &structs.HealthCheck{ + CheckID: "service:foo", + Name: "service:foo", + ServiceID: "foo", + Status: api.HealthPassing, + }, + WriteRequest: structs.WriteRequest{Token: "root"}, + } + if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", ®Arg, nil); err != nil { + t.Fatalf("err: %s", err) + } + const bexprMatchingUserTokenPermissions = "Service.Service matches `f.*`" + const bexprNotMatchingUserTokenPermissions = "Node.Node matches `node-deny.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedCheckServiceNodes{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceNodes", &opt, &reply) + require.NoError(t, err) + + require.Equal(t, 1, len(reply.Nodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedCheckServiceNodes{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceNodes", &opt, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.Nodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + opt := structs.ServiceSpecificRequest{ + Datacenter: "dc1", + ServiceName: "foo", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedCheckServiceNodes{} + err := msgpackrpc.CallWithCodec(codec, "Health.ServiceNodes", &opt, &reply) + require.NoError(t, err) + require.Zero(t, len(reply.Nodes)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestHealth_ChecksInState_FilterACL(t *testing.T) { @@ -1647,11 +1830,59 @@ func TestHealth_ChecksInState_FilterACL(t *testing.T) { require.True(t, found, "missing service 'foo': %#v", reply.HealthChecks) require.True(t, reply.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + const bexprMatchingUserTokenPermissions = "ServiceName matches `f.*`" + const bexprNotMatchingUserTokenPermissions = "ServiceName matches `b.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.ChecksInStateRequest{ + Datacenter: "dc1", + State: api.HealthPassing, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + if err := msgpackrpc.CallWithCodec(codec, "Health.ChecksInState", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.ChecksInStateRequest{ + Datacenter: "dc1", + State: api.HealthPassing, + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + if err := msgpackrpc.CallWithCodec(codec, "Health.ChecksInState", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.HealthChecks)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would normally match but without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := structs.ChecksInStateRequest{ + Datacenter: "dc1", + State: api.HealthPassing, + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexprNotMatchingUserTokenPermissions, + }, + } + reply := structs.IndexedHealthChecks{} + if err := msgpackrpc.CallWithCodec(codec, "Health.ChecksInState", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.HealthChecks)) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestHealth_RPC_Filter(t *testing.T) { diff --git a/agent/consul/intention_endpoint.go b/agent/consul/intention_endpoint.go index df05428145..6fc7f8132a 100644 --- a/agent/consul/intention_endpoint.go +++ b/agent/consul/intention_endpoint.go @@ -550,19 +550,25 @@ func (s *Intention) List(args *structs.IntentionListRequest, reply *structs.Inde } else { reply.DataOrigin = structs.IntentionDataOriginLegacy } + + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := s.srv.filterACL(args.Token, reply); err != nil { + return err + } + raw, err := filter.Execute(reply.Intentions) if err != nil { return err } reply.Intentions = raw.(structs.Intentions) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := s.srv.filterACL(args.Token, reply); err != nil { - return err - } - return nil }, ) diff --git a/agent/consul/intention_endpoint_test.go b/agent/consul/intention_endpoint_test.go index 08480501d7..d1a70fd5bc 100644 --- a/agent/consul/intention_endpoint_test.go +++ b/agent/consul/intention_endpoint_test.go @@ -1639,6 +1639,11 @@ func TestIntentionList_acl(t *testing.T) { token, err := upsertTestTokenWithPolicyRules(codec, TestDefaultInitialManagementToken, "dc1", `service_prefix "foo" { policy = "write" }`) require.NoError(t, err) + const ( + bexprMatch = "DestinationName matches `f.*`" + bexprNoMatch = "DestinationName matches `nomatch.*`" + ) + // Create a few records for _, name := range []string{"foobar", "bar", "baz"} { ixn := structs.IntentionRequest{ @@ -1691,12 +1696,29 @@ func TestIntentionList_acl(t *testing.T) { require.True(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") }) - t.Run("filtered", func(t *testing.T) { + // maskResultsFilteredByACLs() in rpc.go sets ResultsFilteredByACLs to false if the token is an empty string + // after resp.QueryMeta.ResultsFilteredByACLs has been determined to be true from filterACLs(). + t.Run("filtered with no token should return no results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := &structs.IntentionListRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Filter: bexprMatch, + }, + } + + var resp structs.IndexedIntentions + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Intention.List", req, &resp)) + require.Len(t, resp.Intentions, 0) + require.False(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be false") + }) + + // has access to everything + t.Run("filtered with initial management token should return 1 and ResultsFilteredByACLs equal to false", func(t *testing.T) { req := &structs.IntentionListRequest{ Datacenter: "dc1", QueryOptions: structs.QueryOptions{ Token: TestDefaultInitialManagementToken, - Filter: "DestinationName == foobar", + Filter: bexprMatch, }, } @@ -1705,6 +1727,54 @@ func TestIntentionList_acl(t *testing.T) { require.Len(t, resp.Intentions, 1) require.False(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be false") }) + + // ResultsFilteredByACLs should reflect user does not have access to read all intentions but has access to some. + t.Run("filtered with user token whose permissions match filter should return 1 and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := &structs.IntentionListRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token.SecretID, + Filter: bexprMatch, + }, + } + + var resp structs.IndexedIntentions + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Intention.List", req, &resp)) + require.Len(t, resp.Intentions, 1) + require.True(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") + }) + + // ResultsFilteredByACLs need to act as though no filter was applied. + t.Run("filtered with user token whose permissions do match filter should return 0 and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := &structs.IntentionListRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token.SecretID, + Filter: bexprNoMatch, + }, + } + + var resp structs.IndexedIntentions + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Intention.List", req, &resp)) + require.Len(t, resp.Intentions, 0) + require.True(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") + }) + + // ResultsFilteredByACLs should reflect user does not have access to read any intentions + t.Run("filtered with anonymous token should return 0 and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := &structs.IntentionListRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: "anonymous", + Filter: bexprMatch, + }, + } + + var resp structs.IndexedIntentions + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Intention.List", req, &resp)) + require.Len(t, resp.Intentions, 0) + require.True(t, resp.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") + }) } // Test basic matching. We don't need to exhaustively test inputs since this diff --git a/agent/consul/internal_endpoint.go b/agent/consul/internal_endpoint.go index af27842d20..6afb8405f3 100644 --- a/agent/consul/internal_endpoint.go +++ b/agent/consul/internal_endpoint.go @@ -7,15 +7,18 @@ import ( "fmt" "net" + hashstructure_v2 "github.com/mitchellh/hashstructure/v2" + "golang.org/x/exp/maps" + "github.com/hashicorp/go-bexpr" "github.com/hashicorp/go-hclog" "github.com/hashicorp/go-memdb" "github.com/hashicorp/serf/serf" - hashstructure_v2 "github.com/mitchellh/hashstructure/v2" "github.com/hashicorp/consul/acl" "github.com/hashicorp/consul/agent/consul/state" "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/lib/stringslice" ) const MaximumManualVIPsPerService = 8 @@ -117,6 +120,18 @@ func (m *Internal) NodeDump(args *structs.DCSpecificRequest, } reply.Index = maxIndex + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := m.srv.filterACL(args.Token, reply); err != nil { + return err + } + raw, err := filter.Execute(reply.Dump) if err != nil { return fmt.Errorf("could not filter local node dump: %w", err) @@ -129,13 +144,6 @@ func (m *Internal) NodeDump(args *structs.DCSpecificRequest, } reply.ImportedDump = importedRaw.(structs.NodeDump) - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := m.srv.filterACL(args.Token, reply); err != nil { - return err - } - return nil }) } @@ -235,13 +243,26 @@ func (m *Internal) ServiceDump(args *structs.ServiceDumpRequest, reply *structs. } } reply.Index = maxIndex - raw, err := filter.Execute(reply.Nodes) - if err != nil { - return fmt.Errorf("could not filter local service dump: %w", err) - } - reply.Nodes = raw.(structs.CheckServiceNodes) } + // Note: we filter the results with ACLs *before* applying the user-supplied + // bexpr filter to ensure that the user can only run expressions on data that + // they have access to. This is a security measure to prevent users from + // running arbitrary expressions on data they don't have access to. + // QueryMeta.ResultsFilteredByACLs being true already indicates to the user + // that results they don't have access to have been removed. If they were + // also allowed to run the bexpr filter on the data, they could potentially + // infer the specific attributes of data they don't have access to. + if err := m.srv.filterACL(args.Token, reply); err != nil { + return err + } + + raw, err := filter.Execute(reply.Nodes) + if err != nil { + return fmt.Errorf("could not filter local service dump: %w", err) + } + reply.Nodes = raw.(structs.CheckServiceNodes) + if !args.NodesOnly { importedRaw, err := filter.Execute(reply.ImportedNodes) if err != nil { @@ -249,12 +270,6 @@ func (m *Internal) ServiceDump(args *structs.ServiceDumpRequest, reply *structs. } reply.ImportedNodes = importedRaw.(structs.CheckServiceNodes) } - // Note: we filter the results with ACLs *after* applying the user-supplied - // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include - // results that would be filtered out even if the user did have permission. - if err := m.srv.filterACL(args.Token, reply); err != nil { - return err - } return nil }) @@ -770,17 +785,38 @@ func (m *Internal) AssignManualServiceVIPs(args *structs.AssignServiceManualVIPs return fmt.Errorf("cannot associate more than %d manual virtual IPs with the same service", MaximumManualVIPsPerService) } + vipMap := make(map[string]struct{}) for _, ip := range args.ManualVIPs { parsedIP := net.ParseIP(ip) if parsedIP == nil || parsedIP.To4() == nil { return fmt.Errorf("%q is not a valid IPv4 address", parsedIP.String()) } + vipMap[ip] = struct{}{} + } + // Silently ignore duplicates. + args.ManualVIPs = maps.Keys(vipMap) + + psn := structs.PeeredServiceName{ + ServiceName: structs.NewServiceName(args.Service, &args.EnterpriseMeta), + } + + // Check to see if we can skip the raft apply entirely. + { + existingIPs, err := m.srv.fsm.State().ServiceManualVIPs(psn) + if err != nil { + return fmt.Errorf("error checking for existing manual ips for service: %w", err) + } + if existingIPs != nil && stringslice.EqualMapKeys(existingIPs.ManualIPs, vipMap) { + *reply = structs.AssignServiceManualVIPsResponse{ + Found: true, + UnassignedFrom: nil, + } + return nil + } } req := state.ServiceVirtualIP{ - Service: structs.PeeredServiceName{ - ServiceName: structs.NewServiceName(args.Service, &args.EnterpriseMeta), - }, + Service: psn, ManualIPs: args.ManualVIPs, } resp, err := m.srv.raftApplyMsgpack(structs.UpdateVirtualIPRequestType, req) diff --git a/agent/consul/internal_endpoint_test.go b/agent/consul/internal_endpoint_test.go index e4b9a14b70..2a46c85e25 100644 --- a/agent/consul/internal_endpoint_test.go +++ b/agent/consul/internal_endpoint_test.go @@ -12,11 +12,11 @@ import ( "testing" "time" - "github.com/hashicorp/consul-net-rpc/net/rpc" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" msgpackrpc "github.com/hashicorp/consul-net-rpc/net-rpc-msgpackrpc" + "github.com/hashicorp/consul-net-rpc/net/rpc" "github.com/hashicorp/consul/acl" "github.com/hashicorp/consul/agent/structs" @@ -656,11 +656,73 @@ func TestInternal_NodeDump_FilterACL(t *testing.T) { t.Fatal("ResultsFilteredByACLs should be true") } - // We've already proven that we call the ACL filtering function so we - // test node filtering down in acl.go for node cases. This also proves - // that we respect the version 8 ACL flag, since the test server sets - // that to false (the regression value of *not* changing this is better - // for now until we change the sense of the version 8 ACL flag). + // need to ensure that ACLs are filtered prior to bexprFiltering + // Register additional node + regArgs := &structs.RegisterRequest{ + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.1", + WriteRequest: structs.WriteRequest{ + Token: "root", + }, + } + + var out struct{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.Register", regArgs, &out)) + + bexprMatchingUserTokenPermissions := fmt.Sprintf("Node matches `%s.*`", srv.config.NodeName) + const bexpNotMatchingUserTokenPermissions = "Node matches `node-deny.*`" + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodeDump{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.Dump)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodeDump{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.Dump)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would match only record without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: "", + Filter: bexprMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodeDump{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Empty(t, reply.Dump) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestInternal_EventFire_Token(t *testing.T) { @@ -1064,6 +1126,113 @@ func TestInternal_ServiceDump_ACL(t *testing.T) { require.Empty(t, out.Gateways) require.True(t, out.QueryMeta.ResultsFilteredByACLs, "ResultsFilteredByACLs should be true") }) + + // need to ensure that ACLs are filtered prior to bexprFiltering + // Register additional node + regArgs := &structs.RegisterRequest{ + Datacenter: "dc1", + Node: "node-deny", + ID: types.NodeID("e0155642-135d-4739-9853-b1ee6c9f945b"), + Address: "192.18.1.2", + Service: &structs.NodeService{ + Kind: structs.ServiceKindTypical, + ID: "memcached", + Service: "memcached", + Port: 5678, + }, + Check: &structs.HealthCheck{ + Name: "memcached check", + Status: api.HealthPassing, + ServiceID: "memcached", + }, + WriteRequest: structs.WriteRequest{ + Token: "root", + }, + } + + var out struct{} + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.Register", regArgs, &out)) + + const ( + bexprMatchingUserTokenPermissions = "Service.Service matches `redis.*`" + bexpNotMatchingUserTokenPermissions = "Node.Node matches `node-deny.*`" + ) + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + token := tokenWithRules(t, ` + node "node-deny" { + policy = "deny" + } + node "node1" { + policy = "read" + } + service "redis" { + policy = "read" + } + `) + var reply structs.IndexedNodesWithGateways + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexprMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodesWithGateways{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.ServiceDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Equal(t, 1, len(reply.Nodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + token := tokenWithRules(t, ` + node "node-deny" { + policy = "deny" + } + node "node1" { + policy = "read" + } + service "redis" { + policy = "read" + } + `) + var reply structs.IndexedNodesWithGateways + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: token, + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodesWithGateways{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.ServiceDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Zero(t, len(reply.Nodes)) + require.True(t, reply.ResultsFilteredByACLs) + }) + + t.Run("request with filter that would match only record without any token returns zero results and ResultsFilteredByACLs equal to false", func(t *testing.T) { + var reply structs.IndexedNodesWithGateways + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{ + Token: "", // no token + Filter: bexpNotMatchingUserTokenPermissions, + }, + } + + reply = structs.IndexedNodesWithGateways{} + if err := msgpackrpc.CallWithCodec(codec, "Internal.ServiceDump", &req, &reply); err != nil { + t.Fatalf("err: %s", err) + } + require.Empty(t, reply.Nodes) + require.False(t, reply.ResultsFilteredByACLs) + }) } func TestInternal_GatewayServiceDump_Terminating(t *testing.T) { @@ -3716,21 +3885,41 @@ func TestInternal_AssignManualServiceVIPs(t *testing.T) { require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.AssignManualServiceVIPs", req, &resp)) type testcase struct { - name string - req structs.AssignServiceManualVIPsRequest - expect structs.AssignServiceManualVIPsResponse - expectErr string + name string + req structs.AssignServiceManualVIPsRequest + expect structs.AssignServiceManualVIPsResponse + expectAgain structs.AssignServiceManualVIPsResponse + expectErr string + expectIPs []string } - run := func(t *testing.T, tc testcase) { - var resp structs.AssignServiceManualVIPsResponse - err := msgpackrpc.CallWithCodec(codec, "Internal.AssignManualServiceVIPs", tc.req, &resp) - if tc.expectErr != "" { - require.Error(t, err) - require.Contains(t, err.Error(), tc.expectErr) - return + + run := func(t *testing.T, tc testcase, again bool) { + if tc.expectErr != "" && again { + return // we don't retest known errors + } + + var resp structs.AssignServiceManualVIPsResponse + idx1 := s1.raft.CommitIndex() + err := msgpackrpc.CallWithCodec(codec, "Internal.AssignManualServiceVIPs", tc.req, &resp) + idx2 := s1.raft.CommitIndex() + if tc.expectErr != "" { + testutil.RequireErrorContains(t, err, tc.expectErr) + } else { + if again { + require.Equal(t, tc.expectAgain, resp) + require.Equal(t, idx1, idx2, "no raft operations occurred") + } else { + require.Equal(t, tc.expect, resp) + } + + psn := structs.PeeredServiceName{ServiceName: structs.NewServiceName(tc.req.Service, nil)} + got, err := s1.fsm.State().ServiceManualVIPs(psn) + require.NoError(t, err) + require.NotNil(t, got) + require.Equal(t, tc.expectIPs, got.ManualIPs) } - require.Equal(t, tc.expect, resp) } + tcs := []testcase{ { name: "successful manual ip assignment", @@ -3738,7 +3927,19 @@ func TestInternal_AssignManualServiceVIPs(t *testing.T) { Service: "web", ManualVIPs: []string{"1.1.1.1", "2.2.2.2"}, }, - expect: structs.AssignServiceManualVIPsResponse{Found: true}, + expectIPs: []string{"1.1.1.1", "2.2.2.2"}, + expect: structs.AssignServiceManualVIPsResponse{Found: true}, + expectAgain: structs.AssignServiceManualVIPsResponse{Found: true}, + }, + { + name: "successfully ignoring duplicates", + req: structs.AssignServiceManualVIPsRequest{ + Service: "web", + ManualVIPs: []string{"1.2.3.4", "5.6.7.8", "1.2.3.4", "5.6.7.8"}, + }, + expectIPs: []string{"1.2.3.4", "5.6.7.8"}, + expect: structs.AssignServiceManualVIPsResponse{Found: true}, + expectAgain: structs.AssignServiceManualVIPsResponse{Found: true}, }, { name: "reassign existing ip", @@ -3746,6 +3947,7 @@ func TestInternal_AssignManualServiceVIPs(t *testing.T) { Service: "web", ManualVIPs: []string{"8.8.8.8"}, }, + expectIPs: []string{"8.8.8.8"}, expect: structs.AssignServiceManualVIPsResponse{ Found: true, UnassignedFrom: []structs.PeeredServiceName{ @@ -3754,6 +3956,8 @@ func TestInternal_AssignManualServiceVIPs(t *testing.T) { }, }, }, + // When we repeat this operation the second time it's a no-op. + expectAgain: structs.AssignServiceManualVIPsResponse{Found: true}, }, { name: "invalid ip", @@ -3761,13 +3965,19 @@ func TestInternal_AssignManualServiceVIPs(t *testing.T) { Service: "web", ManualVIPs: []string{"3.3.3.3", "invalid"}, }, - expect: structs.AssignServiceManualVIPsResponse{}, expectErr: "not a valid", }, } for _, tc := range tcs { t.Run(tc.name, func(t *testing.T) { - run(t, tc) + t.Run("initial", func(t *testing.T) { + run(t, tc, false) + }) + if tc.expectErr == "" { + t.Run("repeat", func(t *testing.T) { + run(t, tc, true) // only repeat a write if it isn't an known error + }) + } }) } } diff --git a/agent/consul/state/catalog.go b/agent/consul/state/catalog.go index dcfe4ec91f..b8588f17cc 100644 --- a/agent/consul/state/catalog.go +++ b/agent/consul/state/catalog.go @@ -8,6 +8,8 @@ import ( "fmt" "net" "reflect" + "slices" + "sort" "strings" "github.com/hashicorp/go-memdb" @@ -18,6 +20,7 @@ import ( "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/lib" "github.com/hashicorp/consul/lib/maps" + "github.com/hashicorp/consul/lib/stringslice" "github.com/hashicorp/consul/types" ) @@ -1106,6 +1109,9 @@ func (s *Store) AssignManualServiceVIPs(idx uint64, psn structs.PeeredServiceNam for _, ip := range ips { assignedIPs[ip] = struct{}{} } + + txnNeedsCommit := false + modifiedEntries := make(map[structs.PeeredServiceName]struct{}) for ip := range assignedIPs { entry, err := tx.First(tableServiceVirtualIPs, indexManualVIPs, psn.ServiceName.PartitionOrDefault(), ip) @@ -1118,7 +1124,13 @@ func (s *Store) AssignManualServiceVIPs(idx uint64, psn structs.PeeredServiceNam } newEntry := entry.(ServiceVirtualIP) - if newEntry.Service.ServiceName.Matches(psn.ServiceName) { + + var ( + thisServiceName = newEntry.Service.ServiceName + thisPeer = newEntry.Service.Peer + ) + + if thisServiceName.Matches(psn.ServiceName) && thisPeer == psn.Peer { continue } @@ -1130,6 +1142,7 @@ func (s *Store) AssignManualServiceVIPs(idx uint64, psn structs.PeeredServiceNam filteredIPs = append(filteredIPs, existingIP) } } + sort.Strings(filteredIPs) newEntry.ManualIPs = filteredIPs newEntry.ModifyIndex = idx @@ -1137,6 +1150,12 @@ func (s *Store) AssignManualServiceVIPs(idx uint64, psn structs.PeeredServiceNam return false, nil, fmt.Errorf("failed inserting service virtual IP entry: %s", err) } modifiedEntries[newEntry.Service] = struct{}{} + + if err := updateVirtualIPMaxIndexes(tx, idx, thisServiceName.PartitionOrDefault(), thisPeer); err != nil { + return false, nil, err + } + + txnNeedsCommit = true } entry, err := tx.First(tableServiceVirtualIPs, indexID, psn) @@ -1149,23 +1168,37 @@ func (s *Store) AssignManualServiceVIPs(idx uint64, psn structs.PeeredServiceNam } newEntry := entry.(ServiceVirtualIP) - newEntry.ManualIPs = ips - newEntry.ModifyIndex = idx - if err := tx.Insert(tableServiceVirtualIPs, newEntry); err != nil { - return false, nil, fmt.Errorf("failed inserting service virtual IP entry: %s", err) + // Check to see if the slice already contains the same ips. + if !stringslice.EqualMapKeys(newEntry.ManualIPs, assignedIPs) { + newEntry.ManualIPs = slices.Clone(ips) + newEntry.ModifyIndex = idx + + sort.Strings(newEntry.ManualIPs) + + if err := tx.Insert(tableServiceVirtualIPs, newEntry); err != nil { + return false, nil, fmt.Errorf("failed inserting service virtual IP entry: %s", err) + } + if err := updateVirtualIPMaxIndexes(tx, idx, psn.ServiceName.PartitionOrDefault(), psn.Peer); err != nil { + return false, nil, err + } + txnNeedsCommit = true } - if err := updateVirtualIPMaxIndexes(tx, idx, psn.ServiceName.PartitionOrDefault(), psn.Peer); err != nil { - return false, nil, err - } - if err = tx.Commit(); err != nil { - return false, nil, err + + if txnNeedsCommit { + if err = tx.Commit(); err != nil { + return false, nil, err + } } return true, maps.SliceOfKeys(modifiedEntries), nil } func updateVirtualIPMaxIndexes(txn WriteTxn, idx uint64, partition, peerName string) error { + // update global max index (for snapshots) + if err := indexUpdateMaxTxn(txn, idx, tableServiceVirtualIPs); err != nil { + return fmt.Errorf("failed while updating index: %w", err) + } // update per-partition max index if err := indexUpdateMaxTxn(txn, idx, partitionedIndexEntryName(tableServiceVirtualIPs, partition)); err != nil { return fmt.Errorf("failed while updating partitioned index: %w", err) @@ -3086,6 +3119,7 @@ func servicesVirtualIPsTxn(tx ReadTxn, ws memdb.WatchSet) (uint64, []ServiceVirt vips = append(vips, vip) } + // Pull from the global one idx := maxIndexWatchTxn(tx, nil, tableServiceVirtualIPs) return idx, vips, nil diff --git a/agent/consul/state/catalog_test.go b/agent/consul/state/catalog_test.go index cef608bc1c..8445acf987 100644 --- a/agent/consul/state/catalog_test.go +++ b/agent/consul/state/catalog_test.go @@ -13,15 +13,15 @@ import ( "testing" "time" - "github.com/hashicorp/consul/acl" - "github.com/google/go-cmp/cmp" "github.com/google/go-cmp/cmp/cmpopts" - "github.com/hashicorp/go-memdb" - "github.com/hashicorp/go-uuid" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "github.com/hashicorp/go-memdb" + "github.com/hashicorp/go-uuid" + + "github.com/hashicorp/consul/acl" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/lib/stringslice" @@ -1963,81 +1963,289 @@ func TestStateStore_AssignManualVirtualIPs(t *testing.T) { s := testStateStore(t) setVirtualIPFlags(t, s) - // Attempt to assign manual virtual IPs to a service that doesn't exist - should be a no-op. - psn := structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "foo", EnterpriseMeta: *acl.DefaultEnterpriseMeta()}} - found, svcs, err := s.AssignManualServiceVIPs(0, psn, []string{"7.7.7.7", "8.8.8.8"}) - require.NoError(t, err) - require.False(t, found) - require.Empty(t, svcs) - serviceVIP, err := s.ServiceManualVIPs(psn) - require.NoError(t, err) - require.Nil(t, serviceVIP) - - // Create the service registration. - entMeta := structs.DefaultEnterpriseMetaInDefaultPartition() - ns1 := &structs.NodeService{ - ID: "foo", - Service: "foo", - Address: "1.1.1.1", - Port: 1111, - Connect: structs.ServiceConnect{Native: true}, - EnterpriseMeta: *entMeta, + newPSN := func(name, peer string) structs.PeeredServiceName { + return structs.PeeredServiceName{ + ServiceName: structs.ServiceName{ + Name: name, + EnterpriseMeta: *acl.DefaultEnterpriseMeta(), + }, + Peer: peer, + } } - // Service successfully registers into the state store. - testRegisterNode(t, s, 0, "node1") - require.NoError(t, s.EnsureService(1, "node1", ns1)) + checkMaxIndexes := func(t *testing.T, expect, expectImported uint64) { + t.Helper() + tx := s.db.Txn(false) + defer tx.Abort() - // Make sure there's a virtual IP for the foo service. - vip, err := s.VirtualIPForService(psn) - require.NoError(t, err) - assert.Equal(t, "240.0.0.1", vip) + idx := maxIndexWatchTxn(tx, nil, tableServiceVirtualIPs) + require.Equal(t, expect, idx) - // No manual IP should be set yet. - serviceVIP, err = s.ServiceManualVIPs(psn) - require.NoError(t, err) - require.Equal(t, "0.0.0.1", serviceVIP.IP.String()) - require.Empty(t, serviceVIP.ManualIPs) + entMeta := acl.DefaultEnterpriseMeta() - // Attempt to assign manual virtual IPs again. - found, svcs, err = s.AssignManualServiceVIPs(2, psn, []string{"7.7.7.7", "8.8.8.8"}) - require.NoError(t, err) - require.True(t, found) - require.Empty(t, svcs) - serviceVIP, err = s.ServiceManualVIPs(psn) - require.NoError(t, err) - require.Equal(t, "0.0.0.1", serviceVIP.IP.String()) - require.Equal(t, serviceVIP.ManualIPs, []string{"7.7.7.7", "8.8.8.8"}) + importedIdx := maxIndexTxn(tx, partitionedIndexEntryName(tableServiceVirtualIPs+".imported", entMeta.PartitionOrDefault())) + require.Equal(t, expectImported, importedIdx) + } - // Register another service via config entry. - s.EnsureConfigEntry(3, &structs.ServiceResolverConfigEntry{ - Kind: structs.ServiceResolver, - Name: "bar", + assignManual := func( + t *testing.T, + idx uint64, + psn structs.PeeredServiceName, + ips []string, + modified ...structs.PeeredServiceName, + ) { + t.Helper() + found, svcs, err := s.AssignManualServiceVIPs(idx, psn, ips) + require.NoError(t, err) + require.True(t, found) + if len(modified) == 0 { + require.Empty(t, svcs) + } else { + require.ElementsMatch(t, modified, svcs) + } + } + + checkVIP := func( + t *testing.T, + psn structs.PeeredServiceName, + expectVIP string, + ) { + t.Helper() + // Make sure there's a virtual IP for the foo service. + vip, err := s.VirtualIPForService(psn) + require.NoError(t, err) + assert.Equal(t, expectVIP, vip) + } + + checkManualVIP := func( + t *testing.T, + psn structs.PeeredServiceName, + expectIP string, + expectManual []string, + expectIndex uint64, + ) { + t.Helper() + serviceVIP, err := s.ServiceManualVIPs(psn) + require.NoError(t, err) + require.Equal(t, expectIP, serviceVIP.IP.String()) + if len(expectManual) == 0 { + require.Empty(t, serviceVIP.ManualIPs) + } else { + require.Equal(t, expectManual, serviceVIP.ManualIPs) + } + require.Equal(t, expectIndex, serviceVIP.ModifyIndex) + } + + psn := newPSN("foo", "") + + lastIndex := uint64(0) + nextIndex := func() uint64 { + lastIndex++ + return lastIndex + } + + testutil.RunStep(t, "assign to nonexistent service is noop", func(t *testing.T) { + useIdx := nextIndex() + + // Attempt to assign manual virtual IPs to a service that doesn't exist - should be a no-op. + found, svcs, err := s.AssignManualServiceVIPs(useIdx, psn, []string{"7.7.7.7", "8.8.8.8"}) + require.NoError(t, err) + require.False(t, found) + require.Empty(t, svcs) + + serviceVIP, err := s.ServiceManualVIPs(psn) + require.NoError(t, err) + require.Nil(t, serviceVIP) + + checkMaxIndexes(t, 0, 0) }) - psn2 := structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "bar"}} - vip, err = s.VirtualIPForService(psn2) - require.NoError(t, err) - assert.Equal(t, "240.0.0.2", vip) + // Create the service registration. + var regIndex1 uint64 + testutil.RunStep(t, "create service 1", func(t *testing.T) { + useIdx := nextIndex() + regIndex1 = useIdx + + entMeta := acl.DefaultEnterpriseMeta() + ns1 := &structs.NodeService{ + ID: "foo", + Service: "foo", + Address: "1.1.1.1", + Port: 1111, + Connect: structs.ServiceConnect{Native: true}, + EnterpriseMeta: *entMeta, + } + + // Service successfully registers into the state store. + testRegisterNode(t, s, useIdx, "node1") + require.NoError(t, s.EnsureService(useIdx, "node1", ns1)) + + // Make sure there's a virtual IP for the foo service. + checkVIP(t, psn, "240.0.0.1") + + // No manual IP should be set yet. + checkManualVIP(t, psn, "0.0.0.1", []string{}, regIndex1) + + checkMaxIndexes(t, regIndex1, 0) + }) + + // Attempt to assign manual virtual IPs again. + var assignIndex1 uint64 + testutil.RunStep(t, "assign to existent service does something", func(t *testing.T) { + useIdx := nextIndex() + assignIndex1 = useIdx + + // inserting in the wrong order to test the string sort + assignManual(t, useIdx, psn, []string{"7.7.7.7", "8.8.8.8", "6.6.6.6"}) + + checkManualVIP(t, psn, "0.0.0.1", []string{ + "6.6.6.6", "7.7.7.7", "8.8.8.8", + }, assignIndex1) + + checkMaxIndexes(t, assignIndex1, 0) + }) + + psn2 := newPSN("bar", "") + + var regIndex2 uint64 + testutil.RunStep(t, "create service 2", func(t *testing.T) { + useIdx := nextIndex() + regIndex2 = useIdx + + // Register another service via config entry. + s.EnsureConfigEntry(useIdx, &structs.ServiceResolverConfigEntry{ + Kind: structs.ServiceResolver, + Name: "bar", + }) + + checkVIP(t, psn2, "240.0.0.2") + + // No manual IP should be set yet. + checkManualVIP(t, psn2, "0.0.0.2", []string{}, regIndex2) + + checkMaxIndexes(t, regIndex2, 0) + }) // Attempt to assign manual virtual IPs for bar, with one IP overlapping with foo. // This should cause the ip to be removed from foo's list of manual IPs. - found, svcs, err = s.AssignManualServiceVIPs(4, psn2, []string{"7.7.7.7", "9.9.9.9"}) - require.NoError(t, err) - require.True(t, found) - require.ElementsMatch(t, svcs, []structs.PeeredServiceName{psn}) + var assignIndex2 uint64 + testutil.RunStep(t, "assign to existent service and ip is removed from another", func(t *testing.T) { + useIdx := nextIndex() + assignIndex2 = useIdx - serviceVIP, err = s.ServiceManualVIPs(psn) - require.NoError(t, err) - require.Equal(t, "0.0.0.1", serviceVIP.IP.String()) - require.Equal(t, []string{"8.8.8.8"}, serviceVIP.ManualIPs) - require.Equal(t, uint64(4), serviceVIP.ModifyIndex) + assignManual(t, useIdx, psn2, []string{"7.7.7.7", "9.9.9.9"}, psn) - serviceVIP, err = s.ServiceManualVIPs(psn2) - require.NoError(t, err) - require.Equal(t, "0.0.0.2", serviceVIP.IP.String()) - require.Equal(t, []string{"7.7.7.7", "9.9.9.9"}, serviceVIP.ManualIPs) - require.Equal(t, uint64(4), serviceVIP.ModifyIndex) + checkManualVIP(t, psn, "0.0.0.1", []string{ + "6.6.6.6", "8.8.8.8", // 7.7.7.7 was stolen by psn2 + }, assignIndex2) + checkManualVIP(t, psn2, "0.0.0.2", []string{ + "7.7.7.7", "9.9.9.9", + }, assignIndex2) + + checkMaxIndexes(t, assignIndex2, 0) + }) + + psn3 := newPSN("gir", "peer1") + + var regIndex3 uint64 + testutil.RunStep(t, "create peered service 1", func(t *testing.T) { + useIdx := nextIndex() + regIndex3 = useIdx + + // Create the service registration. + entMetaPeer := acl.DefaultEnterpriseMeta() + nsPeer1 := &structs.NodeService{ + ID: "gir", + Service: "gir", + Address: "9.9.9.9", + Port: 2222, + PeerName: "peer1", + Connect: structs.ServiceConnect{Native: true}, + EnterpriseMeta: *entMetaPeer, + } + + // Service successfully registers into the state store. + testRegisterPeering(t, s, useIdx, "peer1") + testRegisterNodeOpts(t, s, useIdx, "node9", func(n *structs.Node) error { + n.PeerName = "peer1" + return nil + }) + require.NoError(t, s.EnsureService(useIdx, "node9", nsPeer1)) + + checkVIP(t, psn3, "240.0.0.3") + + // No manual IP should be set yet. + checkManualVIP(t, psn3, "0.0.0.3", []string{}, regIndex3) + + checkMaxIndexes(t, regIndex3, regIndex3) + }) + + // Assign manual virtual IPs to peered service. + var assignIndex3 uint64 + testutil.RunStep(t, "assign to peered service and steal from non-peered", func(t *testing.T) { + useIdx := nextIndex() + assignIndex3 = useIdx + + // 5.5.5.5 is stolen from psn + assignManual(t, useIdx, psn3, []string{"5.5.5.5", "6.6.6.6"}, psn) + + checkManualVIP(t, psn, "0.0.0.1", []string{ + "8.8.8.8", // 5.5.5.5 was stolen by psn3 + }, assignIndex3) + checkManualVIP(t, psn2, "0.0.0.2", []string{ + "7.7.7.7", "9.9.9.9", + }, assignIndex2) + checkManualVIP(t, psn3, "0.0.0.3", []string{ + "5.5.5.5", "6.6.6.6", + }, assignIndex3) + + checkMaxIndexes(t, assignIndex3, assignIndex3) + }) + + var assignIndex4 uint64 + testutil.RunStep(t, "assign to non-peered service and steal from peered", func(t *testing.T) { + useIdx := nextIndex() + assignIndex4 = useIdx + + // 6.6.6.6 is stolen from psn3 + assignManual(t, useIdx, psn2, []string{ + "7.7.7.7", "9.9.9.9", "6.6.6.6", + }, psn3) + + checkManualVIP(t, psn, "0.0.0.1", []string{ + "8.8.8.8", // 5.5.5.5 was stolen by psn3 + }, assignIndex3) + checkManualVIP(t, psn2, "0.0.0.2", []string{ + "6.6.6.6", "7.7.7.7", "9.9.9.9", + }, assignIndex4) + checkManualVIP(t, psn3, "0.0.0.3", []string{ + "5.5.5.5", + }, assignIndex4) + + checkMaxIndexes(t, assignIndex4, assignIndex4) + }) + + testutil.RunStep(t, "repeat the last write and no indexes should be bumped", func(t *testing.T) { + useIdx := nextIndex() + + assignManual(t, useIdx, psn2, []string{ + "7.7.7.7", "9.9.9.9", "6.6.6.6", + }) // no modified this time + + // no changes + checkManualVIP(t, psn, "0.0.0.1", []string{ + "8.8.8.8", + }, assignIndex3) + checkManualVIP(t, psn2, "0.0.0.2", []string{ + "6.6.6.6", "7.7.7.7", "9.9.9.9", + }, assignIndex4) + checkManualVIP(t, psn3, "0.0.0.3", []string{ + "5.5.5.5", + }, assignIndex4) + + // no change + checkMaxIndexes(t, assignIndex4, assignIndex4) + }) } func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { diff --git a/agent/dns_test.go b/agent/dns_test.go index c5a8c1db2c..56c549cb6b 100644 --- a/agent/dns_test.go +++ b/agent/dns_test.go @@ -2367,7 +2367,7 @@ func TestDNS_trimUDPResponse_NoTrim(t *testing.T) { }, } - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) if trimmed := trimUDPResponse(req, resp, cfg.DNSUDPAnswerLimit); trimmed { t.Fatalf("Bad %#v", *resp) } @@ -2400,7 +2400,7 @@ func TestDNS_trimUDPResponse_NoTrim(t *testing.T) { } func TestDNS_trimUDPResponse_TrimLimit(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) req, resp, expected := &dns.Msg{}, &dns.Msg{}, &dns.Msg{} for i := 0; i < cfg.DNSUDPAnswerLimit+1; i++ { @@ -2439,7 +2439,7 @@ func TestDNS_trimUDPResponse_TrimLimit(t *testing.T) { } func TestDNS_trimUDPResponse_TrimLimitWithNS(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) req, resp, expected := &dns.Msg{}, &dns.Msg{}, &dns.Msg{} for i := 0; i < cfg.DNSUDPAnswerLimit+1; i++ { @@ -2486,7 +2486,7 @@ func TestDNS_trimUDPResponse_TrimLimitWithNS(t *testing.T) { } func TestDNS_trimTCPResponse_TrimLimitWithNS(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) req, resp, expected := &dns.Msg{}, &dns.Msg{}, &dns.Msg{} for i := 0; i < 5000; i++ { @@ -2542,7 +2542,7 @@ func loadRuntimeConfig(t *testing.T, hcl string) *config.RuntimeConfig { } func TestDNS_trimUDPResponse_TrimSize(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) req, resp := &dns.Msg{}, &dns.Msg{} for i := 0; i < 100; i++ { @@ -2594,7 +2594,7 @@ func TestDNS_trimUDPResponse_TrimSize(t *testing.T) { } func TestDNS_trimUDPResponse_TrimSizeEDNS(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) req, resp := &dns.Msg{}, &dns.Msg{} @@ -2672,7 +2672,7 @@ func TestDNS_trimUDPResponse_TrimSizeEDNS(t *testing.T) { } func TestDNS_trimUDPResponse_TrimSizeMaxSize(t *testing.T) { - cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" node_name = "dummy" `) + cfg := loadRuntimeConfig(t, `node_name = "test" data_dir = "a" bind_addr = "127.0.0.1" `) resp := &dns.Msg{} diff --git a/agent/http.go b/agent/http.go index 506377074a..eb6e186cd8 100644 --- a/agent/http.go +++ b/agent/http.go @@ -6,7 +6,6 @@ package agent import ( "encoding/json" "fmt" - "github.com/hashicorp/go-hclog" "io" "net" "net/http" @@ -20,6 +19,8 @@ import ( "sync/atomic" "time" + "github.com/hashicorp/go-hclog" + "github.com/NYTimes/gziphandler" "github.com/armon/go-metrics" "github.com/armon/go-metrics/prometheus" @@ -348,16 +349,24 @@ func withRemoteAddrHandler(next http.Handler) http.Handler { }) } -// Injects content type explicitly if not already set into response to prevent XSS +// ensureContentTypeHeader injects content-type explicitly if not already set into response to prevent XSS func ensureContentTypeHeader(next http.Handler, logger hclog.Logger) http.Handler { - return http.HandlerFunc(func(resp http.ResponseWriter, req *http.Request) { next.ServeHTTP(resp, req) - val := resp.Header().Get(contentTypeHeader) - if val == "" { - resp.Header().Set(contentTypeHeader, plainContentType) - logger.Debug("warning: content-type header not explicitly set.", "request-path", req.URL) + contentType := api.GetContentType(req) + + if req != nil { + logger.Debug("warning: request content-type is not supported", "request-path", req.URL) + req.Header.Set(contentTypeHeader, contentType) + } + + if resp != nil { + respContentType := resp.Header().Get(contentTypeHeader) + if respContentType == "" || respContentType != contentType { + logger.Debug("warning: response content-type header not explicitly set.", "request-path", req.URL) + resp.Header().Set(contentTypeHeader, contentType) + } } }) } diff --git a/agent/http_test.go b/agent/http_test.go index 497789f689..73a599546a 100644 --- a/agent/http_test.go +++ b/agent/http_test.go @@ -617,7 +617,6 @@ func TestHTTPAPI_DefaultACLPolicy(t *testing.T) { }) } } - func TestHTTPAPIResponseHeaders(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") @@ -646,6 +645,87 @@ func TestHTTPAPIResponseHeaders(t *testing.T) { requireHasHeadersSet(t, a, "/", "text/plain; charset=utf-8") } +func TestHTTPAPIValidateContentTypeHeaders(t *testing.T) { + if testing.Short() { + t.Skip("too slow for testing.Short") + } + + t.Parallel() + type testcase struct { + name string + endpoint string + method string + requestBody io.Reader + expectedContentType string + } + + cases := []testcase{ + { + name: "snapshot endpoint expect non-default content type", + method: http.MethodPut, + endpoint: "/v1/snapshot", + requestBody: bytes.NewBuffer([]byte("test")), + expectedContentType: "application/octet-stream", + }, + { + name: "kv endpoint expect non-default content type", + method: http.MethodPut, + endpoint: "/v1/kv", + requestBody: bytes.NewBuffer([]byte("test")), + expectedContentType: "application/octet-stream", + }, + { + name: "event/fire endpoint expect default content type", + method: http.MethodPut, + endpoint: "/v1/event/fire", + requestBody: bytes.NewBuffer([]byte("test")), + expectedContentType: "application/octet-stream", + }, + { + name: "peering/token endpoint expect default content type", + method: http.MethodPost, + endpoint: "/v1/peering/token", + requestBody: bytes.NewBuffer([]byte("test")), + expectedContentType: "application/json", + }, + } + + for _, tc := range cases { + tc := tc + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + + a := NewTestAgent(t, "") + defer a.Shutdown() + + requireContentTypeHeadersSet(t, a, tc.method, tc.endpoint, tc.requestBody, tc.expectedContentType) + }) + } +} + +func requireContentTypeHeadersSet(t *testing.T, a *TestAgent, method, path string, body io.Reader, contentType string) { + t.Helper() + + resp := httptest.NewRecorder() + req, _ := http.NewRequest(method, path, body) + a.enableDebug.Store(true) + + a.srv.handler().ServeHTTP(resp, req) + + reqHdrs := req.Header + respHdrs := resp.Header() + + // require request content-type + require.NotEmpty(t, reqHdrs.Get("Content-Type")) + require.Equal(t, contentType, reqHdrs.Get("Content-Type"), + "Request Header Content-Type value incorrect") + + // require response content-type + require.NotEmpty(t, respHdrs.Get("Content-Type")) + require.Equal(t, contentType, respHdrs.Get("Content-Type"), + "Response Header Content-Type value incorrect") +} + func requireHasHeadersSet(t *testing.T, a *TestAgent, path string, contentType string) { t.Helper() @@ -663,7 +743,7 @@ func requireHasHeadersSet(t *testing.T, a *TestAgent, path string, contentType s "X-XSS-Protection header value incorrect") require.Equal(t, contentType, hdrs.Get("Content-Type"), - "") + "Response Content-Type header value incorrect") } func TestUIResponseHeaders(t *testing.T) { @@ -704,7 +784,7 @@ func TestErrorContentTypeHeaderSet(t *testing.T) { `) defer a.Shutdown() - requireHasHeadersSet(t, a, "/fake-path-doesn't-exist", "text/plain; charset=utf-8") + requireHasHeadersSet(t, a, "/fake-path-doesn't-exist", "application/json") } func TestAcceptEncodingGzip(t *testing.T) { diff --git a/agent/proxycfg-glue/internal_service_dump.go b/agent/proxycfg-glue/internal_service_dump.go index d1c701083d..dd8293f781 100644 --- a/agent/proxycfg-glue/internal_service_dump.go +++ b/agent/proxycfg-glue/internal_service_dump.go @@ -81,19 +81,21 @@ func (s *serverInternalServiceDump) Notify(ctx context.Context, req *structs.Ser return 0, nil, err } + totalNodeLength := len(nodes) + aclfilter.New(authz, s.deps.Logger).Filter(&nodes) + raw, err := filter.Execute(nodes) if err != nil { return 0, nil, fmt.Errorf("could not filter local service dump: %w", err) } nodes = raw.(structs.CheckServiceNodes) - aclfilter.New(authz, s.deps.Logger).Filter(&nodes) - return idx, &structs.IndexedCheckServiceNodes{ Nodes: nodes, QueryMeta: structs.QueryMeta{ - Index: idx, - Backend: structs.QueryBackendBlocking, + Index: idx, + Backend: structs.QueryBackendBlocking, + ResultsFilteredByACLs: totalNodeLength != len(nodes), }, }, nil }, diff --git a/agent/proxycfg-glue/internal_service_dump_test.go b/agent/proxycfg-glue/internal_service_dump_test.go index 1eba4c0438..e3d65ac6ae 100644 --- a/agent/proxycfg-glue/internal_service_dump_test.go +++ b/agent/proxycfg-glue/internal_service_dump_test.go @@ -55,6 +55,10 @@ func TestServerInternalServiceDump(t *testing.T) { Service: "web", Kind: structs.ServiceKindTypical, }, + { + Service: "web-deny", + Kind: structs.ServiceKindTypical, + }, { Service: "db", Kind: structs.ServiceKindTypical, @@ -67,14 +71,14 @@ func TestServerInternalServiceDump(t *testing.T) { })) } - authz := newStaticResolver( - policyAuthorizer(t, ` + policyAuth := policyAuthorizer(t, ` service "mgw" { policy = "read" } service "web" { policy = "read" } + service "web-deny" { policy = "deny" } service "db" { policy = "read" } node_prefix "node-" { policy = "read" } - `), - ) + `) + authz := newStaticResolver(policyAuth) dataSource := ServerInternalServiceDump(ServerDataSourceDeps{ GetStore: func() Store { return store }, @@ -121,6 +125,42 @@ func TestServerInternalServiceDump(t *testing.T) { result := getEventResult[*structs.IndexedCheckServiceNodes](t, eventCh) require.Empty(t, result.Nodes) }) + + const ( + bexprMatchingUserTokenPermissions = "Service.Service matches `web.*`" + bexpNotMatchingUserTokenPermissions = "Service.Service matches `mgw.*`" + ) + + authz.SwapAuthorizer(policyAuthorizer(t, ` + service "mgw" { policy = "deny" } + service "web" { policy = "read" } + service "web-deny" { policy = "deny" } + service "db" { policy = "read" } + node_prefix "node-" { policy = "read" } + `)) + + t.Run("request with filter that matches token permissions returns 1 result and ResultsFilteredByACLs equal to true", func(t *testing.T) { + eventCh := make(chan proxycfg.UpdateEvent) + require.NoError(t, dataSource.Notify(ctx, &structs.ServiceDumpRequest{ + QueryOptions: structs.QueryOptions{Filter: bexprMatchingUserTokenPermissions}, + }, "", eventCh)) + + result := getEventResult[*structs.IndexedCheckServiceNodes](t, eventCh) + require.Len(t, result.Nodes, 1) + require.Equal(t, "web", result.Nodes[0].Service.Service) + require.True(t, result.ResultsFilteredByACLs) + }) + + t.Run("request with filter that does not match token permissions returns 0 results and ResultsFilteredByACLs equal to true", func(t *testing.T) { + eventCh := make(chan proxycfg.UpdateEvent) + require.NoError(t, dataSource.Notify(ctx, &structs.ServiceDumpRequest{ + QueryOptions: structs.QueryOptions{Filter: bexpNotMatchingUserTokenPermissions}, + }, "", eventCh)) + + result := getEventResult[*structs.IndexedCheckServiceNodes](t, eventCh) + require.Len(t, result.Nodes, 0) + require.True(t, result.ResultsFilteredByACLs) + }) }) } diff --git a/agent/proxycfg/api_gateway.go b/agent/proxycfg/api_gateway.go index eb5d246462..0c44e5bfd4 100644 --- a/agent/proxycfg/api_gateway.go +++ b/agent/proxycfg/api_gateway.go @@ -476,16 +476,12 @@ func (h *handlerAPIGateway) handleRouteConfigUpdate(ctx context.Context, u Updat cancelUpstream() delete(snap.APIGateway.WatchedUpstreams[upstreamID], targetID) delete(snap.APIGateway.WatchedUpstreamEndpoints[upstreamID], targetID) - - if targetUID := NewUpstreamIDFromTargetID(targetID); targetUID.Peer != "" { - snap.APIGateway.PeerUpstreamEndpoints.CancelWatch(targetUID) - snap.APIGateway.UpstreamPeerTrustBundles.CancelWatch(targetUID.Peer) - } } cancelDiscoChain() delete(snap.APIGateway.WatchedDiscoveryChains, upstreamID) } + reconcilePeeringWatches(snap.APIGateway.DiscoveryChain, snap.APIGateway.UpstreamConfig, snap.APIGateway.PeeredUpstreams, snap.APIGateway.PeerUpstreamEndpoints, snap.APIGateway.UpstreamPeerTrustBundles) return nil } diff --git a/agent/proxycfg/connect_proxy.go b/agent/proxycfg/connect_proxy.go index 0a8c173792..35c0462cb3 100644 --- a/agent/proxycfg/connect_proxy.go +++ b/agent/proxycfg/connect_proxy.go @@ -380,49 +380,7 @@ func (s *handlerConnectProxy) handleUpdate(ctx context.Context, u UpdateEvent, s // // Clean up data // - - peeredChainTargets := make(map[UpstreamID]struct{}) - for _, discoChain := range snap.ConnectProxy.DiscoveryChain { - for _, target := range discoChain.Targets { - if target.Peer == "" { - continue - } - uid := NewUpstreamIDFromTargetID(target.ID) - peeredChainTargets[uid] = struct{}{} - } - } - - validPeerNames := make(map[string]struct{}) - - // Iterate through all known endpoints and remove references to upstream IDs that weren't in the update - snap.ConnectProxy.PeerUpstreamEndpoints.ForEachKey(func(uid UpstreamID) bool { - // Peered upstream is explicitly defined in upstream config - if _, ok := snap.ConnectProxy.UpstreamConfig[uid]; ok { - validPeerNames[uid.Peer] = struct{}{} - return true - } - // Peered upstream came from dynamic source of imported services - if _, ok := seenUpstreams[uid]; ok { - validPeerNames[uid.Peer] = struct{}{} - return true - } - // Peered upstream came from a discovery chain target - if _, ok := peeredChainTargets[uid]; ok { - validPeerNames[uid.Peer] = struct{}{} - return true - } - snap.ConnectProxy.PeerUpstreamEndpoints.CancelWatch(uid) - return true - }) - - // Iterate through all known trust bundles and remove references to any unseen peer names - snap.ConnectProxy.UpstreamPeerTrustBundles.ForEachKey(func(peerName PeerName) bool { - if _, ok := validPeerNames[peerName]; !ok { - snap.ConnectProxy.UpstreamPeerTrustBundles.CancelWatch(peerName) - } - return true - }) - + reconcilePeeringWatches(snap.ConnectProxy.DiscoveryChain, snap.ConnectProxy.UpstreamConfig, snap.ConnectProxy.PeeredUpstreams, snap.ConnectProxy.PeerUpstreamEndpoints, snap.ConnectProxy.UpstreamPeerTrustBundles) case u.CorrelationID == intentionUpstreamsID: resp, ok := u.Result.(*structs.IndexedServiceList) if !ok { @@ -490,18 +448,13 @@ func (s *handlerConnectProxy) handleUpdate(ctx context.Context, u UpdateEvent, s continue } if _, ok := seenUpstreams[uid]; !ok { - for targetID, cancelFn := range targets { + for _, cancelFn := range targets { cancelFn() - - targetUID := NewUpstreamIDFromTargetID(targetID) - if targetUID.Peer != "" { - snap.ConnectProxy.PeerUpstreamEndpoints.CancelWatch(targetUID) - snap.ConnectProxy.UpstreamPeerTrustBundles.CancelWatch(targetUID.Peer) - } } delete(snap.ConnectProxy.WatchedUpstreams, uid) } } + reconcilePeeringWatches(snap.ConnectProxy.DiscoveryChain, snap.ConnectProxy.UpstreamConfig, snap.ConnectProxy.PeeredUpstreams, snap.ConnectProxy.PeerUpstreamEndpoints, snap.ConnectProxy.UpstreamPeerTrustBundles) for uid := range snap.ConnectProxy.WatchedUpstreamEndpoints { if upstream, ok := snap.ConnectProxy.UpstreamConfig[uid]; ok && !upstream.CentrallyConfigured { continue diff --git a/agent/proxycfg/ingress_gateway.go b/agent/proxycfg/ingress_gateway.go index 3ab5828add..0262ffcb37 100644 --- a/agent/proxycfg/ingress_gateway.go +++ b/agent/proxycfg/ingress_gateway.go @@ -171,18 +171,13 @@ func (s *handlerIngressGateway) handleUpdate(ctx context.Context, u UpdateEvent, delete(snap.IngressGateway.WatchedUpstreams[uid], targetID) delete(snap.IngressGateway.WatchedUpstreamEndpoints[uid], targetID) cancelUpstreamFn() - - targetUID := NewUpstreamIDFromTargetID(targetID) - if targetUID.Peer != "" { - snap.IngressGateway.PeerUpstreamEndpoints.CancelWatch(targetUID) - snap.IngressGateway.UpstreamPeerTrustBundles.CancelWatch(targetUID.Peer) - } } cancelFn() delete(snap.IngressGateway.WatchedDiscoveryChains, uid) } } + reconcilePeeringWatches(snap.IngressGateway.DiscoveryChain, snap.IngressGateway.UpstreamConfig, snap.IngressGateway.PeeredUpstreams, snap.IngressGateway.PeerUpstreamEndpoints, snap.IngressGateway.UpstreamPeerTrustBundles) if err := s.watchIngressLeafCert(ctx, snap); err != nil { return err diff --git a/agent/proxycfg/state.go b/agent/proxycfg/state.go index b6b9c78f32..d0ae44fbab 100644 --- a/agent/proxycfg/state.go +++ b/agent/proxycfg/state.go @@ -13,12 +13,15 @@ import ( "sync/atomic" "time" - "github.com/hashicorp/go-hclog" "golang.org/x/time/rate" + "github.com/hashicorp/go-hclog" + cachetype "github.com/hashicorp/consul/agent/cache-types" + "github.com/hashicorp/consul/agent/proxycfg/internal/watch" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/logging" + "github.com/hashicorp/consul/proto/private/pbpeering" ) const ( @@ -551,3 +554,48 @@ func watchMeshGateway(ctx context.Context, opts gatewayWatchOpts) error { EnterpriseMeta: *structs.DefaultEnterpriseMetaInPartition(opts.key.Partition), }, correlationId, opts.notifyCh) } + +func reconcilePeeringWatches(compiledDiscoveryChains map[UpstreamID]*structs.CompiledDiscoveryChain, upstreams map[UpstreamID]*structs.Upstream, peeredUpstreams map[UpstreamID]struct{}, peerUpstreamEndpoints watch.Map[UpstreamID, structs.CheckServiceNodes], upstreamPeerTrustBundles watch.Map[PeerName, *pbpeering.PeeringTrustBundle]) { + + peeredChainTargets := make(map[UpstreamID]struct{}) + for _, discoChain := range compiledDiscoveryChains { + for _, target := range discoChain.Targets { + if target.Peer == "" { + continue + } + uid := NewUpstreamIDFromTargetID(target.ID) + peeredChainTargets[uid] = struct{}{} + } + } + + validPeerNames := make(map[string]struct{}) + + // Iterate through all known endpoints and remove references to upstream IDs that weren't in the update + peerUpstreamEndpoints.ForEachKey(func(uid UpstreamID) bool { + // Peered upstream is explicitly defined in upstream config + if _, ok := upstreams[uid]; ok { + validPeerNames[uid.Peer] = struct{}{} + return true + } + // Peered upstream came from dynamic source of imported services + if _, ok := peeredUpstreams[uid]; ok { + validPeerNames[uid.Peer] = struct{}{} + return true + } + // Peered upstream came from a discovery chain target + if _, ok := peeredChainTargets[uid]; ok { + validPeerNames[uid.Peer] = struct{}{} + return true + } + peerUpstreamEndpoints.CancelWatch(uid) + return true + }) + + // Iterate through all known trust bundles and remove references to any unseen peer names + upstreamPeerTrustBundles.ForEachKey(func(peerName PeerName) bool { + if _, ok := validPeerNames[peerName]; !ok { + upstreamPeerTrustBundles.CancelWatch(peerName) + } + return true + }) +} diff --git a/agent/proxycfg/upstreams.go b/agent/proxycfg/upstreams.go index 209a3446d9..052e91eb10 100644 --- a/agent/proxycfg/upstreams.go +++ b/agent/proxycfg/upstreams.go @@ -102,6 +102,7 @@ func (s *handlerUpstreams) handleUpdateUpstreams(ctx context.Context, u UpdateEv if err := s.resetWatchesFromChain(ctx, uid, resp.Chain, upstreamsSnapshot); err != nil { return err } + reconcilePeeringWatches(upstreamsSnapshot.DiscoveryChain, upstreamsSnapshot.UpstreamConfig, upstreamsSnapshot.PeeredUpstreams, upstreamsSnapshot.PeerUpstreamEndpoints, upstreamsSnapshot.UpstreamPeerTrustBundles) case strings.HasPrefix(u.CorrelationID, upstreamPeerWatchIDPrefix): resp, ok := u.Result.(*structs.IndexedCheckServiceNodes) @@ -301,12 +302,6 @@ func (s *handlerUpstreams) resetWatchesFromChain( delete(snap.WatchedUpstreams[uid], targetID) delete(snap.WatchedUpstreamEndpoints[uid], targetID) cancelFn() - - targetUID := NewUpstreamIDFromTargetID(targetID) - if targetUID.Peer != "" { - snap.PeerUpstreamEndpoints.CancelWatch(targetUID) - snap.UpstreamPeerTrustBundles.CancelWatch(targetUID.Peer) - } } var ( @@ -479,8 +474,8 @@ func (s *handlerUpstreams) watchUpstreamTarget(ctx context.Context, snap *Config var entMeta acl.EnterpriseMeta entMeta.Merge(opts.entMeta) - ctx, cancel := context.WithCancel(ctx) - err := s.dataSources.Health.Notify(ctx, &structs.ServiceSpecificRequest{ + peerCtx, cancel := context.WithCancel(ctx) + err := s.dataSources.Health.Notify(peerCtx, &structs.ServiceSpecificRequest{ PeerName: opts.peer, Datacenter: opts.datacenter, QueryOptions: structs.QueryOptions{ @@ -506,25 +501,25 @@ func (s *handlerUpstreams) watchUpstreamTarget(ctx context.Context, snap *Config return nil } - if ok := snap.PeerUpstreamEndpoints.IsWatched(uid); !ok { + if !snap.PeerUpstreamEndpoints.IsWatched(uid) { snap.PeerUpstreamEndpoints.InitWatch(uid, cancel) } - // Check whether a watch for this peer exists to avoid duplicates. - if ok := snap.UpstreamPeerTrustBundles.IsWatched(uid.Peer); !ok { - peerCtx, cancel := context.WithCancel(ctx) - if err := s.dataSources.TrustBundle.Notify(peerCtx, &cachetype.TrustBundleReadRequest{ + + if !snap.UpstreamPeerTrustBundles.IsWatched(uid.Peer) { + peerCtx2, cancel2 := context.WithCancel(ctx) + if err := s.dataSources.TrustBundle.Notify(peerCtx2, &cachetype.TrustBundleReadRequest{ Request: &pbpeering.TrustBundleReadRequest{ Name: uid.Peer, Partition: uid.PartitionOrDefault(), }, QueryOptions: structs.QueryOptions{Token: s.token}, }, peerTrustBundleIDPrefix+uid.Peer, s.ch); err != nil { - cancel() + cancel2() return fmt.Errorf("error while watching trust bundle for peer %q: %w", uid.Peer, err) } - snap.UpstreamPeerTrustBundles.InitWatch(uid.Peer, cancel) + snap.UpstreamPeerTrustBundles.InitWatch(uid.Peer, cancel2) } return nil diff --git a/agent/routine-leak-checker/leak_test.go b/agent/routine-leak-checker/leak_test.go index f6b3c2a749..53e1e1ea42 100644 --- a/agent/routine-leak-checker/leak_test.go +++ b/agent/routine-leak-checker/leak_test.go @@ -64,9 +64,7 @@ func setupPrimaryServer(t *testing.T) *agent.TestAgent { config := ` server = true - datacenter = "primary" - primary_datacenter = "primary" - + datacenter = "primary" connect { enabled = true } diff --git a/agent/structs/acl.go b/agent/structs/acl.go index 579e8d231e..e4ced5e6c1 100644 --- a/agent/structs/acl.go +++ b/agent/structs/acl.go @@ -800,6 +800,11 @@ func (policies ACLPolicies) resolveWithCache(cache *ACLCaches, entConf *acl.Conf continue } + //pulling from the cache, we don't want to break any rules that are already in the cache + if entConf == nil { + entConf = &acl.Config{} + } + entConf.WarnOnDuplicateKey = true p, err := acl.NewPolicyFromSource(policy.Rules, entConf, policy.EnterprisePolicyMeta()) if err != nil { return nil, fmt.Errorf("failed to parse %q: %v", policy.Name, err) diff --git a/agent/structs/acl_test.go b/agent/structs/acl_test.go index e1fb35263b..0e6878e612 100644 --- a/agent/structs/acl_test.go +++ b/agent/structs/acl_test.go @@ -403,7 +403,7 @@ func TestStructs_ACLPolicies_resolveWithCache(t *testing.T) { ID: "5d5653a1-2c2b-4b36-b083-fc9f1398eb7b", Name: "policy1", Description: "policy1", - Rules: `node_prefix "" { policy = "read" }`, + Rules: `node_prefix "" { policy = "read", policy = "read", },`, RaftIndex: RaftIndex{ CreateIndex: 1, ModifyIndex: 2, @@ -413,7 +413,7 @@ func TestStructs_ACLPolicies_resolveWithCache(t *testing.T) { ID: "b35541f0-a88a-48da-bc66-43553c60b628", Name: "policy2", Description: "policy2", - Rules: `agent_prefix "" { policy = "read" }`, + Rules: `agent_prefix "" { policy = "read" } `, RaftIndex: RaftIndex{ CreateIndex: 3, ModifyIndex: 4, @@ -433,7 +433,8 @@ func TestStructs_ACLPolicies_resolveWithCache(t *testing.T) { ID: "8bf38965-95e5-4e86-9be7-f6070cc0708b", Name: "policy4", Description: "policy4", - Rules: `service_prefix "" { policy = "read" }`, + //test should still pass even with the duplicate key since its resolving the cache + Rules: `service_prefix "" { policy = "read" policy = "read" }`, RaftIndex: RaftIndex{ CreateIndex: 7, ModifyIndex: 8, diff --git a/api/api.go b/api/api.go index d4d853d5d4..27af1ea569 100644 --- a/api/api.go +++ b/api/api.go @@ -1087,8 +1087,23 @@ func (c *Client) doRequest(r *request) (time.Duration, *http.Response, error) { if err != nil { return 0, nil, err } + + contentType := GetContentType(req) + + if req != nil { + req.Header.Set(contentTypeHeader, contentType) + } + start := time.Now() resp, err := c.config.HttpClient.Do(req) + + if resp != nil { + respContentType := resp.Header.Get(contentTypeHeader) + if respContentType == "" || respContentType != contentType { + resp.Header.Set(contentTypeHeader, contentType) + } + } + diff := time.Since(start) return diff, resp, err } diff --git a/api/api_test.go b/api/api_test.go index e8a03f7218..9a3ed7374c 100644 --- a/api/api_test.go +++ b/api/api_test.go @@ -935,11 +935,11 @@ func TestAPI_Headers(t *testing.T) { _, _, err = kv.Get("test-headers", nil) require.NoError(t, err) - require.Equal(t, "", request.Header.Get("Content-Type")) + require.Equal(t, "application/json", request.Header.Get("Content-Type")) _, err = kv.Delete("test-headers", nil) require.NoError(t, err) - require.Equal(t, "", request.Header.Get("Content-Type")) + require.Equal(t, "application/json", request.Header.Get("Content-Type")) err = c.Snapshot().Restore(nil, strings.NewReader("foo")) require.Error(t, err) diff --git a/api/content_type.go b/api/content_type.go new file mode 100644 index 0000000000..37c8cf60aa --- /dev/null +++ b/api/content_type.go @@ -0,0 +1,81 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package api + +import ( + "net/http" + "strings" +) + +const ( + contentTypeHeader = "Content-Type" + plainContentType = "text/plain; charset=utf-8" + octetStream = "application/octet-stream" + jsonContentType = "application/json" // Default content type +) + +// ContentTypeRule defines a rule for determining the content type of an HTTP request. +// This rule is based on the combination of the HTTP path, method, and the desired content type. +type ContentTypeRule struct { + path string + httpMethod string + contentType string +} + +var ContentTypeRules = []ContentTypeRule{ + { + path: "/v1/snapshot", + httpMethod: http.MethodPut, + contentType: octetStream, + }, + { + path: "/v1/kv", + httpMethod: http.MethodPut, + contentType: octetStream, + }, + { + path: "/v1/event/fire", + httpMethod: http.MethodPut, + contentType: octetStream, + }, +} + +// GetContentType returns the content type for a request +// This function isused as routing logic or middleware to determine and enforce +// the appropriate content type for HTTP requests. +func GetContentType(req *http.Request) string { + reqContentType := req.Header.Get(contentTypeHeader) + + if isIndexPage(req) { + return plainContentType + } + + // For GET, DELETE, or internal API paths, ensure a valid Content-Type is returned. + if req.Method == http.MethodGet || req.Method == http.MethodDelete || strings.HasPrefix(req.URL.Path, "/v1/internal") { + if reqContentType == "" { + // Default to JSON Content-Type if no Content-Type is provided. + return jsonContentType + } + // Return the provided Content-Type if it exists. + return reqContentType + } + + for _, rule := range ContentTypeRules { + if matchesRule(req, rule) { + return rule.contentType + } + } + return jsonContentType +} + +// matchesRule checks if a request matches a content type rule +func matchesRule(req *http.Request, rule ContentTypeRule) bool { + return strings.HasPrefix(req.URL.Path, rule.path) && + (rule.httpMethod == "" || req.Method == rule.httpMethod) +} + +// isIndexPage checks if the request is for the index page +func isIndexPage(req *http.Request) bool { + return req.URL.Path == "/" || req.URL.Path == "/ui" +} diff --git a/envoyextensions/xdscommon/ENVOY_VERSIONS b/envoyextensions/xdscommon/ENVOY_VERSIONS index 884f305732..02f8186aea 100644 --- a/envoyextensions/xdscommon/ENVOY_VERSIONS +++ b/envoyextensions/xdscommon/ENVOY_VERSIONS @@ -8,7 +8,7 @@ # # See https://www.consul.io/docs/connect/proxies/envoy#supported-versions for more information on Consul's Envoy # version support. -1.31.2 -1.30.6 -1.29.9 +1.31.5 +1.30.9 +1.29.12 1.28.7 diff --git a/go.mod b/go.mod index 2cdc8bbaab..13d0e5b75e 100644 --- a/go.mod +++ b/go.mod @@ -35,7 +35,7 @@ require ( github.com/go-jose/go-jose/v3 v3.0.3 github.com/go-openapi/runtime v0.26.2 github.com/go-openapi/strfmt v0.21.10 - github.com/google/go-cmp v0.5.9 + github.com/google/go-cmp v0.6.0 github.com/google/gofuzz v1.2.0 github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 github.com/google/tcpproxy v0.0.0-20180808230851-dfa16c61dad2 @@ -68,7 +68,7 @@ require ( github.com/hashicorp/go-version v1.2.1 github.com/hashicorp/golang-lru v0.5.4 github.com/hashicorp/hcdiag v0.5.1 - github.com/hashicorp/hcl v1.0.0 + github.com/hashicorp/hcl v1.0.1-vault-7 github.com/hashicorp/hcl/v2 v2.14.1 github.com/hashicorp/hcp-scada-provider v0.2.4 github.com/hashicorp/hcp-sdk-go v0.80.0 @@ -114,12 +114,12 @@ require ( go.opentelemetry.io/otel/sdk/metric v0.39.0 go.opentelemetry.io/proto/otlp v1.0.0 go.uber.org/goleak v1.1.10 - golang.org/x/crypto v0.22.0 + golang.org/x/crypto v0.31.0 golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 - golang.org/x/net v0.24.0 + golang.org/x/net v0.33.0 golang.org/x/oauth2 v0.15.0 - golang.org/x/sync v0.4.0 - golang.org/x/sys v0.20.0 + golang.org/x/sync v0.10.0 + golang.org/x/sys v0.28.0 golang.org/x/time v0.3.0 google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 google.golang.org/grpc v1.58.3 @@ -186,7 +186,7 @@ require ( github.com/go-openapi/validate v0.22.4 // indirect github.com/go-ozzo/ozzo-validation v3.6.0+incompatible // indirect github.com/gogo/protobuf v1.3.2 // indirect - github.com/golang-jwt/jwt/v4 v4.5.0 // indirect + github.com/golang-jwt/jwt/v4 v4.5.1 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.4 // indirect github.com/golang/snappy v0.0.4 // indirect @@ -263,10 +263,10 @@ require ( go.opencensus.io v0.24.0 // indirect go.opentelemetry.io/otel/trace v1.17.0 // indirect golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 // indirect - golang.org/x/mod v0.13.0 // indirect - golang.org/x/term v0.19.0 // indirect - golang.org/x/text v0.14.0 // indirect - golang.org/x/tools v0.14.0 // indirect + golang.org/x/mod v0.17.0 // indirect + golang.org/x/term v0.27.0 // indirect + golang.org/x/text v0.21.0 // indirect + golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect google.golang.org/api v0.126.0 // indirect google.golang.org/appengine v1.6.8 // indirect google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98 // indirect diff --git a/go.sum b/go.sum index fd2246257f..d1cde1a67c 100644 --- a/go.sum +++ b/go.sum @@ -287,8 +287,8 @@ github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69 github.com/goji/httpauth v0.0.0-20160601135302-2da839ab0f4d/go.mod h1:nnjvkQ9ptGaCkuDUx6wNykzzlUixGxvkme+H/lnzb+A= github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= github.com/golang-jwt/jwt/v4 v4.2.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= -github.com/golang-jwt/jwt/v4 v4.5.0 h1:7cYmW1XlMY7h7ii7UhUyChSgS5wUJEnm9uZVTGqOWzg= -github.com/golang-jwt/jwt/v4 v4.5.0/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0= +github.com/golang-jwt/jwt/v4 v4.5.1 h1:JdqV9zKUdtaa9gdPlywC3aeoEsR681PlKC+4F5gQgeo= +github.com/golang-jwt/jwt/v4 v4.5.1/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/glog v1.1.0 h1:/d3pCKDPWNnvIWe0vVUpNP32qc8U3PDVxySP/y360qE= github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP3NQ= @@ -348,8 +348,9 @@ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk= github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= @@ -488,8 +489,9 @@ github.com/hashicorp/golang-lru/v2 v2.0.0 h1:Lf+9eD8m5pncvHAOCQj49GSN6aQI8XGfI5O github.com/hashicorp/golang-lru/v2 v2.0.0/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/hashicorp/hcdiag v0.5.1 h1:KZcx9xzRfEOQ2OMbwPxVvHyXwLLRqYpSHxCEOtHfQ6w= github.com/hashicorp/hcdiag v0.5.1/go.mod h1:RMC2KkffN9uJ+5mFSaL67ZFVj4CDeetPF2d/53XpwXo= -github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/hcl v1.0.1-vault-7 h1:ag5OxFVy3QYTFTJODRzTKVZ6xvdfLLCA1cy/Y6xGI0I= +github.com/hashicorp/hcl v1.0.1-vault-7/go.mod h1:XYhtn6ijBSAj6n4YqAaf7RBPS4I06AItNorpy+MoQNM= github.com/hashicorp/hcl/v2 v2.14.1 h1:x0BpjfZ+CYdbiz+8yZTQ+gdLO7IXvOut7Da+XJayx34= github.com/hashicorp/hcl/v2 v2.14.1/go.mod h1:e4z5nxYlWNPdDSNYX+ph14EvWYMFm3eP0zIUqPc2jr0= github.com/hashicorp/hcp-scada-provider v0.2.4 h1:XvctVEd4VqWVlqN1VA4vIhJANstZrc4gd2oCfrFLWZc= @@ -919,8 +921,8 @@ golang.org/x/crypto v0.0.0-20220314234659-1baeb1ce4c0b/go.mod h1:IxCIyHEi3zRg3s0 golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU= -golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30= -golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -961,8 +963,8 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= -golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY= -golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= +golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/net v0.0.0-20180530234432-1e491301e022/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180611182652-db08ff08e862/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -1013,8 +1015,8 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= -golang.org/x/net v0.24.0 h1:1PcaxkF854Fu3+lvBIx5SYn9wRlBzzcnHZSiaFFAb0w= -golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8= +golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= +golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -1045,8 +1047,8 @@ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= -golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= +golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -1126,15 +1128,15 @@ golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= -golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo= golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk= -golang.org/x/term v0.19.0 h1:+ThwsDv+tYfnJFhF4L8jITxu1tdTWRTZpdsWgEgjL6Q= -golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk= +golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q= +golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -1147,8 +1149,9 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= -golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -1214,8 +1217,8 @@ golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= -golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc= -golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/internal/resource/filter.go b/internal/resource/filter.go deleted file mode 100644 index 09e251e218..0000000000 --- a/internal/resource/filter.go +++ /dev/null @@ -1,109 +0,0 @@ -// Copyright (c) HashiCorp, Inc. -// SPDX-License-Identifier: BUSL-1.1 - -package resource - -import ( - "fmt" - - "github.com/hashicorp/go-bexpr" - - "github.com/hashicorp/consul/proto-public/pbresource" -) - -type MetadataFilterableResources interface { - GetMetadata() map[string]string -} - -// FilterResourcesByMetadata will use the provided go-bexpr based filter to -// retain matching items from the provided slice. -// -// The only variables usable in the expressions are the metadata keys prefixed -// by "metadata." -// -// If no filter is provided, then this does nothing and returns the input. -func FilterResourcesByMetadata[T MetadataFilterableResources](resources []T, filter string) ([]T, error) { - if filter == "" || len(resources) == 0 { - return resources, nil - } - - eval, err := createMetadataFilterEvaluator(filter) - if err != nil { - return nil, err - } - - filtered := make([]T, 0, len(resources)) - for _, res := range resources { - vars := &metadataFilterFieldDetails{ - Meta: res.GetMetadata(), - } - match, err := eval.Evaluate(vars) - if err != nil { - return nil, err - } - if match { - filtered = append(filtered, res) - } - } - if len(filtered) == 0 { - return nil, nil - } - return filtered, nil -} - -// FilterMatchesResourceMetadata will use the provided go-bexpr based filter to -// determine if the provided resource matches. -// -// The only variables usable in the expressions are the metadata keys prefixed -// by "metadata." -// -// If no filter is provided, then this returns true. -func FilterMatchesResourceMetadata(res *pbresource.Resource, filter string) (bool, error) { - if res == nil { - return false, nil - } else if filter == "" { - return true, nil - } - - eval, err := createMetadataFilterEvaluator(filter) - if err != nil { - return false, err - } - - vars := &metadataFilterFieldDetails{ - Meta: res.Metadata, - } - match, err := eval.Evaluate(vars) - if err != nil { - return false, err - } - return match, nil -} - -// ValidateMetadataFilter will validate that the provided filter is going to be -// a valid input to the FilterResourcesByMetadata function. -// -// This is best called from a Validate hook. -func ValidateMetadataFilter(filter string) error { - if filter == "" { - return nil - } - - _, err := createMetadataFilterEvaluator(filter) - return err -} - -func createMetadataFilterEvaluator(filter string) (*bexpr.Evaluator, error) { - sampleVars := &metadataFilterFieldDetails{ - Meta: make(map[string]string), - } - eval, err := bexpr.CreateEvaluatorForType(filter, nil, sampleVars) - if err != nil { - return nil, fmt.Errorf("filter %q is invalid: %w", filter, err) - } - return eval, nil -} - -type metadataFilterFieldDetails struct { - Meta map[string]string `bexpr:"metadata"` -} diff --git a/internal/resource/filter_test.go b/internal/resource/filter_test.go deleted file mode 100644 index e15ec08030..0000000000 --- a/internal/resource/filter_test.go +++ /dev/null @@ -1,195 +0,0 @@ -// Copyright (c) HashiCorp, Inc. -// SPDX-License-Identifier: BUSL-1.1 - -package resource - -import ( - "testing" - - "github.com/stretchr/testify/require" - - "github.com/hashicorp/consul/proto-public/pbresource" - "github.com/hashicorp/consul/proto/private/prototest" - "github.com/hashicorp/consul/sdk/testutil" -) - -func TestFilterResourcesByMetadata(t *testing.T) { - type testcase struct { - in []*pbresource.Resource - filter string - expect []*pbresource.Resource - expectErr string - } - - create := func(name string, kvs ...string) *pbresource.Resource { - require.True(t, len(kvs)%2 == 0) - - meta := make(map[string]string) - for i := 0; i < len(kvs); i += 2 { - meta[kvs[i]] = kvs[i+1] - } - - return &pbresource.Resource{ - Id: &pbresource.ID{ - Name: name, - }, - Metadata: meta, - } - } - - run := func(t *testing.T, tc testcase) { - got, err := FilterResourcesByMetadata(tc.in, tc.filter) - if tc.expectErr != "" { - require.Error(t, err) - testutil.RequireErrorContains(t, err, tc.expectErr) - } else { - require.NoError(t, err) - prototest.AssertDeepEqual(t, tc.expect, got) - } - } - - cases := map[string]testcase{ - "nil input": {}, - "no filter": { - in: []*pbresource.Resource{ - create("one"), - create("two"), - create("three"), - create("four"), - }, - filter: "", - expect: []*pbresource.Resource{ - create("one"), - create("two"), - create("three"), - create("four"), - }, - }, - "bad filter": { - in: []*pbresource.Resource{ - create("one"), - create("two"), - create("three"), - create("four"), - }, - filter: "garbage.value == zzz", - expectErr: `Selector "garbage" is not valid`, - }, - "filter everything out": { - in: []*pbresource.Resource{ - create("one"), - create("two"), - create("three"), - create("four"), - }, - filter: "metadata.foo == bar", - }, - "filter simply": { - in: []*pbresource.Resource{ - create("one", "foo", "bar"), - create("two", "foo", "baz"), - create("three", "zim", "gir"), - create("four", "zim", "gaz", "foo", "bar"), - }, - filter: "metadata.foo == bar", - expect: []*pbresource.Resource{ - create("one", "foo", "bar"), - create("four", "zim", "gaz", "foo", "bar"), - }, - }, - "filter prefix": { - in: []*pbresource.Resource{ - create("one", "foo", "bar"), - create("two", "foo", "baz"), - create("three", "zim", "gir"), - create("four", "zim", "gaz", "foo", "bar"), - create("four", "zim", "zzz"), - }, - filter: "(zim in metadata) and (metadata.zim matches `^g.`)", - expect: []*pbresource.Resource{ - create("three", "zim", "gir"), - create("four", "zim", "gaz", "foo", "bar"), - }, - }, - } - - for name, tc := range cases { - t.Run(name, func(t *testing.T) { - run(t, tc) - }) - } -} - -func TestFilterMatchesResourceMetadata(t *testing.T) { - type testcase struct { - res *pbresource.Resource - filter string - expect bool - expectErr string - } - - create := func(name string, kvs ...string) *pbresource.Resource { - require.True(t, len(kvs)%2 == 0) - - meta := make(map[string]string) - for i := 0; i < len(kvs); i += 2 { - meta[kvs[i]] = kvs[i+1] - } - - return &pbresource.Resource{ - Id: &pbresource.ID{ - Name: name, - }, - Metadata: meta, - } - } - - run := func(t *testing.T, tc testcase) { - got, err := FilterMatchesResourceMetadata(tc.res, tc.filter) - if tc.expectErr != "" { - require.Error(t, err) - testutil.RequireErrorContains(t, err, tc.expectErr) - } else { - require.NoError(t, err) - require.Equal(t, tc.expect, got) - } - } - - cases := map[string]testcase{ - "nil input": {}, - "no filter": { - res: create("one"), - filter: "", - expect: true, - }, - "bad filter": { - res: create("one"), - filter: "garbage.value == zzz", - expectErr: `Selector "garbage" is not valid`, - }, - "no match": { - res: create("one"), - filter: "metadata.foo == bar", - }, - "match simply": { - res: create("one", "foo", "bar"), - filter: "metadata.foo == bar", - expect: true, - }, - "match via prefix": { - res: create("four", "zim", "gaz", "foo", "bar"), - filter: "(zim in metadata) and (metadata.zim matches `^g.`)", - expect: true, - }, - "no match via prefix": { - res: create("four", "zim", "zzz", "foo", "bar"), - filter: "(zim in metadata) and (metadata.zim matches `^g.`)", - }, - } - - for name, tc := range cases { - t.Run(name, func(t *testing.T) { - run(t, tc) - }) - } -} diff --git a/lib/stringslice/stringslice.go b/lib/stringslice/stringslice.go index 7c32864b94..71e90dba2a 100644 --- a/lib/stringslice/stringslice.go +++ b/lib/stringslice/stringslice.go @@ -80,3 +80,17 @@ func CloneStringSlice(s []string) []string { copy(out, s) return out } + +// EqualMapKeys returns true if the slice equals the keys of +// the map ignoring any ordering. +func EqualMapKeys[V any](a []string, b map[string]V) bool { + if len(a) != len(b) { + return false + } + for _, ip := range a { + if _, ok := b[ip]; !ok { + return false + } + } + return true +} diff --git a/lib/stringslice/stringslice_test.go b/lib/stringslice/stringslice_test.go index dd25071757..b861d9c38a 100644 --- a/lib/stringslice/stringslice_test.go +++ b/lib/stringslice/stringslice_test.go @@ -63,3 +63,28 @@ func TestMergeSorted(t *testing.T) { }) } } + +func TestEqualMapKeys(t *testing.T) { + for _, tc := range []struct { + a []string + b map[string]int + same bool + }{ + // same + {nil, nil, true}, + {[]string{}, nil, true}, + {nil, map[string]int{}, true}, + {[]string{}, map[string]int{}, true}, + {[]string{"a"}, map[string]int{"a": 1}, true}, + {[]string{"b", "a"}, map[string]int{"a": 1, "b": 1}, true}, + // different + {[]string{"a"}, map[string]int{}, false}, + {[]string{}, map[string]int{"a": 1}, false}, + {[]string{"b", "a"}, map[string]int{"c": 1, "a": 1, "b": 1}, false}, + {[]string{"b", "a"}, map[string]int{"c": 1, "a": 1, "b": 1}, false}, + {[]string{"b", "a", "c"}, map[string]int{"a": 1, "b": 1}, false}, + } { + got := EqualMapKeys(tc.a, tc.b) + require.Equal(t, tc.same, got) + } +} diff --git a/test-integ/go.mod b/test-integ/go.mod index d6293c4744..c7b2e9907f 100644 --- a/test-integ/go.mod +++ b/test-integ/go.mod @@ -5,7 +5,7 @@ go 1.22 toolchain go1.22.5 require ( - github.com/google/go-cmp v0.5.9 + github.com/google/go-cmp v0.6.0 github.com/hashicorp/consul/api v1.29.4 github.com/hashicorp/consul/proto-public v0.6.2 github.com/hashicorp/consul/sdk v0.16.1 @@ -16,7 +16,7 @@ require ( github.com/mitchellh/copystructure v1.2.0 github.com/rboyer/blankspace v0.2.1 github.com/stretchr/testify v1.8.4 - golang.org/x/net v0.24.0 + golang.org/x/net v0.33.0 google.golang.org/grpc v1.58.3 ) @@ -64,7 +64,7 @@ require ( github.com/hashicorp/go-uuid v1.0.3 // indirect github.com/hashicorp/go-version v1.2.1 // indirect github.com/hashicorp/golang-lru v0.5.4 // indirect - github.com/hashicorp/hcl v1.0.0 // indirect + github.com/hashicorp/hcl v1.0.1-vault-7 // indirect github.com/hashicorp/hcl/v2 v2.16.2 // indirect github.com/hashicorp/memberlist v0.5.0 // indirect github.com/hashicorp/serf v0.10.1 // indirect @@ -99,12 +99,13 @@ require ( github.com/teris-io/shortid v0.0.0-20220617161101-71ec9f2aa569 // indirect github.com/testcontainers/testcontainers-go v0.22.0 // indirect github.com/zclconf/go-cty v1.12.1 // indirect - golang.org/x/crypto v0.22.0 // indirect + golang.org/x/crypto v0.31.0 // indirect golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 // indirect - golang.org/x/mod v0.13.0 // indirect - golang.org/x/sys v0.20.0 // indirect - golang.org/x/text v0.14.0 // indirect - golang.org/x/tools v0.14.0 // indirect + golang.org/x/mod v0.17.0 // indirect + golang.org/x/sync v0.10.0 // indirect + golang.org/x/sys v0.28.0 // indirect + golang.org/x/text v0.21.0 // indirect + golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230803162519-f966b187b2e5 // indirect google.golang.org/protobuf v1.33.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect diff --git a/test-integ/go.sum b/test-integ/go.sum index 4e4c0e2d5e..95ba9c3d77 100644 --- a/test-integ/go.sum +++ b/test-integ/go.sum @@ -98,8 +98,9 @@ github.com/google/btree v1.0.1 h1:gK4Kx5IaGY9CD5sPJ36FHiBJ6ZXl0kilRiiCj+jdYp4= github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4= github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= @@ -141,8 +142,8 @@ github.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= -github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= -github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/hcl v1.0.1-vault-7 h1:ag5OxFVy3QYTFTJODRzTKVZ6xvdfLLCA1cy/Y6xGI0I= +github.com/hashicorp/hcl v1.0.1-vault-7/go.mod h1:XYhtn6ijBSAj6n4YqAaf7RBPS4I06AItNorpy+MoQNM= github.com/hashicorp/hcl/v2 v2.16.2 h1:mpkHZh/Tv+xet3sy3F9Ld4FyI2tUpWe9x3XtPx9f1a0= github.com/hashicorp/hcl/v2 v2.16.2/go.mod h1:JRmR89jycNkrrqnMmvPDMd56n1rQJ2Q6KocSLCMCXng= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= @@ -310,8 +311,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU= -golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30= -golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 h1:m64FZMko/V45gv0bNmrNYoDEq8U5YUhetc9cBWKS1TQ= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63/go.mod h1:0v4NqG35kSWCMzLaMeX+IQrlSnVE/bqGSyC2cz/9Le8= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= @@ -319,8 +320,8 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= -golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY= -golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= +golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= @@ -336,8 +337,8 @@ golang.org/x/net v0.0.0-20210726213435-c6fcb2dbf985/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= -golang.org/x/net v0.24.0 h1:1PcaxkF854Fu3+lvBIx5SYn9wRlBzzcnHZSiaFFAb0w= -golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8= +golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= +golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -346,8 +347,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= -golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= +golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -381,8 +382,8 @@ golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= -golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= @@ -395,8 +396,9 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= -golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -408,8 +410,8 @@ golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4f golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= -golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc= -golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/test/integration/consul-container/go.mod b/test/integration/consul-container/go.mod index aaebeb412a..20f5a0d092 100644 --- a/test/integration/consul-container/go.mod +++ b/test/integration/consul-container/go.mod @@ -20,7 +20,7 @@ require ( github.com/hashicorp/go-multierror v1.1.1 github.com/hashicorp/go-uuid v1.0.3 github.com/hashicorp/go-version v1.2.1 - github.com/hashicorp/hcl v1.0.0 + github.com/hashicorp/hcl v1.0.1-vault-7 github.com/hashicorp/serf v0.10.1 github.com/itchyny/gojq v0.12.12 github.com/mitchellh/copystructure v1.2.0 @@ -30,7 +30,7 @@ require ( github.com/stretchr/testify v1.8.4 github.com/teris-io/shortid v0.0.0-20220617161101-71ec9f2aa569 github.com/testcontainers/testcontainers-go v0.22.0 - golang.org/x/mod v0.13.0 + golang.org/x/mod v0.17.0 google.golang.org/grpc v1.58.3 ) @@ -58,7 +58,7 @@ require ( github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/protobuf v1.5.4 // indirect github.com/google/btree v1.0.1 // indirect - github.com/google/go-cmp v0.5.9 // indirect + github.com/google/go-cmp v0.6.0 // indirect github.com/google/uuid v1.4.0 // indirect github.com/hashicorp/consul-server-connection-manager v0.1.4 // indirect github.com/hashicorp/consul/proto-public v0.6.2 // indirect @@ -94,12 +94,13 @@ require ( github.com/prometheus/procfs v0.8.0 // indirect github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect github.com/sirupsen/logrus v1.9.3 // indirect - golang.org/x/crypto v0.22.0 // indirect + golang.org/x/crypto v0.31.0 // indirect golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 // indirect - golang.org/x/net v0.24.0 // indirect - golang.org/x/sys v0.20.0 // indirect - golang.org/x/text v0.14.0 // indirect - golang.org/x/tools v0.14.0 // indirect + golang.org/x/net v0.33.0 // indirect + golang.org/x/sync v0.10.0 // indirect + golang.org/x/sys v0.28.0 // indirect + golang.org/x/text v0.21.0 // indirect + golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230803162519-f966b187b2e5 // indirect google.golang.org/protobuf v1.33.0 // indirect diff --git a/test/integration/consul-container/go.sum b/test/integration/consul-container/go.sum index 2a7b3b8015..b82fe758f9 100644 --- a/test/integration/consul-container/go.sum +++ b/test/integration/consul-container/go.sum @@ -107,8 +107,9 @@ github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4= github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= @@ -150,8 +151,8 @@ github.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= -github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= -github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/hcl v1.0.1-vault-7 h1:ag5OxFVy3QYTFTJODRzTKVZ6xvdfLLCA1cy/Y6xGI0I= +github.com/hashicorp/hcl v1.0.1-vault-7/go.mod h1:XYhtn6ijBSAj6n4YqAaf7RBPS4I06AItNorpy+MoQNM= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/YAJqrc= github.com/hashicorp/memberlist v0.5.0 h1:EtYPN8DpAURiapus508I4n9CzHs2W+8NZGbmmR/prTM= @@ -306,8 +307,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU= -golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30= -golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 h1:m64FZMko/V45gv0bNmrNYoDEq8U5YUhetc9cBWKS1TQ= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63/go.mod h1:0v4NqG35kSWCMzLaMeX+IQrlSnVE/bqGSyC2cz/9Le8= @@ -319,8 +320,8 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= -golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY= -golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= +golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -339,8 +340,8 @@ golang.org/x/net v0.0.0-20210726213435-c6fcb2dbf985/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= -golang.org/x/net v0.24.0 h1:1PcaxkF854Fu3+lvBIx5SYn9wRlBzzcnHZSiaFFAb0w= -golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8= +golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I= +golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -351,8 +352,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= -golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= +golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -387,8 +388,8 @@ golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y= -golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= @@ -401,8 +402,9 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= -golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -418,8 +420,8 @@ golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4f golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= -golang.org/x/tools v0.14.0 h1:jvNa2pY0M4r62jkRQ6RwEZZyPcymeL9XZMLBbV7U2nc= -golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/testing/deployer/go.mod b/testing/deployer/go.mod index e1bd6711a3..e6b26ffe19 100644 --- a/testing/deployer/go.mod +++ b/testing/deployer/go.mod @@ -18,7 +18,7 @@ require ( github.com/mitchellh/copystructure v1.2.0 github.com/rboyer/safeio v0.2.2 github.com/stretchr/testify v1.8.4 - golang.org/x/crypto v0.22.0 + golang.org/x/crypto v0.31.0 golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 google.golang.org/grpc v1.56.3 google.golang.org/protobuf v1.33.0 @@ -55,8 +55,8 @@ require ( github.com/prometheus/procfs v0.6.0 // indirect github.com/zclconf/go-cty v1.12.1 // indirect golang.org/x/net v0.24.0 // indirect - golang.org/x/sys v0.19.0 // indirect - golang.org/x/text v0.14.0 // indirect + golang.org/x/sys v0.28.0 // indirect + golang.org/x/text v0.21.0 // indirect google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/testing/deployer/go.sum b/testing/deployer/go.sum index db7d88bffd..111a704c57 100644 --- a/testing/deployer/go.sum +++ b/testing/deployer/go.sum @@ -226,8 +226,8 @@ golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnf golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30= -golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= +golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= +golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 h1:m64FZMko/V45gv0bNmrNYoDEq8U5YUhetc9cBWKS1TQ= golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63/go.mod h1:0v4NqG35kSWCMzLaMeX+IQrlSnVE/bqGSyC2cz/9Le8= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -277,15 +277,15 @@ golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o= -golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA= +golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= -golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo= +golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= diff --git a/website/content/docs/connect/gateways/api-gateway/configuration/routes.mdx b/website/content/docs/connect/gateways/api-gateway/configuration/routes.mdx index 6723931290..9e2e8265ee 100644 --- a/website/content/docs/connect/gateways/api-gateway/configuration/routes.mdx +++ b/website/content/docs/connect/gateways/api-gateway/configuration/routes.mdx @@ -101,7 +101,7 @@ This field specifies backend services that the `Route` references. The following | Parameter | Description | Type | Required | | --- | --- | --- | --- | -| `group` | Specifies the Kubernetes API Group of the referenced backend. You can specify the following values:
  • `""`: Specifies the core Kubernetes API group. This value must be used when `kind` is set to `Service`. This is the default value if unspecified.
  • `api-gateway.consul.hashicorp.com`: This value must be used when `kind` is set to `MeshService`.
| String | Optional | +| `group` | Specifies the Kubernetes API Group of the referenced backend. You can specify the following values:
  • `""`: Specifies the core Kubernetes API group. This value must be used when `kind` is set to `Service`. This is the default value if unspecified.
  • `consul.hashicorp.com`: This value must be used when `kind` is set to `MeshService`.
| String | Optional | | `kind` | Specifies the Kubernetes Kind of the referenced backend. You can specify the following values:
  • `Service` (default): Indicates that the `backendRef` references a Service in the Kubernetes cluster.
  • `MeshService`: Indicates that the `backendRef` references a service in the Consul mesh. Refer to the `MeshService` [documentation](/consul/docs/connect/gateways/api-gateway/configuration/meshservice) for additional information.
| String | Optional | | `name` | Specifies the name of the Kubernetes Service or Consul mesh service resource. | String | Required | | `namespace` | Specifies the Kubernetes namespace containing the Kubernetes Service or Consul mesh service resource. You must specify a value if the Service or Consul mesh service is defined in a different namespace from the `Route`. Defaults to the namespace of the `Route`.
To create a route for a `backendRef` in a different namespace, you must also create a [ReferenceGrant](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant). Refer to the [example route](#example-cross-namespace-backendref) configured to reference across namespaces. | String | Optional | diff --git a/website/content/docs/connect/gateways/api-gateway/define-routes/route-to-peered-services.mdx b/website/content/docs/connect/gateways/api-gateway/define-routes/route-to-peered-services.mdx index 414ce45f53..e323f8ea9e 100644 --- a/website/content/docs/connect/gateways/api-gateway/define-routes/route-to-peered-services.mdx +++ b/website/content/docs/connect/gateways/api-gateway/define-routes/route-to-peered-services.mdx @@ -67,7 +67,7 @@ spec: ... rules: - backendRefs: - - group: api-gateway.consul.hashicorp.com + - group: consul.hashicorp.com kind: MeshService name: example-mesh-service port: 3000 diff --git a/website/content/docs/dynamic-app-config/sessions/index.mdx b/website/content/docs/dynamic-app-config/sessions/index.mdx index 3eb26f558f..5bbe65e8f1 100644 --- a/website/content/docs/dynamic-app-config/sessions/index.mdx +++ b/website/content/docs/dynamic-app-config/sessions/index.mdx @@ -10,7 +10,7 @@ description: >- Consul provides a session mechanism which can be used to build distributed locks. Sessions act as a binding layer between nodes, health checks, and key/value data. They are designed to provide granular locking and are heavily inspired by -[The Chubby Lock Service for Loosely-Coupled Distributed Systems](http://research.google.com/archive/chubby.html). +[The Chubby Lock Service for Loosely-Coupled Distributed Systems](https://research.google/pubs/the-chubby-lock-service-for-loosely-coupled-distributed-systems/). ## Session Design