consul/agent
hc-github-team-consul-core f7d80b7bb3
Backport of Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter into release/1.16.x (#18153)
## Backport

This PR is auto-generated from #18062 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.

This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.

- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
  protocol = "http"
}
```
- Create two providers 
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"

JSONWebKeySet = {
    Local = {
     JWKS = "eyJrZXlzIjog....."
    }
}
```

```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"

JSONWebKeySet = {
   Local = {
     JWKS = "eyJrZXlzIjogW3...."
    }
}
```

- add a service intention
```
Kind = "service-intentions"
Name = "redis"

JWT = {
  Providers = [
    {
      Name = "okta"
    },
  ]
}

Sources = [
  {
    Name = "*"
    Permissions = [{
      Action = "allow"
      HTTP = {
        PathPrefix = "/workspace"
      }
      JWT = {
        Providers = [
          {
            Name = "okta"
            VerifyClaims = [
              {
                  Path = ["aud"]
                  Value = "my_client_app"
              },
              {
                Path = ["sub"]
                Value = "5be86359073c434bad2da3932222dabe"
              }
            ]
          },
        ]
      }

    },
    {
      Action = "allow"
      HTTP = {
        PathPrefix = "/"
      }
      JWT = {
        Providers = [
          {
            Name = "auth0"
          },
        ]
      }

    }]
  }
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests: 
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

Successful request
```


### TODO

* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added


---

<details>
<summary> Overview of commits </summary>

  - 70536f5a38 

</details>

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-07-17 15:50:21 +00:00
..
ae copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
auto-config Backport of agent: remove agent cache dependency from service mesh leaf certificate management into release/1.16.x (#17704) 2023-06-13 16:12:43 +00:00
blockingquery [OSS] gRPC Blocking Queries (#17426) 2023-05-23 17:29:10 -04:00
cache Revert "cache: refactor agent cache fetching to prevent unnecessary f… (#16818) (#17046) 2023-04-19 13:17:21 -05:00
cache-types Backport of agent: remove agent cache dependency from service mesh leaf certificate management into release/1.16.x (#17704) 2023-06-13 16:12:43 +00:00
checks ci: remove test-integrations CircleCI workflow (#16928) 2023-04-19 16:19:29 +00:00
config Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
configentry backport of commit fe5c145cdd (#18010) 2023-07-05 14:50:59 +00:00
connect Backport of Passes configured role name to Vault for AWS auth in Connect CA into release/1.16.x (#18099) 2023-07-12 15:43:36 +00:00
consul Backport of feat: include nodes count in operator usage endpoint and cli command into release/1.16.x (#18012) 2023-07-05 15:37:51 +00:00
debug copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
dns copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
envoyextensions backport of commit c0afba3a0c (#17975) 2023-06-30 14:20:50 +00:00
exec copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
grpc-external backport of commit f2e26c36ec (#17870) 2023-06-26 12:40:50 +00:00
grpc-internal Various bits of cleanup detected when using Go Workspaces (#17462) 2023-06-05 16:08:39 -04:00
grpc-middleware Various bits of cleanup detected when using Go Workspaces (#17462) 2023-06-05 16:08:39 -04:00
hcp HCP Add node id/name to config (#17750) (#17796) 2023-06-16 22:25:33 +00:00
leafcert Backport of agent: remove agent cache dependency from service mesh leaf certificate management into release/1.16.x (#17704) 2023-06-13 16:12:43 +00:00
local fix some testing.T retry.R mixups (#17600) 2023-06-07 13:53:27 -05:00
log-drop copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
metadata add necessary plumbing to implement per server ip based rate limiting (#17436) 2023-05-23 15:37:01 -04:00
metrics copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
mock copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
pool Raft storage backend (#16619) 2023-04-04 17:30:06 +01:00
proxycfg Backport of [OSS] Fix initial_fetch_timeout to wait for all xDS resources into release/1.16.x (#18065) 2023-07-10 21:27:56 +00:00
proxycfg-glue Backport of Fix issue with streaming service health watches. into release/1.16.x (#17776) 2023-06-15 18:06:09 +00:00
proxycfg-sources copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
router copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
routine-leak-checker copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
rpc fix rate limiting mapping to be the same between api and struct packages (#17599) 2023-06-07 14:50:22 -04:00
rpcclient [COMPLIANCE] Add Copyright and License Headers (#16854) 2023-04-20 12:40:22 +00:00
structs backport of commit 8ad61b4e29 (#18077) 2023-07-11 15:44:44 +00:00
submatview * added Sameness Group to proto files (#16998) 2023-04-14 09:24:46 -07:00
systemd copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
token copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
uiserver copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
xds Backport of Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter into release/1.16.x (#18153) 2023-07-17 15:50:21 +00:00
acl.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
acl_endpoint.go Backport of fixes #17732 - AccessorID in request body should be optional when updating ACL token into release/1.16.x (#17833) 2023-06-21 18:53:18 +00:00
acl_endpoint_test.go Backport of fixes #17732 - AccessorID in request body should be optional when updating ACL token into release/1.16.x (#17833) 2023-06-21 18:53:18 +00:00
acl_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
acl_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
agent.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
agent_endpoint.go Backport of agent: remove agent cache dependency from service mesh leaf certificate management into release/1.16.x (#17704) 2023-06-13 16:12:43 +00:00
agent_endpoint_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
agent_endpoint_oss_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
agent_endpoint_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
agent_oss.go Backport of Add writeAuditRPCEvent to agent_oss into release/1.16.x (#17608) 2023-06-07 19:01:05 -04:00
agent_oss_test.go feat: add reporting config with reload (#16890) 2023-04-11 15:04:02 -04:00
agent_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
apiserver.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
apiserver_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
catalog_endpoint.go Add /v1/internal/service-virtual-ip for manually setting service VIPs (#17294) 2023-05-12 00:38:52 +00:00
catalog_endpoint_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
catalog_endpoint_test.go Add /v1/internal/service-virtual-ip for manually setting service VIPs (#17294) 2023-05-12 00:38:52 +00:00
check.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
config_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
config_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
connect_auth.go Add sameness groups to service intentions. (#17064) 2023-04-20 12:16:04 -04:00
connect_ca_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
connect_ca_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
coordinate_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
coordinate_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
delegate_mock_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
denylist.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
denylist_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
discovery_chain_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
discovery_chain_endpoint_test.go Attach service virtual IP info to compiled discovery chain (#17295) 2023-05-12 02:28:16 +00:00
dns.go * added Sameness Group to proto files (#16998) 2023-04-14 09:24:46 -07:00
dns_oss.go * added Sameness Group to proto files (#16998) 2023-04-14 09:24:46 -07:00
dns_oss_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
dns_test.go fix some testing.T retry.R mixups (#17600) 2023-06-07 13:53:27 -05:00
enterprise_delegate_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
event_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
event_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
federation_state_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
health_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
health_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
http.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
http_decode_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
http_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
http_oss_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
http_register.go Add version endpoint (#17506) 2023-05-30 17:25:48 -05:00
http_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
intentions_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
intentions_endpoint_oss_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
intentions_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
keyring.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
keyring_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
kvs_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
kvs_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
metrics.go log warning about certificate expiring sooner and with more details 2023-04-07 20:38:07 +00:00
metrics_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
nodeid.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
nodeid_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
notify.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
notify_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
operator_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
operator_endpoint_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
operator_endpoint_oss_test.go Backport of feat: include nodes count in operator usage endpoint and cli command into release/1.16.x (#18012) 2023-07-05 15:37:51 +00:00
operator_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
peering_endpoint.go [OSS] gRPC Blocking Queries (#17426) 2023-05-23 17:29:10 -04:00
peering_endpoint_oss_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
peering_endpoint_test.go [OSS] gRPC Blocking Queries (#17426) 2023-05-23 17:29:10 -04:00
prepared_query_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
prepared_query_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
proxycfg_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
reload.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
remote_exec.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
remote_exec_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
retry_join.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
retry_join_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
service_checks_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
service_manager.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
service_manager_test.go Remove deprecated service-defaults upstream behavior. (#16957) 2023-04-11 10:20:33 -05:00
session_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
session_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
setup.go HCP Add node id/name to config (#17750) (#17796) 2023-06-16 22:25:33 +00:00
setup_oss.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
sidecar_service.go sidecar-proxy refactor (#17328) 2023-05-12 16:49:42 -04:00
sidecar_service_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
signal_unix.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
signal_windows.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
snapshot_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
snapshot_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
status_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
status_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
streaming_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
testagent.go Revert "cache: refactor agent cache fetching to prevent unnecessary f… (#16818) (#17046) 2023-04-19 13:17:21 -05:00
testagent_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
translate_addr.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
txn_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
txn_endpoint_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
ui_endpoint.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
ui_endpoint_oss_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
ui_endpoint_test.go Backport of feature - [NET - 4005] - [Supportability] Reloadable Configuration - enable_debug into release/1.16.x (#17969) 2023-06-30 18:40:20 +05:30
user_event.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
user_event_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
util.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
util_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
watch_handler.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00
watch_handler_test.go copyright headers for agent folder (#16704) 2023-03-28 14:39:22 -04:00