* backport of commit c77fe8fb20
* backport of commit 7c3963cd2c
* backport of commit e706f4ccfd
* Add FIPS reference to consul enterprise docs (#18028)`
* Add FIPS reference to consul enterprise docs
* Update website/content/docs/enterprise/index.mdx
Co-authored-by: David Yu <dyu@hashicorp.com>
* remove support for ecs client (fips)
---------
Co-authored-by: David Yu <dyu@hashicorp.com>
---------
Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)
* Fix a bug that wrongly trims domains when there is an overlap with DC name
Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.
Example:
Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"
This change, fixes the issue by adding dot (.) before trimming
* trimDomain: ensure domain trimmed without modyfing original domains
* update changelog
---------
Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
* backport of commit 57bb6f3d72
* backport of commit b2dad88065
* backport of commit 753d3c0d3f
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* no-op commit due to failed cherry-picking
* add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139)
This is to correct a code problem because this assumes all segments, but
when you get to Enterprise, you can be in partition that is not the
default partition, in which case specifying all segments does not
validate and fails. This is to correct the setting of this filter with
`AllSegments` to `true` to only occur when in the the `default`
partition.
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
<!--
Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.
Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.
-->
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---------
Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
* no-op commit due to failed cherry-picking
* [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129)
### Description
This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.
go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.
[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)
### Testing & Reproduction steps
Check CI tests.
### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---------
Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
## Backport
This PR is auto-generated from #17754 to be assessed for backporting due
to the inclusion of the label backport/1.16.
🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.
The person who merged in the original PR is:
@WenInCode
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.
> merge conflict error: unable to process merge commit:
"1c757b8a2c1160ad53421b7b8bd7f74b205c4b89", automatic backport requires
rebase workflow
The below text is copied from the body of the original PR.
---
fixes#17097 Consul version of each nodes in UI nodes section
@jkirschner-hashicorp @huikang @team @Maintainers
Updated consul version in the request to register consul.
Added this as Node MetaData.
Fetching this new metadata in UI
<img width="1512" alt="Screenshot 2023-06-15 at 4 21 33 PM"
src="https://github.com/hashicorp/consul/assets/3139634/94f7cf6b-701f-4230-b9f7-d8c4342d0737">
Also made this backward compatible and tested.
Backward compatible in this context means - If consul binary with above
PR changes is deployed to one of node, and if UI is run from this node,
then the version of not only current (upgraded) node is displayed in UI
, but also of older nodes given that they are consul servers only.
For older (non-server or client) nodes the version is not added in
NodeMeta Data and hence the version will not be displayed for them.
If a old node is consul server, the version will be displayed. As the
endpoint - "v1/internal/ui/nodes?dc=dc1" was already returning version
in service meta. This is made use of in current UI changes.
<img width="1480" alt="Screenshot 2023-06-16 at 6 58 32 PM"
src="https://github.com/hashicorp/consul/assets/3139634/257942f4-fbed-437d-a492-37849d2bec4c">
---
<details>
<summary> Overview of commits </summary>
- 931fdfc7ec -
b3e2ec1cca -
8d0e9a5490 -
04e5d88cca -
28286a2e98 -
43e50ad382 -
0cf1b7077c -
27f34ce1c2 -
2ac76d62b8 -
3d618df9ef -
1c757b8a2c -
23ce82b4ce -
4dc1c9b4c5 -
85a12a9252 -
25d30a3fa9 -
7f1d6192dc -
5174cbff84
</details>
---------
Co-authored-by: Vijay Srinivas <vijayraghav22@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
## Backport
This PR is auto-generated from #18154 to be assessed for backporting due
to the inclusion of the label backport/1.16.
The below text is copied from the body of the original PR.
---
### Description
Addresses
https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705
### Testing & Reproduction steps
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
### Links
<!--
Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.
Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.
-->
### PR Checklist
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---
<details>
<summary> Overview of commits </summary>
- f5a6411ce7
</details>
Co-authored-by: David Yu <dyu@hashicorp.com>
## Backport
This PR is auto-generated from #18062 to be assessed for backporting due
to the inclusion of the label backport/1.16.
The below text is copied from the body of the original PR.
---
### Description
<!-- Please describe why you're making this change, in plain English.
-->
- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.
This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.
### Testing & Reproduction steps
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.
- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
protocol = "http"
}
```
- Create two providers
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjog....."
}
}
```
```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjogW3...."
}
}
```
- add a service intention
```
Kind = "service-intentions"
Name = "redis"
JWT = {
Providers = [
{
Name = "okta"
},
]
}
Sources = [
{
Name = "*"
Permissions = [{
Action = "allow"
HTTP = {
PathPrefix = "/workspace"
}
JWT = {
Providers = [
{
Name = "okta"
VerifyClaims = [
{
Path = ["aud"]
Value = "my_client_app"
},
{
Path = ["sub"]
Value = "5be86359073c434bad2da3932222dabe"
}
]
},
]
}
},
{
Action = "allow"
HTTP = {
PathPrefix = "/"
}
JWT = {
Providers = [
{
Name = "auth0"
},
]
}
}]
}
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests:
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
Successful request
```
### TODO
* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added
---
<details>
<summary> Overview of commits </summary>
- 70536f5a38
</details>
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
## Backport
This PR is auto-generated from #18134 to be assessed for backporting due
to the inclusion of the label backport/1.16.
The below text is copied from the body of the original PR.
---
### Description
- Fix unmatched bracket in the
[doc](https://developer.hashicorp.com/consul/docs/services/usage/checks#ttl-check-configuration)
(see the following screenshot of the page)
<img width="618" alt="Screenshot 2023-07-13 at 9 01 19 PM"
src="https://github.com/hashicorp/consul/assets/463631/20707735-906f-4b06-999d-44e6329a9fec">
### Testing & Reproduction steps
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
### Links
<!--
Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.
Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.
-->
### PR Checklist
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---
<details>
<summary> Overview of commits </summary>
- d40243b3a3
</details>
Co-authored-by: cskh <hui.kang@hashicorp.com>
## Backport
This PR is auto-generated from #18004 to be assessed for backporting due
to the inclusion of the label backport/1.16.
The below text is copied from the body of the original PR.
---
### Description
<!-- Please describe why you're making this change, in plain English.
-->
- Add jwt-provider docs for jwks cluster configuration. The
configuration was added here:
https://github.com/hashicorp/consul/pull/17978
---
<details>
<summary> Overview of commits </summary>
- 1ab3c3be1e
</details>
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
## Backport
This PR is auto-generated from #18124 to be assessed for backporting due
to the inclusion of the label backport/1.16.
🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.
The person who merged in the original PR is:
@jmurret
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.
> merge conflict error: POST
https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict
[]
The below text is copied from the body of the original PR.
---
### Description
The following jobs started failing when go 1.20.6 was released:
- `go-test-api-1-19`
- `go-test-api-1-20`
- `compatibility-integration-tests`
- `upgrade-integration-tests`
`compatibility-integration-tests` and `compatibility-integration-tests`
to this testcontainers issue:
https://github.com/testcontainers/testcontainers-go/issues/1359. This
issue calls for testcontainers to release a new version when one of
their dependencies is fixed. When that is done, we will unpin the go
versions in `compatibility-integration-tests` and
`compatibility-integration-tests`.
### Testing & Reproduction steps
See these jobs broken in CI and then see them work with this PR.
---
<details>
<summary> Overview of commits </summary>
- 747195f7aa -
516492420b -
f4d6ca19f8 -
a47407115e -
8c03b36e00 -
c50b17c46e -
7b55f66218 -
93ce5fcc61
</details>
---------
Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
* backport of commit 0d7bee8adc
* backport of commit 408cbe8ae0
* backport of commit a0854784dc
* backport of commit 71c4c6564f
* backport of commit 0c060fa2ba
---------
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* backport of commit 4034bb2b3e
* backport of commit 9c4c3c50f0
* backport of commit 7282078993
---------
Co-authored-by: Tom Davies <thomas.23.davies@bt.com>
hc-github-team-consul-core