* Add response header filters to http-route config entry definitions
* Map response header filters from config entry when constructing route destination
* Support response header modifiers at the service level as well
* Update protobuf definitions
* Update existing unit tests
* Add response filters to route consolidation logic
* Make existing unit tests more robust
* Add missing docstring
* Add changelog entry
* Add response filter modifiers to existing integration test
* Add more robust testing for response header modifiers in the discovery chain
* Add more robust testing for request header modifiers in the discovery chain
* Modify test to verify that service filter modifiers take precedence over rule filter modifiers
* [NET-5325] ACL templated policies support in tokens and roles
- Add API support for creating tokens/roles with templated-policies
- Add CLI support for creating tokens/roles with templated-policies
* adding changelog
* This controller generates and saves ProxyStateTemplate for sidecar proxies.
* It currently supports single-port L4 ports only.
* It keeps a cache of all destinations to make it easier to compute and retrieve destinations.
* It will update the status of the pbmesh.Upstreams resource if anything is invalid.
* This commit also changes service endpoints to include workload identity. This made the implementation a bit easier as we don't need to look up as many workloads and instead rely on endpoints data.
update ENVOY_VERSION and documentation of it used in the bats envoy tests.
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
* Add Enabler interface to turn sink on/off
* Use h for hcpProviderImpl vars, fix PR feeback and fix errors
* Keep nil check in exporter and fix tests
* Clarify comment and fix function name
* Use disable instead of enable
* Fix errors nit in otlp_transform
* Add test for refreshInterval of updateConfig
* Add disabled field in MetricsConfig struct
* Fix PR feedback: improve comment and remove double colons
* Fix deps test which requires a maybe
* Update hcp-sdk-go to v0.61.0
* use disabled flag in telemetry_config.go
* Handle 4XX errors in telemetry_provider
* Fix deps test
* Check 4XX instead
* Run make go-mod-tidy
* Support locality testing in consul-container
Support including locality in client sidecar config.
Also align test config structs with Ent to avoid future conflicts.
* Refactor consul-container fortio helpers
Refactor fortio test helpers to separate HTTP retries from waiting on
fortio result changes due to e.g. service startup and failovers.
* Adding explicit MPL license for sub-package
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
* Update copyright file headers to BUSL-1.1
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
* bump testcontainers-go from 0.22.0 and remove pinned go version in integ test
* go mod tidy
* Replace deprecated target.Authority with target.URL.Host
* OTElExporter now uses an EndpointProvider to discover the endpoint
* OTELSink uses a ConfigProvider to obtain filters and labels configuration
* improve tests for otel_sink
* Regex logic is moved into client for a method on the TelemetryConfig object
* Create a telemetry_config_provider and update deps to use it
* Fix conversion
* fix import newline
* Add logger to hcp client and move telemetry_config out of the client.go file
* Add a telemetry_config.go to refactor client.go
* Update deps
* update hcp deps test
* Modify telemetry_config_providers
* Check for nil filters
* PR review updates
* Fix comments and move around pieces
* Fix comments
* Remove context from client struct
* Moved ctx out of sink struct and fixed filters, added a test
* Remove named imports, use errors.New if not fformatting
* Remove HCP dependencies in telemetry package
* Add success metric and move lock only to grab the t.cfgHahs
* Update hash
* fix nits
* Create an equals method and add tests
* Improve telemetry_config_provider.go tests
* Add race test
* Add missing godoc
* Remove mock for MetricsClient
* Avoid goroutine test panics
* trying to kick CI lint issues by upgrading mod
* imprve test code and add hasher for testing
* Use structure logging for filters, fix error constants, and default to allow all regex
* removed hashin and modify logic to simplify
* Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test
* Ran make go-mod-tidy
* Use errtypes in the test
* Add changelog
* add safety check for exporter endpoint
* remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter
* Fixed race test to have changing config values
* Send success metric before modifying config
* Avoid the defer and move the success metric under
* wasm integration tests for local and remote wasm files
refactoring and cleanup for wasm testing
remove wasm debug logging
PR feedback, wasm build lock
correct path pattern for wasm build files
Add new helper function to minimize changes to existing test code
Remove extra param
mod tidy
add custom service setup to test lib
add wait until static server sidecar can reach nginx sidecar
Doc comments
PR feedback
Update workflows to compile wasm for integration tests
Fix docker build path
Fix package name for linter
Update makefile, fix redeclared function
Update expected wasm filename
Debug test ls in workflow
remove pwd in favor of relative path
more debugging
Build wasm in compatability tests as well
Build wasm directly in ci rather than in container
Debug tinygo and llvm version
Change wasm file extension
Remove tinygo debugging
Remove extra comments
* Add compiled wasm and build instructions
Bump golang.org/x/net to 0.12.0
While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.
See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.
Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.
### Description
<!-- Please describe why you're making this change, in plain English.
-->
- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.
This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.
### Testing & Reproduction steps
<!--
* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding
-->
- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.
- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
protocol = "http"
}
```
- Create two providers
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjog....."
}
}
```
```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"
JSONWebKeySet = {
Local = {
JWKS = "eyJrZXlzIjogW3...."
}
}
```
- add a service intention
```
Kind = "service-intentions"
Name = "redis"
JWT = {
Providers = [
{
Name = "okta"
},
]
}
Sources = [
{
Name = "*"
Permissions = [{
Action = "allow"
HTTP = {
PathPrefix = "/workspace"
}
JWT = {
Providers = [
{
Name = "okta"
VerifyClaims = [
{
Path = ["aud"]
Value = "my_client_app"
},
{
Path = ["sub"]
Value = "5be86359073c434bad2da3932222dabe"
}
]
},
]
}
},
{
Action = "allow"
HTTP = {
PathPrefix = "/"
}
JWT = {
Providers = [
{
Name = "auth0"
},
]
}
}]
}
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests:
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
RBAC filter denied
curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v
Successful request
```
### TODO
* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added
* fix(cli): remove failing check from 'connect envoy' registration for api gateway
* test(integration): add tests to check catalog statsus of gateways on startup
* remove extra sleep comment
* Update test/integration/consul-container/libs/assert/service.go
* changelog
- Provide more realistics examples for setting properties not already
supported natively by Consul
- Remove superfluous commas from HCL, correct target service name, and
fix service defaults vs. proxy defaults in examples
- Align existing integration test to updated docs
* Implement the Catalog V2 controller integration container tests
This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing.
* Add an upgrade test for the new catalog
Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2.
* Make testutil.Retry capable of performing cleanup operations
These cleanup operations are executed after each retry attempt.
* Move TestContext to taking an interface instead of a concrete testing.T
This allows this to be used on a retry.R or generally anything that meets the interface.
* Move to using TestContext instead of background contexts
Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface.
Co-authored-by: Daniel Upton <daniel@floppy.co>
Ensure that the embedded api struct is properly parsed when
deserializing config containing a set ResourceFilter.Services field.
Also enhance existing integration test to guard against bugs and
exercise this field.
TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors.
This commit:
1. Fixes envoy-library-references.sh to work again
2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older)
3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it.
4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field.
4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules
5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment.
6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible
7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner.
8. Updated linter rules to prevent usage of golang/protobuf
9. Updated a leader peering test to account for an extra colon in a grpc error message.
* endpoints xds cluster configuration
* resources test fix
* fix reversion in resources_test
* Update agent/proxycfg/api_gateway.go
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
* gofmt
* Modify getReadyUpstreams to filter upstreams by listener (#17410)
Each listener would previously have all upstreams from any route that bound to the listener. This is problematic when a route bound to one listener also binds to other listeners and so includes upstreams for multiple listeners. The list for a given listener would then wind up including upstreams for other listeners.
* Update agent/proxycfg/api_gateway.go
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
* Restore import blocking
* Skip to next route if route has no upstreams
* cleanup
* change set from bool to empty struct
---------
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Nathan Coleman