Commit Graph

137 Commits (d565056fb0352f67b110fdba2c3948822624be9d)

Author SHA1 Message Date
hashicorp-copywrite[bot] 5fb9df1640
[COMPLIANCE] License changes (#18443)
* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

* Update copyright file headers to BUSL-1.1

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-08-11 09:12:13 -04:00
Dan Stough 948ce8bc23
build: updates for 1.16.1 release (#18415)
* build: updates for 1.16.1 release

* build: fix missing replace directive for envoyextensions

* build: go mod tidy testing/deployer
2023-08-09 17:12:34 -04:00
cskh 43d8898e08
bump testcontainers-go from 0.22.0 and remove pinned go version in in… (#18395)
* bump testcontainers-go from 0.22.0 and remove pinned go version in integ test

* go mod tidy

* Replace deprecated target.Authority with target.URL.Host
2023-08-08 18:08:14 +00:00
cskh 7902ae20a1
Upgrade test: remove outdated test and disable log due to verbosity (#18403)
* remove outdated test

* disable log since we have too many parallel tests
2023-08-08 13:16:32 -04:00
R.B. Boyer 48effe5f8a
chore: make go-mod-tidy (#18388) 2023-08-07 14:02:34 -05:00
Michael Zalimeni 905e371607
[NET-5146] security: Update Go version to 1.20.7 and `x/net` to 0.13.0 (#18358)
* Update Go version to 1.20.7

This resolves [CVE-2023-29409]
(https://nvd.nist.gov/vuln/detail/CVE-2023-29409)(`crypto/tls`).

* Bump golang.org/x/net to 0.13.0

Addresses [CVE-2023-3978](https://nvd.nist.gov/vuln/detail/CVE-2023-3978)
for security scans (non-impacting).
2023-08-02 13:10:29 -04:00
Ashvitha 828567c62e
[HCP Telemetry] Periodic Refresh for Dynamic Telemetry Configuration (#18168)
* OTElExporter now uses an EndpointProvider to discover the endpoint

* OTELSink uses a ConfigProvider to obtain filters and labels configuration

* improve tests for otel_sink

* Regex logic is moved into client for a method on the TelemetryConfig object

* Create a telemetry_config_provider and update deps to use it

* Fix conversion

* fix import newline

* Add logger to hcp client and move telemetry_config out of the client.go file

* Add a telemetry_config.go to refactor client.go

* Update deps

* update hcp deps test

* Modify telemetry_config_providers

* Check for nil filters

* PR review updates

* Fix comments and move around pieces

* Fix comments

* Remove context from client struct

* Moved ctx out of sink struct and fixed filters, added a test

* Remove named imports, use errors.New if not fformatting

* Remove HCP dependencies in telemetry package

* Add success metric and move lock only to grab the t.cfgHahs

* Update hash

* fix nits

* Create an equals method and add tests

* Improve telemetry_config_provider.go tests

* Add race test

* Add missing godoc

* Remove mock for MetricsClient

* Avoid goroutine test panics

* trying to kick CI lint issues by upgrading mod

* imprve test code and add hasher for testing

* Use structure logging for filters, fix error constants, and default to allow all regex

* removed hashin and modify logic to simplify

* Improve race test and fix PR feedback by removing hash equals and avoid testing the timer.Ticker logic, and instead unit test

* Ran make go-mod-tidy

* Use errtypes in the test

* Add changelog

* add safety check for exporter endpoint

* remove require.Contains by using error types, fix structure logging, and fix success metric typo in exporter

* Fixed race test to have changing config values

* Send success metric before modifying config

* Avoid the defer and move the success metric under
2023-08-01 17:20:18 -04:00
John Landa 2a8bf5df61
Wasm integration tests for local and remote wasm files (#17756)
* wasm integration tests for local and remote wasm files

refactoring and cleanup for wasm testing

remove wasm debug logging

PR feedback, wasm build lock

correct path pattern for wasm build files

Add new helper function to minimize changes to existing test code

Remove extra param

mod tidy

add custom service setup to test lib

add wait until static server sidecar can reach nginx sidecar

Doc comments

PR feedback

Update workflows to compile wasm for integration tests

Fix docker build path

Fix package name for linter

Update makefile, fix redeclared function

Update expected wasm filename

Debug test ls in workflow

remove pwd in favor of relative path

more debugging

Build wasm in compatability tests as well

Build wasm directly in ci rather than in container

Debug tinygo and llvm version

Change wasm file extension

Remove tinygo debugging

Remove extra comments

* Add compiled wasm and build instructions
2023-08-01 15:49:39 -05:00
cskh c2066b9b22
NET-4222 take config file consul container (#18218)
Net 4222 take config file consul container
2023-07-20 21:30:22 -04:00
Michael Zalimeni 72999bb735
Update submodules to latest following 1.16.0 (#18197)
Align all our internal use of submodules on the latest versions.
2023-07-19 18:57:34 -04:00
Michael Zalimeni 29cdb75d05
[NET-4865] Bump golang.org/x/net to 0.12.0 (#18186)
Bump golang.org/x/net to 0.12.0

While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.

See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.

Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.
2023-07-19 11:37:58 -04:00
Nick Irvine cd3fc9e1d0
add peering_commontopo tests [NET-3700] (#17951)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: NiniOak <anita.akaeze@hashicorp.com>
2023-07-18 16:41:30 -07:00
Ronald 921445712e
[NET-4792] Add integrations tests for jwt-auth (#18169) 2023-07-18 14:59:01 -04:00
Ronald 03cf37e7b8
Re-order expected/actual for assertContainerState in consul container tests (#18157)
Re-order expected/actual, consul container tests
2023-07-17 18:11:19 +00:00
Ronald bcc6a9d752
Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter (#18062)
### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.

This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.

- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
  protocol = "http"
}
```
- Create two providers 
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"

JSONWebKeySet = {
    Local = {
     JWKS = "eyJrZXlzIjog....."
    }
}
```

```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"

JSONWebKeySet = {
   Local = {
     JWKS = "eyJrZXlzIjogW3...."
    }
}
```

- add a service intention
```
Kind = "service-intentions"
Name = "redis"

JWT = {
  Providers = [
    {
      Name = "okta"
    },
  ]
}

Sources = [
  {
    Name = "*"
    Permissions = [{
      Action = "allow"
      HTTP = {
        PathPrefix = "/workspace"
      }
      JWT = {
        Providers = [
          {
            Name = "okta"
            VerifyClaims = [
              {
                  Path = ["aud"]
                  Value = "my_client_app"
              },
              {
                Path = ["sub"]
                Value = "5be86359073c434bad2da3932222dabe"
              }
            ]
          },
        ]
      }

    },
    {
      Action = "allow"
      HTTP = {
        PathPrefix = "/"
      }
      JWT = {
        Providers = [
          {
            Name = "auth0"
          },
        ]
      }

    }]
  }
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests: 
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

Successful request
```


### TODO

* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added
2023-07-17 11:32:49 -04:00
Nick Irvine 05b665e856
chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] (#18130) 2023-07-14 17:13:41 -04:00
Dan Stough 1b08626358
[OSS] Fix initial_fetch_timeout to wait for all xDS resources (#18024)
* fix(connect): set initial_fetch_time to wait indefinitely

* changelog

* PR feedback 1
2023-07-10 17:08:06 -04:00
Ronald ada3938115
Add first integration test for jwt auth with intention (#18005) 2023-07-06 07:27:30 -04:00
Dan Stough b94095d92e
[OSS] Improve Gateway Test Coverage of Catalog Health (#18011)
* fix(cli): remove failing check from 'connect envoy' registration for api gateway

* test(integration): add tests to check catalog statsus of gateways on startup

* remove extra sleep comment

* Update test/integration/consul-container/libs/assert/service.go

* changelog
2023-07-05 11:30:48 -04:00
Chris Thain 4f0bdd35e6
Integration test for ext-authz Envoy extension (#17980) 2023-07-04 08:09:17 -07:00
Dan Bond 7dbba6c94d
deps: coredns v1.10.1 (#17912) 2023-06-28 07:53:43 -07:00
wangxinyi7 b782f2e39a
counter part of ent pr (#17618) 2023-06-22 10:50:46 -07:00
Michael Zalimeni d0797c4a0d
Fixup consul-container/test/debugging.md (#17815)
Add missing `-t` flag and fix minor typo.
2023-06-21 17:52:00 +00:00
Eric Haberkorn a3ba559149
Make locality aware routing xDS changes (#17826) 2023-06-21 12:39:53 -04:00
John Murret 6d39328771
Add documentation for remote debugging of integration tests. (#17800)
* Add documentation for remote debugging of integration tests.

* add link from main docs page.

* changes related to PR feedback
2023-06-20 15:14:15 +00:00
Matt Keeler 37636eab71
Catalog V2 Container Based Integration Test (#17674)
* Implement the Catalog V2 controller integration container tests

This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing.

* Add an upgrade test for the new catalog

Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2.

* Make testutil.Retry capable of performing cleanup operations

These cleanup operations are executed after each retry attempt.

* Move TestContext to taking an interface instead of a concrete testing.T

This allows this to be used on a retry.R or generally anything that meets the interface.

* Move to using TestContext instead of background contexts

Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface.


Co-authored-by: Daniel Upton <daniel@floppy.co>
2023-06-16 16:29:50 -04:00
Matt Keeler 77f44fa878
Various bits of cleanup detected when using Go Workspaces (#17462)
TLDR with many modules the versions included in each diverged quite a bit. Attempting to use Go Workspaces produces a bunch of errors.

This commit:

1. Fixes envoy-library-references.sh to work again
2. Ensures we are pulling in go-control-plane@v0.11.0 everywhere (previously it was at that version in some modules and others were much older)
3. Remove one usage of golang/protobuf that caused us to have a direct dependency on it.
4. Remove deprecated usage of the Endpoint field in the grpc resolver.Target struct. The current version of grpc (v1.55.0) has removed that field and recommended replacement with URL.Opaque and calls to the Endpoint() func when needing to consume the previous field.
4. `go work init <all the paths to go.mod files>` && `go work sync`. This syncrhonized versions of dependencies from the main workspace/root module to all submodules
5. Updated .gitignore to ignore the go.work and go.work.sum files. This seems to be standard practice at the moment.
6. Update doc comments in protoc-gen-consul-rate-limit to be go fmt compatible
7. Upgraded makefile infra to perform linting, testing and go mod tidy on all modules in a flexible manner.
8. Updated linter rules to prevent usage of golang/protobuf
9. Updated a leader peering test to account for an extra colon in a grpc error message.
2023-06-05 16:08:39 -04:00
Poonam Jadhav 217248ba31
feat: expose logs method on container interface (#17526) 2023-05-31 13:18:00 -04:00
Paul Glass f0ba3f40bc
Integration test for permissive mTLS (#17205)
* Integration test for permissive mTLS
2023-05-23 11:11:27 -05:00
Paul Glass 7f4fd2735a
Only synthesize anonymous token in primary DC (#17231)
* Only synthesize anonymous token in primary DC
* Add integration test for wan fed issue
2023-05-23 09:38:04 -05:00
cskh 1339c79f8d
consul-container test: no splitting and on single runner (#17394) 2023-05-17 14:57:12 -04:00
R.B. Boyer 2f5256ec7a
test: slight refactoring ahead of peering testing improvements (#17387) 2023-05-16 14:57:24 -05:00
R.B. Boyer 06481bf03a
test: fix oss/ent drift in gateway container tests (#17365) 2023-05-16 11:49:27 -05:00
cskh 17f06b8808
upgrade test: fix on-the-fly-image build and downsize runner (#17331) 2023-05-15 09:33:05 -04:00
cskh 2edfda998a
consul-container: mitigate the drift from ent repo (#17323) 2023-05-12 13:03:30 -04:00
cskh c61e994fc0
Container test: fix container test slow image build (#17316)
Container integ test: fix container test slow image build
2023-05-11 22:49:49 +00:00
cskh 83ad0dfa74
Upgrade test target image (#17226)
* upgrade test: add targetimage name as parameter to upgrade function

- the image name of latest version and target version could be
  different. Add the parameter of targetImage to the upgrade
  function

* fix a bug of expected error
2023-05-08 12:02:31 -04:00
Paul Glass b431b04d0c
TProxy integration test (#17103)
* TProxy integration test
* Fix GHA compatibility integration test command

Previously, when test splitting allocated multiple test directories to a
runner, the workflow ran `go tests "./test/dir1 ./test/dir2"` which
results in a directory not found error. This fixes that.
2023-04-26 11:49:38 -05:00
Semir Patel e7bb8fdf15
Fix or disable pipeline breaking changes that made it into main in last day or so (#17130)
* Fix straggler from renaming Register->RegisterTypes

* somehow a lint failure got through previously

* Fix lint-consul-retry errors

* adding in fix for success jobs getting skipped. (#17132)

* Temporarily disable inmem backend conformance test to get green pipeline

* Another test needs disabling

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-04-25 15:17:48 -05:00
Paul Banks 9e35c47bbd
De-flake snapshot test (#17120) 2023-04-25 15:25:26 +01:00
John Maguire e47f3216e5
APIGW Normalize Status Conditions (#16994)
* normalize status conditions for gateways and routes

* Added tests for checking condition status and panic conditions for
validating combinations, added dummy code for fsm store

* get rid of unneeded gateway condition generator struct

* Remove unused file

* run go mod tidy

* Update tests, add conflicted gateway status

* put back removed status for test

* Fix linting violation, remove custom conflicted status

* Update fsm commands oss

* Fix incorrect combination of type/condition/status

* cleaning up from PR review

* Change "invalidCertificate" to be of accepted status

* Move status condition enums into api package

* Update gateways controller and generated code

* Update conditions in fsm oss tests

* run go mod tidy on consul-container module to fix linting

* Fix type for gateway endpoint test

* go mod tidy from changes to api

* go mod tidy on troubleshoot

* Fix route conflicted reason

* fix route conflict reason rename

* Fix text for gateway conflicted status

* Add valid certificate ref condition setting

* Revert change to resolved refs to be handled in future PR
2023-04-24 16:22:55 -04:00
R.B. Boyer 9db223f54b
fix the linter (#17077) 2023-04-20 17:49:08 -04:00
Anita Akaeze fece53c48e
NET-3648: Add script to get consul and envoy version (#17060) 2023-04-20 13:11:11 -04:00
hashicorp-copywrite[bot] 9f81fc01e9
[COMPLIANCE] Add Copyright and License Headers (#16854)
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
2023-04-20 12:40:22 +00:00
Dhia Ayachi 711179db96
add ability to start container tests in debug mode and attach a debugger (#16887)
* add ability to start container tests in debug mode and attach a debugger to consul while running it.

* add a debug message with the debug port

* use pod to get the right port

* fix image used in basic test

* add more data to identify which container to debug.

* fix comment

Co-authored-by: Evan Culver <eculver@users.noreply.github.com>

* rename debugUri to debugURI

---------

Co-authored-by: Evan Culver <eculver@users.noreply.github.com>
2023-04-18 09:49:53 -04:00
Michael Wilkerson 0dd4ea2033
* added Sameness Group to proto files (#16998)
- added Sameness Group to config entries
- added Sameness Group to subscriptions

* generated proto files

* added Sameness Group events to the state store
- added test cases

* Refactored health RPC Client
- moved code that is common to rpcclient under rpcclient common.go. This will help set us up to support future RPC clients

* Refactored proxycfg glue views
- Moved views to rpcclient config entry. This will allow us to reuse this code for a config entry client

* added config entry RPC Client
- Copied most of the testing code from rpcclient/health

* hooked up new rpcclient in agent

* fixed documentation and comments for clarity
2023-04-14 09:24:46 -07:00
cskh 75bda84873
upgrade test: config nodeName, nodeid, and inherited persistent data for consul container (#16931) 2023-04-12 18:00:56 -04:00
Nick Irvine 4ae59923ff
port ENT upgrade tests flattening (#16824) 2023-03-30 13:07:16 -07:00
Nick Irvine 32358ddf7e
port ENT ingress gateway upgrade tests [NET-2294] [NET-2296] (#16804) 2023-03-29 09:51:21 -07:00
Ronald b64674623e
Copyright headers for missing files/folders (#16708)
* copyright headers for agent folder
2023-03-28 18:48:58 -04:00