Commit Graph

20229 Commits (a4ad2560850a76fe7994226707fd6d10bf01b658)

Author SHA1 Message Date
hc-github-team-consul-core a4ad256085
Backport of Clarify license reporting timing and GDPR compliance into release/1.16.x (#18238)
backport of commit 360c3dfad5

Co-authored-by: Judith Malnick <judith@hashicorp.com>
2023-07-21 23:24:20 +00:00
hc-github-team-consul-core 1e88432a13
Backport of [NET-4122] Doc guidance for federation with externalServers into release/1.16.x (#18233)
backport of commit c5522ca0e9

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-07-21 19:51:40 +00:00
hc-github-team-consul-core 32761e2403
Backport of [CC-5718] Remove HCP token requirement during bootstrap into release/1.16.x (#18229)
* [OSS] Post Consul 1.16 updates (#17606)

* chore: update dev build to 1.17

* chore(ci): add nightly 1.16 test

Drop the oldest and add the newest running release branch to nightly
builds.

* Add writeAuditRPCEvent to agent_oss (#17607)

* Add writeAuditRPCEvent to agent_oss

* fix the other diffs

* backport change log

* Add Envoy and Consul version constraints to Envoy extensions (#17612)

* [API Gateway] Fix trust domain for external peered services in synthesis code (#17609)

* [API Gateway] Fix trust domain for external peered services in synthesis code

* Add changelog

* backport ent changes to oss (#17614)

* backport ent changes to oss

* Update .changelog/_5669.txt

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

* Update intentions.mdx (#17619)

Make behaviour of L7 intentions clearer

* enterprise changelog update for audit (#17625)

* Update list of Envoy versions (#17546)

* [API Gateway] Fix rate limiting for API gateways (#17631)

* [API Gateway] Fix rate limiting for API gateways

* Add changelog

* Fix failing unit tests

* Fix operator usage tests for api package

* sort some imports that are wonky between oss and ent (#17637)

* PmTLS and tproxy improvements with failover and L7 traffic mgmt for k8s (#17624)

* porting over changes from enterprise repo to oss

* applied feedback on service mesh for k8s overview

* fixed typo

* removed ent-only build script file

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>

* Delete check-legacy-links-format.yml (#17647)

* docs: Reference doc updates for permissive mTLS settings (#17371)

* Reference doc updates for permissive mTLS settings
* Document config entry filtering
* Fix minor doc errors (double slashes in link url paths)

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Add generic experiments configuration and use it to enable catalog v2 resources (#17604)

* Add generic experiments configuration and use it to enable catalog v2 resources

* Run formatting with -s as CI will validate that this has been done

* api-gateway: stop adding all header filters to virtual host when generating xDS (#17644)

* Add header filter to api-gateway xDS golden test

* Stop adding all header filters to virtual host when generating xDS for api-gateway

* Regenerate xDS golden file for api-gateway w/ header filter

* fix: add agent info reporting log (#17654)

* Add new Consul 1.16 docs (#17651)

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* fix build errors

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Default `ProxyType` for builtin extensions (#17657)

* Post 1.16.0-rc1 updates (#17663)

- Update changelog to include new entries from release
- Update submodule versions to latest published

* Update service-defaults.mdx (#17656)

* docs: Sameness Groups (#17628)

* port from enterprise branch

* Apply suggestions from code review

Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>

* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx

* next steps

* Update website/content/docs/connect/cluster-peering/usage/create-sameness-groups.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/connect/cluster-peering/usage/create-sameness-groups.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Remove "BETA" marker from config entries (#17670)

* CAPIgw for K8s installation updates for 1.16 (#17627)

* trimmed CRD step and reqs from installation

* updated tech specs

* Apply suggestions from code review

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* added upgrade instruction

* removed tcp port req

* described downtime and DT-less upgrades

* applied additional review feedback

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* additional feedback on API gateway upgrades (#17677)

* additional feedback

* Update website/content/docs/api-gateway/upgrades.mdx

Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

---------

Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* docs: JWT Authorization for intentions (#17643)

* Initial page/nav creation

* configuration entry reference page

* Usage + fixes

* service intentions page

* usage

* description

* config entry updates

* formatting fixes

* Update website/content/docs/connect/config-entries/service-intentions.mdx

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* service intentions review fixes

* Overview page review fixes

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* docs: minor fixes to JWT auth docs (#17680)

* Fixes

* service intentions fixes

* Fix two WAL metrics in docs/agent/telemetry.mdx (#17593)

* updated failover for k8s w-tproxy page title (#17683)

* Add release notes 1.16 rc (#17665)

* Merge pull request #5773 from hashicorp/docs/rate-limiting-from-ip-addresses-1.16

updated docs for rate limiting for IP addresses - 1.16

* Merge pull request #5609 from hashicorp/docs/enterprise-utilization-reporting

Add docs for enterprise utilization reporting

* Merge pull request #5734 from hashicorp/docs/envoy-ext-1.16

Docs/envoy ext 1.16

* Add release notes for 1.16-rc

* Add consul-e license utlization reporting

* Update with rc absolute links

* Update with rc absolute links

* fix typo

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update to use callout component

* address typo

* docs: FIPS 140-2 Compliance (#17668)

* Page + nav + formatting

* link fix

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* link fix

* Apply suggestions from code review

Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* Update website/content/docs/enterprise/fips.mdx

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* fix apigw install values file

* fix typos in release notes

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>

* fix release notes links (#17687)

* adding redirects for tproxy and envoy extensions (#17688)

* adding redirects

* Apply suggestions from code review

* Fix FIPS copy (#17691)

* fix release notes links

* fix typos on fips docs

* [NET-4107][Supportability] Log Level set to TRACE and duration set to 5m for consul-debug (#17596)

* changed duration to 5 mins and log level to trace

* documentation update

* change log

* ENT merge of ext-authz extension updates (#17684)

* docs: Update default values for Envoy extension proxy types (#17676)

* fix: stop peering delete routine on leader loss (#17483)

* Refactor disco chain prioritize by locality structs (#17696)

This includes prioritize by localities on disco chain targets rather than
resolvers, allowing different targets within the same partition to have
different policies.

* agent: remove agent cache dependency from service mesh leaf certificate management (#17075)

* agent: remove agent cache dependency from service mesh leaf certificate management

This extracts the leaf cert management from within the agent cache.

This code was produced by the following process:

1. All tests in agent/cache, agent/cache-types, agent/auto-config,
   agent/consul/servercert were run at each stage.

    - The tests in agent matching .*Leaf were run at each stage.

    - The tests in agent/leafcert were run at each stage after they
      existed.

2. The former leaf cert Fetch implementation was extracted into a new
   package behind a "fake RPC" endpoint to make it look almost like all
   other cache type internals.

3. The old cache type was shimmed to use the fake RPC endpoint and
   generally cleaned up.

4. I selectively duplicated all of Get/Notify/NotifyCallback/Prepopulate
   from the agent/cache.Cache implementation over into the new package.
   This was renamed as leafcert.Manager.

    - Code that was irrelevant to the leaf cert type was deleted
      (inlining blocking=true, refresh=false)

5. Everything that used the leaf cert cache type (including proxycfg
   stuff) was shifted to use the leafcert.Manager instead.

6. agent/cache-types tests were moved and gently replumbed to execute
   as-is against a leafcert.Manager.

7. Inspired by some of the locking changes from derek's branch I split
   the fat lock into N+1 locks.

8. The waiter chan struct{} was eventually replaced with a
   singleflight.Group around cache updates, which was likely the biggest
   net structural change.

9. The awkward two layers or logic produced as a byproduct of marrying
   the agent cache management code with the leaf cert type code was
   slowly coalesced and flattened to remove confusion.

10. The .*Leaf tests from the agent package were copied and made to work
    directly against a leafcert.Manager to increase direct coverage.

I have done a best effort attempt to port the previous leaf-cert cache
type's tests over in spirit, as well as to take the e2e-ish tests in the
agent package with Leaf in the test name and copy those into the
agent/leafcert package to get more direct coverage, rather than coverage
tangled up in the agent logic.

There is no net-new test coverage, just coverage that was pushed around
from elsewhere.

* [core]: Pin github action workflows (#17695)

* docs: missing changelog for _5517 (#17706)

* add enterprise notes for IP-based rate limits (#17711)

* add enterprise notes for IP-based rate limits

* Apply suggestions from code review

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>

* added bolded 'Enterprise' in list items.

---------

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>

* Update compatibility.mdx (#17713)

* Remove extraneous version info for Config entries (#17716)

* Update terminating-gateway.mdx
* Update exported-services.mdx
* Update mesh.mdx

* fix: typo in link to section (#17527)

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Bump Alpine to 3.18 (#17719)

* Update Dockerfile

* Create 17719.txt

* NET-1825: New ACL token creation docs (#16465)

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* nits

* updated docs

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* OSS merge: Update error handling login when applying extensions (#17740)

* Bump atlassian/gajira-transition from 3.0.0 to 3.0.1 (#17741)

Bumps [atlassian/gajira-transition](https://github.com/atlassian/gajira-transition) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/atlassian/gajira-transition/releases)
- [Commits](4749176faf...38fc9cd61b)

---
updated-dependencies:
- dependency-name: atlassian/gajira-transition
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add truncation to body (#17723)

* docs: Failover overview minor fix (#17743)

* Incorrect symbol

* Clarification

* slight edit for clarity

* docs - update Envoy and Dataplane compat matrix (#17752)

* Update envoy.mdx

added more detail around default versus other compatible versions

* validate localities on agent configs and registration endpoints (#17712)

* Updated docs added explanation. (#17751)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* nits

* updated docs

* explanation added

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Update index.mdx (#17749)

* added redirects and updated links (#17764)

* Add transparent proxy enhancements changelog (#17757)

* docs - remove use of consul leave during upgrade instructions (#17758)

* Fix issue with streaming service health watches. (#17775)

Fix issue with streaming service health watches.

This commit fixes an issue where the health streams were unaware of service
export changes. Whenever an exported-services config entry is modified, it is
effectively an ACL change.

The bug would be triggered by the following situation:

- no services are exported
- an upstream watch to service X is spawned
- the streaming backend filters out data for service X (due to lack of exports)
- service X is finally exported

In the situation above, the streaming backend does not trigger a refresh of its
data.  This means that any events that were supposed to have been received prior
to the export are NOT backfilled, and the watches never see service X spawning.

We currently have decided to not trigger a stream refresh in this situation due
to the potential for a thundering herd effect (touching exports would cause a
re-fetch of all watches for that partition, potentially).  Therefore, a local
blocking-query approach was added by this commit for agentless.

It's also worth noting that the streaming subscription is currently bypassed
most of the time with agentful, because proxycfg has a `req.Source.Node != ""`
which prevents the `streamingEnabled` check from passing.  This means that while
agents should technically have this same issue, they don't experience it with
mesh health watches.

Note that this is a temporary fix that solves the issue for proxycfg, but not
service-discovery use cases.

* Property Override validation improvements (#17759)

* Reject inbound Prop Override patch with Services

Services filtering is only supported for outbound TrafficDirection patches.

* Improve Prop Override unexpected type validation

- Guard against additional invalid parent and target types
- Add specific error handling for Any fields (unsupported)

* Fixes (#17765)

* Update license get explanation (#17782)

This PR is to clarify what happens if the license get command is run on a follower if the leader hasn't been updated with a newer license.

* Add Patch index to Prop Override validation errors (#17777)

When a patch is found invalid, include its index for easier debugging
when multiple patches are provided.

* Stop referenced jwt providers from being deleted (#17755)

* Stop referenced jwt providers from being deleted

* Implement a Catalog Controllers Lifecycle Integration Test (#17435)

* Implement a Catalog Controllers Lifecycle Integration Test

* Prevent triggering the race detector.

This allows defining some variables for protobuf constants and using those in comparisons. Without that, something internal in the fmt package ended up looking at the protobuf message size cache and triggering the race detector.

* HCP Add node id/name to config (#17750)

* Catalog V2 Container Based Integration Test (#17674)

* Implement the Catalog V2 controller integration container tests

This now allows the container tests to import things from the root module. However for now we want to be very restrictive about which packages we allow importing.

* Add an upgrade test for the new catalog

Currently this should be dormant and not executed. However its put in place to detect breaking changes in the future and show an example of how to do an upgrade test with integration tests structured like catalog v2.

* Make testutil.Retry capable of performing cleanup operations

These cleanup operations are executed after each retry attempt.

* Move TestContext to taking an interface instead of a concrete testing.T

This allows this to be used on a retry.R or generally anything that meets the interface.

* Move to using TestContext instead of background contexts

Also this forces all test methods to implement the Cleanup method now instead of that being an optional interface.


Co-authored-by: Daniel Upton <daniel@floppy.co>

* Fix Docs for Trails Leader By (#17763)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* nits

* updated docs

* explanation added

* fix doc

* fix docs

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Improve Prop Override docs examples (#17799)

- Provide more realistics examples for setting properties not already
  supported natively by Consul
- Remove superfluous commas from HCL, correct target service name, and
  fix service defaults vs. proxy defaults in examples
- Align existing integration test to updated docs

* Test permissive mTLS filter chain not configured with tproxy disabled (#17747)

* Add documentation for remote debugging of integration tests. (#17800)

* Add documentation for remote debugging of integration tests.

* add link from main docs page.

* changes related to PR feedback

* Clarify limitations of Prop Override extension (#17801)

Explicitly document the limitations of the extension, particularly what
kind of fields it is capable of modifying.

* Fix formatting for webhook-certs Consul tutorial (#17810)

* Fix formatting for webhook-certs Consul tutorial
* Make a small grammar change to also pick up whitespace changes necessary for formatting

---------

Co-authored-by: David Yu <dyu@hashicorp.com>

* Add jwt-authn metrics to jwt-provider docs (#17816)

* [NET-3095] add jwt-authn metrics docs

* Change URLs for redirects from RC to default latest (#17822)

* Set GOPRIVATE for all hashicorp repos in CI (#17817)

Consistently set GOPRIVATE to include all hashicorp repos, s.t. private
modules are successfully pulled in enterprise CI.

* Make locality aware routing xDS changes (#17826)

* Fixup consul-container/test/debugging.md (#17815)

Add missing `-t` flag and fix minor typo.

* fixes #17732 - AccessorID in request body should be optional when updating ACL token (#17739)

* AccessorID in request body should be optional when updating ACL token

* add a test case

* fix test case

* add changelog entry for PR #17739

* CA provider doc updates and Vault provider minor update (#17831)

Update CA provider docs

Clarify that providers can differ between
primary and secondary datacenters

Provide a comparison chart for consul vs
vault CA providers

Loosen Vault CA provider validation for RootPKIPath

Update Vault CA provider documentation

* ext-authz Envoy extension: support `localhost` as a valid target URI. (#17821)

* CI Updates (#17834)

* Ensure that git access to private repos uses the ELEVATED_GITHUB_TOKEN

* Bump the runner size for the protobuf generation check

This has failed previously when the runner process that communicates with GitHub gets starved causing the job to fail.

* counter part of ent pr (#17618)

* watch: support -filter for consul watch: checks, services, nodes, service (#17780)

* watch: support -filter for watch checks

* Add filter for watch nodes, services, and service
- unit test added
- Add changelog
- update doc

* Trigger OSS => ENT merge for all release branches (#17853)

Previously, this only triggered for release/*.*.x branches; however, our release process involves cutting a release/1.16.0 branch, for example, at time of code freeze these days. Any PRs to that branch after code freeze today do not make their way to consul-enterprise. This will make behavior for a .0 branch consistent with current behavior for a .x branch.

* Update service-mesh.mdx (#17845)

Deleted two commas which looks quite like some leftovers.

* Add docs for sameness groups with resolvers. (#17851)

* docs: add note about path prefix matching behavior for HTTPRoute config (#17860)

* Add note about path prefix matching behavior for HTTPRoute config

* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* docs: update upgrade to consul-dataplane docs on k8s (#17852)

* resource: add `AuthorizerContext` helper method (#17393)

* resource: enforce consistent naming of resource types (#17611)

For consistency, resource type names must follow these rules:

- `Group` must be snake case, and in most cases a single word.
- `GroupVersion` must be lowercase, start with a "v" and end with a number.
- `Kind` must be pascal case.

These were chosen because they map to our protobuf type naming
conventions.

* tooling: generate protoset file (#17364)

Extends the `proto` make target to generate a protoset file for use with
grpcurl etc.

* Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)

* Fix a bug that wrongly trims domains when there is an overlap with DC name

Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.

Example:

Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"

This change, fixes the issue by adding dot (.) before trimming

* trimDomain: ensure domain trimmed without modyfing original domains

* update changelog

---------

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

* deps: aws-sdk-go v1.44.289 (#17876)

Signed-off-by: Dan Bond <danbond@protonmail.com>

* api-gateway: add operation cannot be fulfilled error to common errors (#17874)

* add error message

* Update website/content/docs/api-gateway/usage/errors.mdx

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* fix formating issues

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* api-gateway: add step to upgrade instructions for creating intentions (#17875)

* Changelog - add 1.13.9, 1.14.8, and 1.15.4 (#17889)

* docs: update config enable_debug (#17866)

* update doc for config enable_debug

* Update website/content/docs/agent/config/config-files.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update wording on WAN fed and intermediate_pki_path (#17850)

* Allow service identity tokens the ability to read jwt-providers (#17893)

* Allow service identity tokens the ability to read jwt-providers

* more tests

* service_prefix tests

* Update docs (#17476)

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Add emit_tags_as_labels to envoy bootstrap config when using Consul Telemetry Collector (#17888)

* Fix command from kg to kubectl get (#17903)

* Create and update release notes for 1.16 and 1.2 (#17895)

* update release notes for 1.16 and 1.2

* update latest consul core release

* Propose new changes to APIgw upgrade instructions (#17693)

* Propose new changes to APIgw upgrade instructions

* fix build error

* update callouts to render correctly

* Add hideClipboard to log messages

* Added clarification around consul k8s and crds

* Add workflow to verify linux release packages (#17904)

* adding docker files to verify linux packages.

* add verifr-release-linux.yml

* updating name

* pass inputs directly into jobs

* add other linux package platforms

* remove on push

* fix TARGETARCH on debian and ubuntu so it can check arm64 and amd64

* fixing amazon to use the continue line

* add ubuntu i386

* fix comment lines

* working

* remove commented out workflow jobs

* Apply suggestions from code review

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* update fedora and ubuntu to use latest tag

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* Reference hashicorp/consul instead of consul for Docker image (#17914)

* Reference hashicorp/consul instead of consul for Docker image

* Update Make targets that pull consul directly

* Update Consul K8s Upgrade Doc Updates (#17921)

Updating upgrade procedures to encompass expected errors during upgrade process from v1.13.x to v1.14.x.

* Update sameness-group.mdx (#17915)

* Update create-sameness-groups.mdx (#17927)

* deps: coredns v1.10.1 (#17912)

* Ensure RSA keys are at least 2048 bits in length (#17911)

* Ensure RSA keys are at least 2048 bits in length

* Add changelog

* update key length check for FIPS compliance

* Fix no new variables error and failing to return when error exists from
validating

* clean up code for better readability

* actually return value

* tlsutil: Fix check TLS configuration (#17481)

* tlsutil: Fix check TLS configuration
* Rewording docs.
* Update website/content/docs/services/configuration/checks-configuration-reference.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Fix typos and add changelog entry.
---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* docs: Deprecations for connect-native SDK and specific connect native APIs (#17937)

* Update v1_16_x.mdx
* Update connect native golang page

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Revert "Add workflow to verify linux release packages (#17904)" (#17942)

This reverts commit 3368f14fab.

* Fixes Secondary ConnectCA update (#17846)

This fixes a bug that was identified which resulted in subsequent
ConnectCA configuration update not to persist in the cluster.

* fixing typo in link to jwt-validations-with-intentions doc (#17955)

* Fix streaming backend link (#17958)

* Fix streaming backend link
* Update health.mdx

* Dynamically create jwks clusters for jwt-providers (#17944)

* website: remove deprecated agent rpc docs (#17962)

* Fix missing BalanceOutboundConnections in v2 catalog. (#17964)

* feature - [NET - 4005]  - [Supportability] Reloadable Configuration - enable_debug (#17565)

* # This is a combination of 9 commits.
# This is the 1st commit message:

init without tests

# This is the commit message #2:

change log

# This is the commit message #3:

fix tests

# This is the commit message #4:

fix tests

# This is the commit message #5:

added tests

# This is the commit message #6:

change log breaking change

# This is the commit message #7:

removed breaking change

# This is the commit message #8:

fix test

# This is the commit message #9:

keeping the test behaviour same

* # This is a combination of 12 commits.
# This is the 1st commit message:

init without tests

# This is the commit message #2:

change log

# This is the commit message #3:

fix tests

# This is the commit message #4:

fix tests

# This is the commit message #5:

added tests

# This is the commit message #6:

change log breaking change

# This is the commit message #7:

removed breaking change

# This is the commit message #8:

fix test

# This is the commit message #9:

keeping the test behaviour same

# This is the commit message #10:

made enable debug atomic bool

# This is the commit message #11:

fix lint

# This is the commit message #12:

fix test true enable debug

* parent 10f500e895
author absolutelightning <ashesh.vidyut@hashicorp.com> 1687352587 +0530
committer absolutelightning <ashesh.vidyut@hashicorp.com> 1687352592 +0530

init without tests

change log

fix tests

fix tests

added tests

change log breaking change

removed breaking change

fix test

keeping the test behaviour same

made enable debug atomic bool

fix lint

fix test true enable debug

using enable debug in agent as atomic bool

test fixes

fix tests

fix tests

added update on correct locaiton

fix tests

fix reloadable config enable debug

fix tests

fix init and acl 403

* revert commit

* Fix formatting codeblocks on APIgw docs (#17970)

* fix formatting codeblocks

* remove unnecessary indents

* Remove POC code (#17974)

* update doc (#17910)

* update doc

* update link

* Remove duplicate and unused newDecodeConfigEntry func (#17979)

* docs: samenessGroup YAML examples (#17984)

* configuration entry syntax

* Example config

* Add changelog entry for 1.16.0 (#17987)

* Fix typo (#17198)

servcies => services

* Expose JWKS cluster config through JWTProviderConfigEntry (#17978)

* Expose JWKS cluster config through JWTProviderConfigEntry

* fix typos, rename trustedCa to trustedCA

* Integration test for ext-authz Envoy extension (#17980)

* Fix incorrect protocol for transparent proxy upstreams. (#17894)

This PR fixes a bug that was introduced in:
https://github.com/hashicorp/consul/pull/16021

A user setting a protocol in proxy-defaults would cause tproxy implicit
upstreams to not honor the upstream service's protocol set in its
`ServiceDefaults.Protocol` field, and would instead always use the
proxy-defaults value.

Due to the fact that upstreams configured with "tcp" can successfully contact
upstream "http" services, this issue was not recognized until recently (a
proxy-defaults with "tcp" and a listening service with "http" would make
successful requests, but not the opposite).

As a temporary work-around, users experiencing this issue can explicitly set
the protocol on the `ServiceDefaults.UpstreamConfig.Overrides`, which should
take precedence.

The fix in this PR removes the proxy-defaults protocol from the wildcard
upstream that tproxy uses to configure implicit upstreams. When the protocol
was included, it would always overwrite the value during discovery chain
compilation, which was not correct. The discovery chain compiler also consumes
proxy defaults to determine the protocol, so simply excluding it from the
wildcard upstream config map resolves the issue.

* feat: include nodes count in operator usage endpoint and cli command (#17939)

* feat: update operator usage api endpoint to include nodes count

* feat: update operator usange cli command to includes nodes count

* [OSS] Improve Gateway Test Coverage of Catalog Health (#18011)

* fix(cli): remove failing check from 'connect envoy' registration for api gateway

* test(integration): add tests to check catalog statsus of gateways on startup

* remove extra sleep comment

* Update test/integration/consul-container/libs/assert/service.go

* changelog

* Fixes Traffic rate limitting docs (#17997)

* Fix removed service-to-service peering links (#17221)

* docs: fix removed service-to-service peering links

* docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam)

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* docs: Sameness "beta" warning (#18017)

* Warning updates

* .x

* updated typo in tab heading (#18022)

* updated typo in tab heading

* updated tab group typo, too

* Document that DNS lookups can target cluster peers (#17990)

Static DNS lookups, in addition to explicitly targeting a datacenter,
can target a cluster peer. This was added in 95dc0c7b30 but didn't make the documentation.

The driving function for the change is `parseLocality` here: 0b1299c28d/agent/dns_oss.go (L25)

The biggest change in this is to adjust the standard lookup syntax to tie
`.<datacenter>` to `.dc` as required-together, and to append in the similar `.<cluster-peer>.peer` optional argument, both to A record and SRV record lookups.

Co-authored-by: David Yu <dyu@hashicorp.com>

* Add first integration test for jwt auth with intention (#18005)

* fix stand-in text for name field (#18030)

* removed sameness conf entry from failover nav (#18033)

* docs - add service sync annotations and k8s service weight annotation (#18032)

* Docs for https://github.com/hashicorp/consul-k8s/pull/2293
* remove versions for enterprise features since they are old

---------

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

* docs - add jobs use case for service mesh k8s (#18037)

* docs - add jobs use case for service mesh k8s
* add code blocks

* address feedback (#18045)

* Add verify server hostname to tls default (#17155)

* [CC-5718] Remove HCP token requirement during bootstrap

* Re-add error for loading HCP management token

* backport of commit f1225f5d47

* backport of commit 5958ae0921

* backport of commit 536e6f3b3b

* backport of commit a99dd82929

* backport of commit fc680e806e

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Bryce Kalow <bkalow@hashicorp.com>
Co-authored-by: Paul Glass <pglass@hashicorp.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
Co-authored-by: Hariram Sankaran <56744845+ramramhariram@users.noreply.github.com>
Co-authored-by: shanafarkas <105076572+shanafarkas@users.noreply.github.com>
Co-authored-by: Thomas Eckert <teckert@hashicorp.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
Co-authored-by: Joshua Timmons <josh.timmons@hashicorp.com>
Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Curt Bushko <cbushko@gmail.com>
Co-authored-by: Tobias Birkefeld <t@craxs.de>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chappie <6537530+chapmanc@users.noreply.github.com>
Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Mark Campbell-Vincent <mnmvincent@gmail.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Steven Zamborsky <97125550+stevenzamborsky@users.noreply.github.com>
Co-authored-by: George Bolo <george.bolo@gmail.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com>
Co-authored-by: cskh <hui.kang@hashicorp.com>
Co-authored-by: V. K <cn007b@gmail.com>
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
Co-authored-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Gerard Nguyen <gerard@hashicorp.com>
Co-authored-by: mr-miles <miles.waller@gmail.com>
Co-authored-by: natemollica-dev <57850649+natemollica-nm@users.noreply.github.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Samantha <hello@entropy.cat>
Co-authored-by: Ranjandas <thejranjan@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Co-authored-by: Michael Hofer <karras@users.noreply.github.com>
Co-authored-by: J.C. Jones <james.jc.jones@gmail.com>
Co-authored-by: Fulvio <fulviodenza823@gmail.com>
Co-authored-by: Jeremy Jacobson <jeremy.jacobson@hashicorp.com>
Co-authored-by: Jeremy Jacobson <jjacobson93@users.noreply.github.com>
2023-07-21 17:53:53 +00:00
hc-github-team-consul-core e8fdb44e9e
Backport of ci: don't verify s390x into release/1.16.x (#18226)
backport of commit 45700f1e89

Co-authored-by: Dan Bond <danbond@protonmail.com>
2023-07-21 17:18:53 +00:00
hc-github-team-consul-core 713c8867fa
Backport of fix typos and update ecs compat table into release/1.16.x (#18220)
* backport of commit 216f79353f

* backport of commit 1df03269ef

* backport of commit 12df6a1ac9

---------

Co-authored-by: trujillo-adam <ajosetru@gmail.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-21 09:18:17 -07:00
hc-github-team-consul-core dd7b57883d
Backport of Explicitly enable WebSocket upgrades into release/1.16.x (#18211)
This PR explicitly enables WebSocket upgrades in Envoy's UpgradeConfig for all
proxy types. (API Gateway, Ingress, and Sidecar.)

Fixes #8283

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
2023-07-20 14:57:13 -07:00
hc-github-team-consul-core e9f9a8a6a5
Backport of docs: fix the description of client rpc into release/1.16.x (#18213)
backport of commit 4e8cdddb9f

Co-authored-by: cskh <hui.kang@hashicorp.com>
2023-07-20 20:55:05 +00:00
hc-github-team-consul-core 4c637e4ead
Backport of Fix Backport Assistant PR commenting into release/1.16.x (#18203)
* backport of commit de63f65b1c

* backport of commit f6c0fff6ce

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-07-20 16:14:15 +00:00
hc-github-team-consul-core dc58b454a7
Backport of Update submodules to latest following 1.16.0 into release/1.16.x (#18198)
Update submodules to latest following 1.16.0

Align all our internal use of submodules on the latest versions.

Manual backport of 235d9c5ca5.
2023-07-20 01:35:24 +00:00
hc-github-team-consul-core ec0072cec3
Backport of [NET-4865] security: Update Go version to 1.20.6 into release/1.16.x (#18195)
backport of commit 93f3209a69

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2023-07-19 21:57:40 +00:00
hc-github-team-consul-core 18db0624a1
Backport of Improve XDS test coverage: JWT auth edition into release/1.16.x (#18196)
* backport of commit 613db16e56

* backport of commit 40e085ad3f

---------

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-07-19 21:35:15 +00:00
hc-github-team-consul-core f9482b6620
Backport of [NET-4865] Bump golang.org/x/net to 0.12.0 into release/1.16.x (#18189)
Bump golang.org/x/net to 0.12.0

While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.

See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.

Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.

Manual backport of 84cbf09185.
2023-07-19 12:22:18 -04:00
Michael Zalimeni edda4f8bf9
[NET-3700] add `omitempty` to api prepared query targets (#18184)
add `omitempty` to api prepared query targets

Manual backport of c2bbe67714.

Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com>
2023-07-19 11:57:09 -04:00
Tu Nguyen 453052c2b4
Fix typo on docs telemetry page (#18185) 2023-07-19 08:11:19 -07:00
hc-github-team-consul-core 409becb085
Backport of docs - remove Sentinel from enterprise features list into release/1.16.x (#18182)
* resolve conflicts

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-19 02:03:30 +00:00
hc-github-team-consul-core 9b3bbccbcd
Backport of Add FIPS reference to consul enterprise docs into release/1.16.x (#18175)
* backport of commit c77fe8fb20

* backport of commit 7c3963cd2c

* backport of commit e706f4ccfd

* Add FIPS reference to consul enterprise docs (#18028)`
* Add FIPS reference to consul enterprise docs

* Update website/content/docs/enterprise/index.mdx

Co-authored-by: David Yu <dyu@hashicorp.com>

* remove support for ecs client (fips)

---------

Co-authored-by: David Yu <dyu@hashicorp.com>

---------

Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-18 13:37:08 -07:00
hc-github-team-consul-core 113dc42cea
Backport of [NET-4792] Add integrations tests for jwt-auth into release/1.16.x (#18173)
backport of commit e588acf2ee

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-07-18 19:14:05 +00:00
hc-github-team-consul-core 8012e069ad
Backport of chore: bump upgrade integrations tests to 1.15, 116 [NET-4743] into release/1.16.x (#18148)
Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com>
2023-07-18 10:04:11 -07:00
Dhia Ayachi bd336b96cf
Fix a bug that wrongly trims domains when there is an overlap with DC 1.16.x (#18164)
Fix a bug that wrongly trims domains when there is an overlap with DC name (#17160)

* Fix a bug that wrongly trims domains when there is an overlap with DC name

Before this change, when DC name and domain/alt-domain overlap, the domain name incorrectly trimmed from the query.

Example:

Given: datacenter = dc-test, alt-domain = test.consul.
Querying for "test-node.node.dc-test.consul" will faile, because the
code was trimming "test.consul" instead of just ".consul"

This change, fixes the issue by adding dot (.) before trimming

* trimDomain: ensure domain trimmed without modyfing original domains

* update changelog

---------

Co-authored-by: Alex Simenduev <shamil.si@gmail.com>
2023-07-18 10:03:41 -04:00
hc-github-team-consul-core 96bbb7821d
Backport of Fix bug with Vault CA provider into release/1.16.x (#18161)
* backport of commit 57bb6f3d72

* backport of commit b2dad88065

* backport of commit 753d3c0d3f

---------

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2023-07-17 18:46:33 +00:00
hc-github-team-consul-core 73bd9e8cc0
Backport of Re-order expected/actual for assertContainerState in consul container tests into release/1.16.x (#18158)
backport of commit d5bed27d98

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-07-17 18:29:36 +00:00
hc-github-team-consul-core fedb250230
Backport of add a conditional around setting LANFilter.AllSegments to make sure it is valid into release/1.16.x (#18146)
* no-op commit due to failed cherry-picking

* add a conditional around setting LANFilter.AllSegments to make sure it is valid (#18139)

This is to correct a code problem because this assumes all segments, but
when you get to Enterprise, you can be in partition that is not the
default partition, in which case specifying all segments does not
validate and fails. This is to correct the setting of this filter with
`AllSegments` to `true` to only occur when in the the `default`
partition.

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern

---------

Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-07-17 11:59:44 -06:00
hc-github-team-consul-core d6f57fcda1
Backport of [NET-4897] net/http host header is now verified and request.host that contains socked now error into release/1.16.x (#18145)
* no-op commit due to failed cherry-picking

* [NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129)

### Description

This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.

go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.

For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.

[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)

### Testing & Reproduction steps

Check CI tests.

### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern

---------

Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-07-17 11:44:56 -06:00
hc-github-team-consul-core 6ed2c23572
Backport of Displays Consul version of each nodes in UI nodes section into release/1.16.x (#18113)
## Backport

This PR is auto-generated from #17754 to be assessed for backporting due
to the inclusion of the label backport/1.16.


🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.

The person who merged in the original PR is:
@WenInCode
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.

> merge conflict error: unable to process merge commit:
"1c757b8a2c1160ad53421b7b8bd7f74b205c4b89", automatic backport requires
rebase workflow



The below text is copied from the body of the original PR.

---

fixes #17097 Consul version of each nodes in UI nodes section

@jkirschner-hashicorp @huikang @team @Maintainers

Updated consul version in the request to register consul.
Added this as Node MetaData.
Fetching this new metadata in UI

<img width="1512" alt="Screenshot 2023-06-15 at 4 21 33 PM"
src="https://github.com/hashicorp/consul/assets/3139634/94f7cf6b-701f-4230-b9f7-d8c4342d0737">

Also made this backward compatible and tested.

Backward compatible in this context means - If consul binary with above
PR changes is deployed to one of node, and if UI is run from this node,
then the version of not only current (upgraded) node is displayed in UI
, but also of older nodes given that they are consul servers only.
For older (non-server or client) nodes the version is not added in
NodeMeta Data and hence the version will not be displayed for them.
If a old node is consul server, the version will be displayed. As the
endpoint - "v1/internal/ui/nodes?dc=dc1" was already returning version
in service meta. This is made use of in current UI changes.

<img width="1480" alt="Screenshot 2023-06-16 at 6 58 32 PM"
src="https://github.com/hashicorp/consul/assets/3139634/257942f4-fbed-437d-a492-37849d2bec4c">




---

<details>
<summary> Overview of commits </summary>

- 931fdfc7ec -
b3e2ec1cca -
8d0e9a5490 -
04e5d88cca -
28286a2e98 -
43e50ad382 -
0cf1b7077c -
27f34ce1c2 -
2ac76d62b8 -
3d618df9ef -
1c757b8a2c -
23ce82b4ce -
4dc1c9b4c5 -
85a12a9252 -
25d30a3fa9 -
7f1d6192dc -
5174cbff84

</details>

---------

Co-authored-by: Vijay Srinivas <vijayraghav22@gmail.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2023-07-17 17:27:50 +00:00
hc-github-team-consul-core f02d214546
Backport of Change docs to say 168h instead of 7d for server_rejoin_age_max into release/1.16.x (#18156)
## Backport

This PR is auto-generated from #18154 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

Addresses
https://github.com/hashicorp/consul/pull/17171#issuecomment-1636930705
 
### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

### Links

<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

### PR Checklist

* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern


---

<details>
<summary> Overview of commits </summary>

  - f5a6411ce7 

</details>

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-17 17:12:38 +00:00
hc-github-team-consul-core f7d80b7bb3
Backport of Use JWT-auth filter in metadata mode & Delegate validation to RBAC filter into release/1.16.x (#18153)
## Backport

This PR is auto-generated from #18062 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Currently the jwt-auth filter doesn't take into account the service
identity when validating jwt-auth, it only takes into account the path
and jwt provider during validation. This causes issues when multiple
source intentions restrict access to an endpoint with different JWT
providers.
- To fix these issues, rather than use the JWT auth filter for
validation, we use it in metadata mode and allow it to forward the
successful validated JWT token payload to the RBAC filter which will
make the decisions.

This PR ensures requests with and without JWT tokens successfully go
through the jwt-authn filter. The filter however only forwards the data
for successful/valid tokens. On the RBAC filter level, we check the
payload for claims and token issuer + existing rbac rules.

### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

- This test covers a multi level jwt requirements (requirements at top
level and permissions level). It also assumes you have envoy running,
you have a redis and a sidecar proxy service registered, and have a way
to generate jwks with jwt. I mostly use:
https://www.scottbrady91.com/tools/jwt for this.

- first write your proxy defaults
```
Kind = "proxy-defaults"
name = "global"
config {
  protocol = "http"
}
```
- Create two providers 
```
Kind = "jwt-provider"
Name = "auth0"
Issuer = "https://ronald.local"

JSONWebKeySet = {
    Local = {
     JWKS = "eyJrZXlzIjog....."
    }
}
```

```
Kind = "jwt-provider"
Name = "okta"
Issuer = "https://ronald.local"

JSONWebKeySet = {
   Local = {
     JWKS = "eyJrZXlzIjogW3...."
    }
}
```

- add a service intention
```
Kind = "service-intentions"
Name = "redis"

JWT = {
  Providers = [
    {
      Name = "okta"
    },
  ]
}

Sources = [
  {
    Name = "*"
    Permissions = [{
      Action = "allow"
      HTTP = {
        PathPrefix = "/workspace"
      }
      JWT = {
        Providers = [
          {
            Name = "okta"
            VerifyClaims = [
              {
                  Path = ["aud"]
                  Value = "my_client_app"
              },
              {
                Path = ["sub"]
                Value = "5be86359073c434bad2da3932222dabe"
              }
            ]
          },
        ]
      }

    },
    {
      Action = "allow"
      HTTP = {
        PathPrefix = "/"
      }
      JWT = {
        Providers = [
          {
            Name = "auth0"
          },
        ]
      }

    }]
  }
]
```
- generate 3 jwt tokens: 1 from auth0 jwks, 1 from okta jwks with
different claims than `/workspace` expects and 1 with correct claims
- connect to your envoy (change service and address as needed) to view
logs and potential errors. You can add: `-- --log-level debug` to see
what data is being forwarded
```
consul connect envoy -sidecar-for redis1 -grpc-addr 127.0.0.1:8502
```
- Make the following requests: 
```
curl -s -H "Authorization: Bearer $Auth0_TOKEN" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_wrong_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

RBAC filter denied

curl -s -H "Authorization: Bearer $Okta_TOKEN_with_correct_claims" --insecure --cert leaf.cert --key leaf.key --cacert connect-ca.pem https://localhost:20000/workspace -v

Successful request
```


### TODO

* [x] Update test coverage
* [ ] update integration tests (follow-up PR)
* [x] appropriate backport labels added


---

<details>
<summary> Overview of commits </summary>

  - 70536f5a38 

</details>

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-07-17 15:50:21 +00:00
hc-github-team-consul-core 530744a6d3
Backport of Add ingress gateway deprecation notices to docs into release/1.16.x (#18131)
## Backport

This PR is auto-generated from #18102 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

This adds notices, that ingress gateway is deprecated, to several places
in the product docs where ingress gateway is the topic.

### Testing & Reproduction steps

Tested with a local copy of the website.

### Links

Deprecation of ingress gateway was announced in the Release Notes for
Consul 1.16 and Consul-K8s 1.2. See:

[https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul/v1_16_x#what-s-deprecated
)

[https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated](https://developer.hashicorp.com/consul/docs/release-notes/consul-k8s/v1_2_x#what-s-deprecated)

### PR Checklist

* [N/A] updated test coverage
* [X] external facing docs updated
* [X] appropriate backport labels added
* [X] not a security concern


---

<details>
<summary> Overview of commits </summary>

- 8aa89b446c -
8500ad0f7d -
4a7777930a

</details>

---------

Co-authored-by: Jeff-Apple <79924108+Jeff-Apple@users.noreply.github.com>
2023-07-14 17:29:39 +00:00
hc-github-team-consul-core 89b8ceccfd
Backport of Docs: fix unmatched bracket for health checks page into release/1.16.x (#18137)
## Backport

This PR is auto-generated from #18134 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

- Fix unmatched bracket in the
[doc](https://developer.hashicorp.com/consul/docs/services/usage/checks#ttl-check-configuration)
(see the following screenshot of the page)

<img width="618" alt="Screenshot 2023-07-13 at 9 01 19 PM"
src="https://github.com/hashicorp/consul/assets/463631/20707735-906f-4b06-999d-44e6329a9fec">


### Testing & Reproduction steps

<!--

* In the case of bugs, describe how to replicate
* If any manual tests were done, document the steps and the conditions
to replicate
* Call out any important/ relevant unit tests, e2e tests or integration
tests you have added or are adding

-->

### Links



<!--

Include any links here that might be helpful for people reviewing your
PR (Tickets, GH issues, API docs, external benchmarks, tools docs, etc).
If there are none, feel free to delete this section.

Please be mindful not to leak any customer or confidential information.
HashiCorp employees may want to use our internal URL shortener to
obfuscate links.

-->

### PR Checklist

* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern


---

<details>
<summary> Overview of commits </summary>

  - d40243b3a3 

</details>

Co-authored-by: cskh <hui.kang@hashicorp.com>
2023-07-14 14:02:16 +00:00
hc-github-team-consul-core a5c713a109
Backport of Add docs for jwt cluster configuration into release/1.16.x (#18135)
## Backport

This PR is auto-generated from #18004 to be assessed for backporting due
to the inclusion of the label backport/1.16.



The below text is copied from the body of the original PR.

---

### Description

<!-- Please describe why you're making this change, in plain English.
-->

- Add jwt-provider docs for jwks cluster configuration. The
configuration was added here:
https://github.com/hashicorp/consul/pull/17978


---

<details>
<summary> Overview of commits </summary>

  - 1ab3c3be1e 

</details>

Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
2023-07-14 11:26:38 +00:00
hc-github-team-consul-core de1bcd79e0
Backport of [NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. into release/1.16.x (#18128)
## Backport

This PR is auto-generated from #18124 to be assessed for backporting due
to the inclusion of the label backport/1.16.


🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.

The person who merged in the original PR is:
@jmurret
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.

> merge conflict error: POST
https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict
[]



The below text is copied from the body of the original PR.

---

### Description
The following jobs started failing when go 1.20.6 was released:
- `go-test-api-1-19`
- `go-test-api-1-20`
- `compatibility-integration-tests`
- `upgrade-integration-tests`

`compatibility-integration-tests` and `compatibility-integration-tests`
to this testcontainers issue:
https://github.com/testcontainers/testcontainers-go/issues/1359. This
issue calls for testcontainers to release a new version when one of
their dependencies is fixed. When that is done, we will unpin the go
versions in `compatibility-integration-tests` and
`compatibility-integration-tests`.

### Testing & Reproduction steps

See these jobs broken in CI and then see them work with this PR.


---

<details>
<summary> Overview of commits </summary>

- 747195f7aa -
516492420b -
f4d6ca19f8 -
a47407115e -
8c03b36e00 -
c50b17c46e -
7b55f66218 -
93ce5fcc61

</details>

---------

Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
2023-07-13 14:55:12 -06:00
hc-github-team-consul-core b86af2873b
backport of commit b9b58ec763 (#18110)
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-07-13 16:00:12 +00:00
hc-github-team-consul-core bbcbec99d3
backport of commit 5403b62315 (#18123)
Co-authored-by: nv-hashi <80716011+nv-hashi@users.noreply.github.com>
2023-07-12 19:44:07 -07:00
hc-github-team-consul-core feb7dad8f5
ci: build s390x (#18118)
Co-authored-by: Dan Bond <danbond@protonmail.com>
2023-07-12 17:00:22 -07:00
hc-github-team-consul-core f5e86acc91
Backport of Docs for dataplane upgrade on k8s into release/1.16.x (#18105)
* backport of commit 0d7bee8adc

* backport of commit 408cbe8ae0

* backport of commit a0854784dc

* backport of commit 71c4c6564f

* backport of commit 0c060fa2ba

---------

Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-12 17:11:46 +00:00
hc-github-team-consul-core 295ca3cb7d
Backport of ci: use gotestsum v1.10.1 [NET-4042] into release/1.16.x (#18094)
Co-authored-by: Nick Irvine <115657443+nfi-hashicorp@users.noreply.github.com>
2023-07-12 09:57:28 -07:00
hc-github-team-consul-core b26f7955b2
Backport of Passes configured role name to Vault for AWS auth in Connect CA into release/1.16.x (#18099)
* backport of commit 4034bb2b3e

* backport of commit 9c4c3c50f0

* backport of commit 7282078993

---------

Co-authored-by: Tom Davies <thomas.23.davies@bt.com>
2023-07-12 15:43:36 +00:00
hc-github-team-consul-core 9d6dff8b8d
Backport of Docs: Update proxy lifecycle annotations and consul-dataplane flags into release/1.16.x (#18095)
* backport of commit 08cd196252

* backport of commit 9fd806a458

---------

Co-authored-by: Curt Bushko <cbushko@gmail.com>
2023-07-12 03:27:38 +00:00
hc-github-team-consul-core 4ea7c7f631
Backport of docs updates - cluster peering and virtual services into release/1.16.x (#18087)
* backport of commit 3ef758cefb

* backport of commit f7c54b6ce2

* backport of commit 6b2e88c154

* backport of commit 87dc79fddb

* backport of commit 3d9805c133

* backport of commit e76ec0a193

* backport of commit 4b03ba27c1

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-11 23:11:53 +00:00
hc-github-team-consul-core 18c9a87176
Backport of Fix a couple typos in Agent Telemetry Metrics docs into release/1.16.x (#18084)
* backport of commit 58ef5fdd8f

* backport of commit 857427ae8c

---------

Co-authored-by: josh <josh.timmons@hashicorp.com>
2023-07-11 20:33:39 +00:00
hc-github-team-consul-core bc36b64a0c
backport of commit 8ad61b4e29 (#18077)
Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-07-11 15:44:44 +00:00
hc-github-team-consul-core efb87e4a68
backport of commit 38301b5ccc (#17899)
Co-authored-by: mr-miles <miles.waller@gmail.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-10 14:54:54 -07:00
Nathan Coleman 2956cc2a98
Update VERSION to reflect next patch release (#17913) 2023-07-10 21:31:39 +00:00
hc-github-team-consul-core 2ef95b62ca
Backport of [OSS] Fix initial_fetch_timeout to wait for all xDS resources into release/1.16.x (#18065)
* backport of commit 8a2f60ddae

* backport of commit e17e53c933

* backport of commit d919d55c2e

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-07-10 21:27:56 +00:00
hc-github-team-consul-core 5432554679
Backport of Fix removed service-to-service peering links into release/1.16.x (#18019)
* no-op commit due to failed cherry-picking

* Fix removed service-to-service peering links (#17221)

* docs: fix removed service-to-service peering links

* docs: extend peering-via-mesh-gateways intro (thanks @trujillo-adam)

---------

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

---------

Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: Michael Hofer <karras@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2023-07-07 15:53:45 -07:00
Thomas Eckert f3c28bda50
Update Consul on Kubernetes Helm Docs (#18054)
* Render Consul K8s Helm Docs
---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-07 21:54:51 +00:00
hc-github-team-consul-core cd780c0358
Backport of docs - add jobs use case for service mesh k8s into release/1.16.x (#18043)
* resolve conflicts
* fix backport
* address feedback

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-07 10:30:11 -07:00
hc-github-team-consul-core b3a88e845f
Backport of docs - add service sync annotations and k8s service weight annotation into release/1.16.x (#18036)
* backport of commit e1bf428494

* backport of commit ddf214e638

* backport of commit e41bd9c4e3

* backport of commit b9cfc86e14

* backport of commit 0ddf013d6c

* backport of commit 1b0b513b05

* backport of commit 29442ad641

* backport of commit 5e7ddf5c7e

* backport of commit f2b6fa7b43

* backport of commit 83b84a985a

* backport of commit 56d81738cc

* backport of commit 0ab44f06c7

* backport of commit 69c99fbccb

* backport of commit b79e1245c1

* backport of commit fb1441976b

* backport of commit 3b7b2a0424

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
2023-07-06 20:00:56 +00:00
hc-github-team-consul-core b9bb0a2cb1
backport of commit 8d547a80b8 (#18034)
Co-authored-by: trujillo-adam <ajosetru@gmail.com>
2023-07-06 17:52:31 +00:00
hc-github-team-consul-core dc112763a6
backport of commit 7f8d860bee (#18031)
Co-authored-by: trujillo-adam <ajosetru@gmail.com>
2023-07-06 16:53:39 +00:00
hc-github-team-consul-core c3c28c48ff
Add first integration test for jwt auth with intention (#18005) (#18029)
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
2023-07-06 07:54:30 -04:00