* server: run the api checks against the path without params (#19205)
* Clone proto into deepcopy correctly (#19204)
* chore: update version and nightly CI for 1.17 (#19208)
Update version file to 1.18-dev, and replace 1.13 nightly test with
1.17.
* mesh: add validation hook to proxy configuration (#19186)
* mesh: add more validations to Destinations resource (#19202)
* catalog, mesh: implement missing ACL hooks (#19143)
This change adds ACL hooks to the remaining catalog and mesh resources, excluding any computed ones. Those will for now continue using the default operator:x permissions.
It refactors a lot of the common testing functions so that they can be re-used between resources.
There are also some types that we don't yet support (e.g. virtual IPs) that this change adds ACL hooks to for future-proofing.
* NET-5073 - ProxyConfiguration: implement various connection options (#19187)
* NET-5073 - ProxyConfiguration: implement various connection options
* PR feedback - LocalConnection and InboundConnection do not affect exposed routes. configure L7 route destinations. fix connection proto sequence numbers.
* add timeout to L7 Route Destinations
* Relplat 897 copywrite bot workarounds (#19200)
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
* mesh: add xRoute ACL hook tenancy tests (#19177)
Enhance the xRoute ACL hook tests to cover tenanted situations.
These tests will only execute in enterprise.
* resource: enforce lowercase v2 resource names (#19218)
* mesh: add DestinationPolicy ACL hook tenancy tests (#19178)
Enhance the DestinationPolicy ACL hook tests to cover tenanted situations.
These tests will only execute in enterprise.
* catalog: add FailoverPolicy ACL hook tenancy test (#19179)
* docs: Multi-port corrections (#19224)
* typo fixes and instruction corrections
* typo
* link path correction
* Add reason why port 53 is not used by default (#19222)
* Update dns-configuration.mdx
* Update website/content/docs/services/discovery/dns-configuration.mdx
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* v2tenancy: rename v1alpha1 -> v2beta1 (#19227)
* [NET-5944] security: Update Go version to 1.20.10 and `x/net` to 0.17.0 (#19225)
* Bump golang.org/x/net to 0.17.0
This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487).
* Update Go version to 1.20.10
This resolves [CVE-2023-39325](https://nvd.nist.gov/vuln/detail/CVE-2023-39325)
/ [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487)
(`net/http`).
* NET-6097 - sidecar proxy controller - give name to first failover policy target (#19239)
* Cc 5545: Upgrade HDS packages and modifiers (#19226)
* Upgrade @hashicorp/design-system-tokens to 1.9.0
* Upgrade @hashicorp/design-system-components to 1.8.1
* Upgrade @hashicorp/design-system-components and ember-in-viewport
* Explicitly install ember-modifier@4.1.0
* rename copy-button
* Fix how cleanup is done in with-copyable
* Update aria-menu modifier for new structure
* Update css-prop modifier to new structure
* Convert did-upsert to regular class modifier
* Update notification modifier for new structure
* Update on-oustside modifier for new structure
* Move destroy handler registration in with-copyable
* Update style modifier for new structure
* Update validate modifier for new structure
* Guard against setting on destroyed object
* Upgrade @hashicorp/design-system-components to 2.14.1
* Remove debugger
* Guard against null in aria-menu
* Fix undefined hash in validate addon
* Upgrade ember-on-resize-modifier
* Fix copy button import, missing import and array destructuring
---------
Co-authored-by: wenincode <tyler.wendlandt@hashicorp.com>
* [NET-5810] CE changes for multiple virtual hosts (#19246)
CE changes for multiple virtual hosts
* Net 4893- Ensure we're testing all the latest versions of Vault/Nomad (#19119)
* NET-5592 - update Nomad integration testing
* NET-4893: Ensure we're testing all the latest versions of Vault/Nomad
* docs: Fix example control-plane-request-limit HCL and JSON (#19105)
The control-plane-request-limit config entry does not support
specifying parameter names in snake case format.
This commit updates the HCL and JSON examples to use the supported
camel case key format.
* test: add 1.17 nightly integrations test (#19253)
* fix expose paths (#19257)
When testing adding http probes to apps, I ran into some issues which I fixed here:
- The listener should be listening on the exposed listener port, updated that.
- The listener and route names were pointing to the path of the exposed path. In my test, the path was "/" resulting in an empty string path. Also, the path may not be unique across exposed path listeners, so I decided to use the path+exposed port as the unique identifier.
* docs: Multiport HCP constraint update (#19261)
* Add sentence
* link text adjustment
* docs: Fix multi-port install (#19262)
* Update configure.mdx
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Prevent circular dependencies between v2 resources and generate a mermaid diagram with their dependencies (#19230)
* build(docker): always publish full and minor version tags for dev images (#19278)
* fix nightly integration test: envoy version and n-2 version (#19286)
* [NET-6221] Ensure LB policy set for locality-aware routing (CE) (#19283)
Ensure LB policy set for locality-aware routing (CE)
`overprovisioningFactor` should be overridden with the expected value
(100,000) when there are multiple endpoint groups. Update code and
tests to enforce this.
This is an Enterprise feature. This commit represents the CE portions of
the change; tests are added in the corresponding `consul-enterprise`
change.
* fix: allow snake case keys for ip based rate limit config entry (#19277)
* fix: allow snake case keys for ip based rate limit config entry
* chore: add changelog
* reformatted the JSON schema server conf ref (#19288)
* acls,catalog,mesh: properly authorize workload selectors on writes (#19260)
To properly enforce writes on resources that have workload selectors with prefixes, we need another service authorization rule that allows us to check whether read is allowed within a given prefix. Specifically we need to only allow writes if the policy prefix allows for a wider set of names than the prefix selector on the resource. We should also not allow policies with exact names for prefix matches.
Part of [NET-3993]
* NET-6239: Temporarily disable verify envoy check (#19299)
* skip verify envoy version
* cleanup
* Update supported Envoy versions (#19276)
* mesh: provide missing domain to route configurations in ProxyStateTemplate (#19298)
* add empty domains
* update unit tests
* enable verify envoy script (#19303)
* Vault CA bugfixes (#19285)
* Re-add retry logic to Vault token renewal
* Fix goroutine leak
* Add test for detecting goroutine leak
* Add changelog
* Rename tests
* Add comment
* Backout Envoy 1.28.0 (#19306)
* added ent to ce downgrade changes (#19311)
* added ent to ce downgrade changes
* added changelog
* added busl headers
* skip envoy version check in ci (#19315)
* Tenancy Bridge v2 (#19220)
* tenancy bridge v2 for v2 resources
* add missing copywrite headers
* remove branch name causing conflicts (#19319)
* mesh: ensure route configs are named uniquely per port (#19323)
* [NET-5327] Templated policies api/cli docs (#19270)
* More templated policies docs (#19312)
[NET-5327]More templated policies docs
* Fixing docs to add more templated policies references (#19335)
* Upgrade Consul UI to Node 18 (#19252)
* Upgrading node to node 18
* Ensure we're on latest version of yarn as well
* add comma to make frontend tests run
* Use Node 18 Alpine image in UI build dockerfile
* delete package-lock.json
---------
Co-authored-by: wenincode <tyler.wendlandt@hashicorp.com>
Co-authored-by: Ella Cai <ella.y.cai@gmail.com>
Co-authored-by: Ella Cai <ella@hashicorp.com>
* resource: default peername to local in list endpoints (#19340)
* Add grpc keepalive configuration. (#19339)
Prior to the introduction of this configuration, grpc keepalive messages were
sent after 2 hours of inactivity on the stream. This posed issues in various
scenarios where the server-side xds connection balancing was unaware that envoy
instances were uncleanly killed / force-closed, since the connections would
only be cleaned up after ~5 minutes of TCP timeouts occurred. Setting this
config to a 30 second interval with a 20 second timeout ensures that at most,
it should take up to 50 seconds for a dead xds connection to be closed.
* bump raft-wal version to 0.4.1 (#19314)
* bump raft-wal version to 0.4.1
* changelog
* go mod tidy integration tests
* go mod tidy test-integ
* NET-5397 - wire up destination golden tests from sidecar-proxy controller for xds controller and xdsv2 (#19167)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
* gvk partial inference (#19058)
* gvk partial inference
* NET-6079 - wire up sidecarproxy golden file inputs into xds controller - sources (#19241)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
* first source test works
* WIP
* modify all source files
* source tests pass
* fixing tests after bug fix in main
* NET-6080 - xds controller golden file inputs into xds resources - destinations (#19244)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
* first source test works
* WIP
* modify all source files
* source tests pass
* fixing tests after bug fix in main
* got first destination working.
* adding destinations
* fix docstring for test
* fixing tests after bug fix in main
* NET-6081 - xds controller golden file inputs into xds resources - sources (#19250)
* NET-5397 - wire up golden tests from sidecar-proxy controller for xds controller and xdsv2
* WIP
* WIP
* everything matching except leafCerts. need to mock those
* single port destinations working except mixed destinations
* golden test input to xds controller tests for destinations
* proposed fix for failover group naming errors
* clean up test to use helper.
* clean up test to use helper.
* fix test file
* add docstring for test function.
* add docstring for test function.
* fix linting error
* fixing test after route fix merged into main
* first source test works
* WIP
* modify all source files
* source tests pass
* fixing tests after bug fix in main
* got first destination working.
* adding destinations
* fix docstring for test
* fixing tests after bug fix in main
* adding source proxies
* fixing tests after bug fix in main
* got first destination working.
* adding destinations
* fix docstring for test
* fixing tests after bug fix in main
* got first destination working.
* adding destinations
* fix docstring for test
* fixing tests after bug fix in main
* Use strict DNS for mesh gateways with hostnames (#19268)
* Use strict DNS for mesh gateways with hostnames
* Add changelog
* [NET-6305] xds: Ensure v2 route match and protocol are populated for gRPC (#19343)
* xds: Ensure v2 route match is populated for gRPC
Similar to HTTP, ensure that route match config (which is required by
Envoy) is populated when default values are used.
Because the default matches generated for gRPC contain a single empty
`GRPCRouteMatch`, and that proto does not directly support prefix-based
config, an interpretation of the empty struct is needed to generate the
same output that the `HTTPRouteMatch` is explicitly configured to
provide in internal/mesh/internal/controllers/routes/generate.go.
* xds: Ensure protocol set for gRPC resources
Add explicit protocol in `ProxyStateTemplate` builders and validate it
is always set on clusters. This ensures that HTTP filters and
`http2_protocol_options` are populated in all the necessary places for
gRPC traffic and prevents future unintended omissions of non-TCP
protocols.
Co-authored-by: John Murret <john.murret@hashicorp.com>
---------
Co-authored-by: John Murret <john.murret@hashicorp.com>
* Add clarification for route match behavior (#19363)
* Add clarification for route match behavior
* Update website/content/docs/connect/config-entries/service-defaults.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Fix 1.17.x release notes and added templated policies (#19370)
* docs - release notes (add enterprise label and example of non compatible service name) (#19377)
* Update v1_17_x.mdx
* Update v1_17_x.mdx
* feat: read resource namespace (#19320)
* test: add missing tests for read resource namespace
* refactor: remove redundant test
* refactor: rename import aliases
* fix: typo var name
* refctor: remove another redundant test
* Net 5875 - Create the Exported Services Resources (#19117)
* init
* computed exported service
* make proto
* exported services resource
* exported services test
* added some tests and namespace exported service
* partition exported services
* computed service
* computed services tests
* register types
* fix comment
* make proto lint
* fix proto format make proto
* make codegen
* Update proto-public/pbmulticluster/v1alpha1/computed_exported_services.proto
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/computed_exported_services.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* using different way of resource creation in tests
* make proto
* fix computed exported services test
* fix tests
* differnet validation for computed services for ent and ce
* Acls for exported services
* added validations for enterprise features in ce
* fix error
* fix acls test
* Update internal/multicluster/internal/types/validation_exported_services_ee.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* removed the create method
* update proto
* removed namespace
* created seperate function for ce and ent
* test files updated and validations fixed
* added nil checks
* fix tests
* added comments
* removed tenancy check
* added mutation function
* fix mutation method
* fix list permissions in test
* fix pr comments
* fix tests
* lisence
* busl license
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Update internal/multicluster/internal/types/helpers_ce.go
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* make proto
* some pr comments addressed
* some pr comments addressed
* acls helper
* some comment changes
* removed unused files
* fixes
* fix function in file
* caps
* some positioing
* added test for validation error
* fix names
* made valid a function
* remvoed patch
* removed mutations
* v2 beta1
* v2beta1
* rmeoved v1alpha1
* validate error
* merge ent
* some nits
* removed dup func
* removed nil check
---------
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* test: add missing tests for list endpoint (#19364)
* Add enterprise label for rate limiting (#19384)
* test deployer: fix a bug when deploying cluster with various ent images (#19381)
* Stop use of templated-policy and templated-policy-file simultaneously (#19389)
* Resource Hook Pre-Decode Utilities (#18548)
Add some generic type hook wrappers to first decode the data
There seems to be a pattern for Validation, Mutation and Write Authorization hooks where they first need to decode the Any data before doing the domain specific work.
This PR introduces 3 new functions to generate wrappers around the other hooks to pre-decode the data into a DecodedResource and pass that in instead of the original pbresource.Resource.
This PR also updates the various catalog data types to use the new hook generators.
* resource: resource service now checks for `v2tenancy` feature flag (#19400)
* Fix casing in example yaml config (#19369)
* Delete unused files (#19402)
* NET-6294 - v1 Agentless proxycfg datasource errors after v2 changes (#19365)
* increasing unit tests timeout from 10m to 30m (#19423)
* [NET-6138] security: Bump `google.golang.org/grpc` to 1.56.3 (CVE-2023-44487) (#19414)
Bump google.golang.org/grpc to 1.56.3
This resolves [CVE-2023-44487](https://nvd.nist.gov/vuln/detail/CVE-2023-44487).
Co-authored-by: Chris Thain <chris.m.thain@gmail.com>
* Update multi-port examples to remove spec.template.metadata.name (#19430)
* integ test: snapshot mesh frozen bug test (#19435)
* integ test: snapshot mesh frozen bug test
* docs - Update k8s compat matrix (#19378)
* Update compatibility.mdx
* Update README.md (#19413)
* Fix typo in kind for JWT config on API Gateway (#19441)
* NET-5186 Add NET_BIND_SERVICE capability to consul-dataplane requirements (#18512)
* Add NET_BIND_SERVICE capability to list of consul-dataplane requirements
* Add link to related Kubernetes documentation
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* added redirect for conf entries 1.8.x (#19460)
* Update docs for service splitter example typo (#19469)
* Regen expired test certs (#19476)
* build: ensure we pull through the hashicorp proxy instead of going directly to the docker hub (#19482)
* temporarily disallow L7 traffic permissions (#19322)
* Source / local_app golden tests to include all protocols. (#19436)
* cover all protocols in local_app golden tests
* fix xds tests
* updating latest
* fix broken test
* add sorting of routers to TestBuildLocalApp to get rid of the flaking
* [NET-5916] Fix locality-aware routing config and tests (CE) (#19483)
Fix locality-aware routing config and tests
* testing/deployer: update deployer to use v2 catalog constructs when requested (#19046)
This updates the testing/deployer (aka "topology test") framework to conditionally
configure and launch catalog constructs using v2 resources. This is controlled via a
Version field on the Node construct in a topology.Config. This only functions for a
dataplane type and has other restrictions that match the rest of v2 (no peering, no
wanfed, no mesh gateways).
Like config entries, you can statically provide a set of initial resources to be synced
when bringing up the cluster (beyond those that are generated for you such as
workloads, services, etc).
If you want to author a test that can be freely converted between v1 and v2 then that
is possible. If you switch to the multi-port definition on a topology.Service (aka
"workload/instance") then that makes v1 ineligible.
This also adds a starter set of "on every PR" integration tests for single and multiport
under test-integ/catalogv2
* resource: misc finalizer apis (#19474)
* testing/deployer: support tproxy in v2 for dataplane (#19094)
This updates the testing/deployer (aka "topology test") framework to allow for a
v2-oriented topology to opt services into enabling TransparentProxy. The restrictions
are similar to that of #19046
The multiport Ports map that was added in #19046 was changed to allow for the
protocol to be specified at this time, but for now the only supported protocol is TCP
as only L4 functions currently on main.
As part of making transparent proxy work, the DNS server needed a new zonefile
for responding to virtual.consul requests, since there is no Kubernetes DNS and
the Consul DNS work for v2 has not happened yet. Once Consul DNS supports v2 we should switch over. For now the format of queries is:
<service>--<namespace>--<partition>.virtual.consul
Additionally:
- All transparent proxy enabled services are assigned a virtual ip in the 10.244.0/24
range. This is something Consul will do in v2 at a later date, likely during 1.18.
- All services with exposed ports (non-mesh) are assigned a virtual port number for use
with tproxy
- The consul-dataplane image has been made un-distroless, and gotten the necessary
tools to execute consul connect redirect-traffic before running dataplane, thus simulating
a kubernetes init container in plain docker.
* update v2 changelog (#19446)
* NET-6319 - L7 routes have statePrefix of upstream. and should have a full path (#19473)
* resource: finalizer aware delete endpoint (2 of 5) (#19493)
resource: make delete endpoint finalizer aware
* build: dependency updates for 1.17.0 (#19453)
* chore: apply enterprise changes that were missed to some testing files (#19504)
This should align between CE ef35525 and ENT 7f95226dbe40151c8f17dd4464784b60cf358dc1 in:
- testing/integration/consul-container
- test-integ
- testing/deployer
* Net-6291/fix/watch resources (#19467)
* fix: update watch endpoint to default based on scope
* test: additional test
* refactor: rename list validate function
* refactor: rename validate<Op>Request() -> ensure<Op>RequestValid() for consistency
* XDS V1 should not make runs for TCP Disco Chains. (#19496)
* XDS V1 should not make runs for TCP Disco Chains.
* update TestEnvoyExtenderWithSnapshot
* testing: disable v2 linkage to nodes in integration tests (#19509)
* Shuffle CICD tests to spread worker load. (#19501)
* [NET-6459] Fix issue with wanfed lan ip conflicts. (#19503)
Fix issue with wanfed lan ip conflicts.
Prior to this commit, the connection pools were unaware which datacenter the
connection was associated with. This meant that any time servers with
overlapping LAN IP addresses and node shortnames existed, they would be
incorrectly co-located in the same pool. Whenever this occurred, the servers
would get stuck in an infinite loop of forwarding RPCs to themselves (rather
than the intended remote DC) until they eventually run out of memory.
Most notably, this issue can occur whenever wan federation through mesh
gateways is enabled.
This fix adds extra metadata to specify which DC the connection is associated
with in the pool.
* CC-5545: Side Nav (#19342)
* Initial work for sidenav
* Use HDS::Text
* Add resolution for ember-element-helper
* WIP dc selector
* Update HCP Home link
* DC selector
* Hook up remaining selectors
* Fix settings and tutorial links
* Remove comments
* Remove skip-links
* Replace auth with new dropdown
* Use href-to helper for sidenav links
* Changelog
* Add description to NavSelector
* Wrap version in footer and role
* Fix login tests
* Add data-test selectors for namespaces
* Fix datacenter disclosure menu test
* Stop rendering auth dialog if acls are disabled
* Update disabled selector state and token selector
* Fix logic in ACL selector
* Fix HCP Home integration test
* Remove toggling the sidenav in tests
* Add sidenav to eng docs
* Re-add debug navigation for eng docs
* Remove ember-in-viewport
* Remove unused styles
* Upgrade @hashicorp/design-system-componentseee
* Add translations for side-nav
* Only show back to hcp link if url is present
* Disable responsive due to a11y-dialog issue
* fixed typos in redirect for api gateways (#19526)
* fixed typos in redirect for api gateways
* one more typo
* final typo
* [NET-5916] Update locality-aware routing docs (#19529)
* docs: Update locality-aware routing docs
- Align locality-aware routing documentation to the recommended use of the
feature and incorporate engineer feedback.
- Remove docs for unreleased multi-cluster failover feature.
- Fix minor typos and formatting in examples.
* docs: Fix rate limit docs typo
* [NET-5916] docs: Remove locality proxy startup section (#19534)
docs: Remove locality proxy startup section
This section is not necessary as it is not unique to the feature. The
instructions for starting proxies are available in other pages.
* Ci upgrade test 1 17 (#19536)
CI: upgrade test from 1.17
* added 1.17 features to enterprise overview (#19514)
* added 1.17 features to enterprise overview
* added features to runtime tables
* Apply suggestions from code review
Co-authored-by: David Yu <dyu@hashicorp.com>
* add ecs release notes
* add draft of 1.3.x consul-k8s release notes
* update nav with new release notes
* Apply suggestions from code review
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
---------
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
* Added tenancy tests for WorkloadHealth controller (#19530)
* v2tenancy: register tenancy controller deps (#19531)
* NET-6385 - Static routes that are inlined in listener filters are also created as a resource. (#19459)
* cover all protocols in local_app golden tests
* fix xds tests
* updating latest
* fix broken test
* add sorting of routers to TestBuildLocalApp to get rid of the flaking
* cover all protocols in local_app golden tests
* cover all protocols in local_app golden tests
* cover all protocols in local_app golden tests
* process envoy resource by walking the map. use a map rather than array for envoy resource to prevent duplication.
* cleanup. doc strings.
* update to latest
* fix broken test
* update tests after adding sorting of routers in local_app builder tests
* do not make endpoints for local_app
* fix catalog destinations only by creating clusters for any cluster not already created by walking the graph.
* Configure TestAllResourcesFromSnapshot to run V2 tests
* wip
* fix processing of failover groups
* add endpoints and clusters for any clusters that were not created from walking the listener -> path
* fix xds v2 golden files for clusters to include failover group clusters
* Add kubebuilder annotations to enums (#19454)
* test: fix some of the peering topology tests to safely run without tenancy in CE (#19540)
* Fix more test flakes (#19533)
Fix flaky snapshot and metrics tests.
* NET-6390 Initialize MeshGateway proto (#19548)
* ui: clear peer on home link (#19549)
Clear peer on home link
* test: update makefile to include ways to trigger deployer integration tests (#19553)
* test: update deployer default images (#19554)
* test: update certs for 10 year expiry (#19481)
* Update enterprise features table with 1.17 features (#19558)
update enterprise features table with 1.17 features
* Fix typo in GatewayClassConfig docs (#19563)
* Fix typo in GatewayClassConfig docs
* Fix broken links
* docs: Multi-port support for v1.17 GA (#19401)
* Catalog concept pages creation
* Multi-port services overview - initial update
* CLI command page creation
* File location adjustment
* nav
* New resource pages - creation
* nav fix
* resource info
* specs start
* GRPCRoute specs and structure
* GRPCRoute configuration model
* gRPCRoute models and examples
* HTTP copy
* Resource configuration alignment
* Catalogs
* Deployment error fix
* HTTPRoute specs
* TCP Route specifications
* proxy configuration model
* ProxyConfiguration specifications
* Example
* basic traffic permissions info
* complete config structure
* tab spacing
* Traffic permission specifications
* Proxy config example description
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
* v1 concept review updates
* v2 catalog code review updates
* V2 catalog contraints + guidance
* Proxyconfiguration code review changes
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Cross-apply suggestions to reference pages
* More code review suggestions
* comment fix
* Apply suggestions from code review
* Index + usage updates
* TCP clarification
* Minor fixes
* remove references to unsupported features
* comment fix
* Resource command section removed
* Tested instructions
* More updates based on testing
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Multi-port traffic permissions example
* parent/child alignment
* Dataplanes requirement
* Update version-specific CLI install
* Method 2
* Tab fix
* Consul resource updates
* nav fix
* Catalog groups
* Catalog `group` info
* time formatting
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
* test/deployer: add the method of deregistering services (#19525)
* add DeliverLatest as common function for use by Manager and ProxyTracker Open (#19564)
Open
add DeliverLatest as common function for use by Manager and ProxyTracker
* Migrate individual resource tests for Discovery Chains to TestAllResourcesFromSnapshot (#19508)
migrate disco chain tests to resources_test.go
* NET 6354 - Add tenancy in Node Health Controller (#19457)
* node health controller tenancy
* some prog
* some fixes
* revert
* pr comment resolved
* removed name
* cleanup nodes
* some fixes
* merge main
* Migrate individual resource tests for expose paths and checks to TestAllResourcesFromSnapshot (#19513)
* migrate expose checks and paths tests to resources_test.go
* fix failing expose paths tests
* Introduce randomized timings and reproducible randomization into controller integration tests. (#19393)
As the V2 architecture hinges on eventual consistency and controllers reconciling the existing state in response to writes, there are potential issues we could run into regarding ordering and timing of operations. We want to be able to guarantee that given a set of resources the system will always eventually get to the desired correct state. The order of resource writes and delays in performing those writes should not alter the final outcome of reaching the desired state.
To that end, this commit introduces arbitrary randomized delays before performing resources writes into the `resourcetest.Client`. Its `PublishResources` method was already randomizing the order of resource writes. By default, no delay is added to normal writes and deletes but tests can opt-in via either passing hard coded options when creating the `resourcetest.Client` or using the `resourcetest.ConfigureTestCLIFlags` function to allow processing of CLI parameters.
In addition to allowing configurability of the request delay min and max, the client also has a configurable random number generator seed. When Using the CLI parameter helpers, a test log will be written noting the currently used settings. If the test fails then you can reproduce the same delays and order randomizations by providing the seed during the previous test failure.
* Migrate individual resource tests for custom configuration to TestAllResourcesFromSnapshot (#19512)
* Configure TestAllResourcesFromSnapshot to run V2 tests
* migrate custom configuration tests to resources_test.go
* Update Helm docs for consul-k8s 1.3.0 (#19581)
* Migrate individual resource tests for JWT Provider to TestAllResourcesFromSnapshot (#19511)
migrate jwt provider tests to resources_test.go
* Fix xds v2 from creating envoy endpoint resources when already inlined in the cluster (#19580)
* migrate expose checks and paths tests to resources_test.go
* fix failing expose paths tests
* fix the way endpoint resources get created to make expose tests pass.
* wip
* remove endpoint resources that are already inlined on local_app clusters
* renaiming and comments
* test: add a v2 container integration test of xRoute splits (#19570)
This adds a deployer-based integration test verifying that a 90/10 traffic
split works for: HTTPRoute, GRPCRoute, and TCPRoute.
* docs: spike of info about how to use deployer topology tests (#19576)
Initial developer facing docs on how to write deployer (topology) integration tests.
* Net 6439 (#19517)
* node health controller tenancy
* some prog
* some fixes
* revert
* pr comment resolved
* removed name
* Add namespace and tenancy in sidecar proxy controller test
* revert node health controller
* clean up data
* fix local
* copy from ENT
* removed dup code
* removed tenancy
* add test tenancies
* Migrate individual resource tests for Terminating Gateway to TestAllResourcesFromSnapshot (#19505)
migrate terminating-gateway tests to resources_test.go
* Migrate individual resource tests for Ingress Gateway to TestAllResourcesFromSnapshot (#19506)
migrate ingress-gateway tests to resources_test.go
* Update links and fix route kind for APIGW JWT Docs (#19585)
Update links and fix route kind
* DNS token doc updates
Review feedback
Update website/content/commands/acl/set-agent-token.mdx
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Update dns token doc to link to policy templates
* backport of commit 1562cb9749
* backport of commit 400620e10c
---------
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: modrake <12264057+modrake@users.noreply.github.com>
Co-authored-by: Ronald Ekambi <ronekambi@gmail.com>
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Chris Hut <tophernuts@gmail.com>
Co-authored-by: wenincode <tyler.wendlandt@hashicorp.com>
Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Sophie Gairo <97480023+sophie-gairo@users.noreply.github.com>
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
Co-authored-by: cskh <hui.kang@hashicorp.com>
Co-authored-by: Poonam Jadhav <poonam.jadhav@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Anita Akaeze <anita.akaeze@hashicorp.com>
Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: aahel <aahel.guha@hashicorp.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
Co-authored-by: Ella Cai <ella.y.cai@gmail.com>
Co-authored-by: Ella Cai <ella@hashicorp.com>
Co-authored-by: Derek Menteer <105233703+hashi-derek@users.noreply.github.com>
Co-authored-by: wangxinyi7 <121973291+wangxinyi7@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Chris Thain <chris.m.thain@gmail.com>
Co-authored-by: skpratt <sarah.pratt@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com>
Co-authored-by: Ganesh S <ganesh.seetharaman@hashicorp.com>
Co-authored-by: John Landa <john.landa@hashicorp.com>
Add grpc keepalive configuration. (#19339)
Prior to the introduction of this configuration, grpc keepalive messages were
sent after 2 hours of inactivity on the stream. This posed issues in various
scenarios where the server-side xds connection balancing was unaware that envoy
instances were uncleanly killed / force-closed, since the connections would
only be cleaned up after ~5 minutes of TCP timeouts occurred. Setting this
config to a 30 second interval with a 20 second timeout ensures that at most,
it should take up to 50 seconds for a dead xds connection to be closed.
* updated nav; renamed L7 traffic folder
* Added locality-aware routing to traffic mgmt overview
* Added route to local upstreams topic
* Updated agent configuration reference
* Added locality param to services conf ref
* Added locality param to conf entries
* mentioned traffic management in proxies overview
* added locality-aware to failover overview
* added docs for service rate limiting
* updated service defaults conf entry
* Apply suggestions from code review
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* updated links and added redirects
---------
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* first commit; reformat PD conf entry
* updated proxies overview page
* added Deploy SM proxy usage and removed reg index
* moved sidecar proxy usage to main proxy folder
* recast sidecar reg page as Deploy sidecar services
* fix typos
* recast SM reg as conf reference- set the sidebar
* add redirects
* fix links
* add PD conf entry usage to appropro pages
* edits to proxy conf ref
* fix links on index page
* example command to write PD conf entry
* updated links to old SM proxy reg page
* updated links to sidecar service reg page
* tryna fix front matter issues
* Apply suggestions from code review
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
* added paragraph about SM proxies to overivew
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Ronald <roncodingenthusiast@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* fix broken link caught in weekly report
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* agent: configure server lastseen timestamp
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use correct config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use default age in test golden data
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add changelog
Signed-off-by: Dan Bond <danbond@protonmail.com>
* fix runtime test
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add server_metadata
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* correctly check if metadata file does not exist
Signed-off-by: Dan Bond <danbond@protonmail.com>
* follow instructions for adding new config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* Update agent/agent.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* agent/config: add validation for duration with min
Signed-off-by: Dan Bond <danbond@protonmail.com>
* docs: add new server_rejoin_age_max config definition
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add unit test for checking server last seen
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: log continually for 60s before erroring
Signed-off-by: Dan Bond <danbond@protonmail.com>
* pr comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* remove unneeded todo
* agent: fix error message
Signed-off-by: Dan Bond <danbond@protonmail.com>
---------
Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
Remove outdated usage of "Consul Connect" instead of Consul service mesh.
The connect subsystem in Consul provides Consul's service mesh capabilities.
However, the term "Consul Connect" should not be used as an alternative to
the name "Consul service mesh".
* Fix broken links in Consul docs
* more broken link fixes
* more 404 fixes
* 404 fixes
* broken link fix
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* fixed cross-links between new topics
* updated links to the new services pages
* fixed bad links in scale file
* tweaks to titles and phrasing
* fixed typo in checks.mdx
* started updating the conf ref to latest template
* update SD conf ref to match latest CT standard
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* remove previous version of the checks page
* fixed cross-links
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
---------
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* Add WAL documentation. Also fix some minor metrics registration details
* Add tests to verify metrics are registered correctly
* refactor and move wal docs
* Updates to the WAL overview page
* updates to enable WAL usage topic
* updates to the monitoring WAL backend topic
* updates for revert WAL topic
* a few tweaks to overview and udpated metadescriptions
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* make revert docs consistent with enable
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* address feedback
* address final feedback
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: trujillo-adam <ajosetru@gmail.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* added a NS folder and refactored main page into the overview page
* added NS usage page to NS folder
* updated links to NS docs
* updated nav
* addressed feedback from review
* server: add placeholder glue for rate limit handler
This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.
This allows us to start working on the net/rpc and gRPC interceptors and
config logic.
* Add handler errors
* Set the global read and write limits
* fixing multilimiter moving packages
* Fix typo
* Simplify globalLimit usage
* add multilimiter and tests
* exporting LimitedEntity
* Apply suggestions from code review
Co-authored-by: John Murret <john.murret@hashicorp.com>
* add config update and rename config params
* add doc string and split config
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* use timer to avoid go routine leak and change the interface
* add comments to tests
* fix failing test
* add prefix with config edge, refactor tests
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* refactor to apply configs for limiters under a prefix
* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic
* make KeyType an exported type
* split the config and limiter trees to fix race conditions in config update
* rename variables
* fix race in test and remove dead code
* fix reconcile loop to not create a timer on each loop
* add extra benchmark tests and fix tests
* fix benchmark test to pass value to func
* server: add placeholder glue for rate limit handler
This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.
This allows us to start working on the net/rpc and gRPC interceptors and
config logic.
* Set the global read and write limits
* fixing multilimiter moving packages
* add server configuration for global rate limiting.
* remove agent test
* remove added stuff from handler
* remove added stuff from multilimiter
* removing unnecessary TODOs
* Removing TODO comment from handler
* adding in defaulting to infinite
* add disabled status in there
* adding in documentation for disabled mode.
* make disabled the default.
* Add mock and agent test
* addig documentation and missing mock file.
* Fixing test TestLoad_IntegrationWithFlags
* updating docs based on PR feedback.
* Updating Request Limits mode to use int based on PR feedback.
* Adding RequestLimits struct so we have a nested struct in ReloadableConfig.
* fixing linting references
* Update agent/consul/rate/handler.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* Update agent/consul/config.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* removing the ignore of the request limits in JSON. addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf
* added conversion function to convert request limits object to handler config.
* Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well.
* Updating values for TestLoad_FullConfig() so that they were different and discernable.
* Updating TestRuntimeConfig_Sanitize
* Fixing TestLoad_IntegrationWithFlags test
* putting nil check in place
* fixing rebase
* removing change for missing error checks. will put in another PR
* Rebasing after default multilimiter config change
* resolving rebase issues
* updating reference for incomingRPCLimiter to use interface
* updating interface
* Updating interfaces
* Fixing mock reference
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
* auto-config: relax node name validation for JWT authorization
This changes the JWT authorization logic to allow all non-whitespace,
non-quote characters when validating node names. Consul had previously
allowed these characters in node names, until this validation was added
to fix a security vulnerability with whitespace/quotes being passed to
the `bexpr` library. This unintentionally broke node names with
characters like `.` which aren't related to this vulnerability.
* Update website/content/docs/agent/config/cli-flags.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Prevent serving TLS via ports.grpc
We remove the ability to run the ports.grpc in TLS mode to avoid
confusion and to simplify configuration. This breaking change
ensures that any user currently using ports.grpc in an encrypted
mode will receive an error message indicating that ports.grpc_tls
must be explicitly used.
The suggested action for these users is to simply swap their ports.grpc
to ports.grpc_tls in the configuration file. If both ports are defined,
or if the user has not configured TLS for grpc, then the error message
will not be printed.
Fix an issue where rpc_hold_timeout was being used as the timeout for non-blocking queries. Users should be able to tune read timeouts without fiddling with rpc_hold_timeout. A new configuration `rpc_read_timeout` is created.
Refactor some implementation from the original PR 11500 to remove the misleading linkage between RPCInfo's timeout (used to retry in case of certain modes of failures) and the client RPC timeouts.
Adds a user-configurable rate limiter to proxycfg snapshot delivery,
with a default limit of 250 updates per second.
This addresses a problem observed in our load testing of Consul
Dataplane where updating a "global" resource such as a wildcard
intention or the proxy-defaults config entry could starve the Raft or
Memberlist goroutines of CPU time, causing general cluster instability.
hc-github-team-consul-core