Commit Graph

5519 Commits (ent-changelog-1.19.4)

Author SHA1 Message Date
hc-github-team-consul-core 18354fc302
Backport of [Security] Fix XSS Vulnerability where content-type header wasn't explicitly set into release/1.19.x (#21709)
* backport of commit 52f4b86c5c

* backport of commit ede97520b0

* backport of commit 4446c25617

* backport of commit 957301e092

* backport of commit 55c0ece134

---------

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
2024-09-13 11:22:02 -04:00
hc-github-team-consul-core cffdc98198
Backport of [NET-10952] fix cluster dns lookup family to gracefully handle ipv6 into release/1.19.x (#21720)
* backport of commit a03603a6e4

* backport of commit e70dfacad3

* backport of commit fd58a6bbb5

---------

Co-authored-by: jm96441n <john.maguire@hashicorp.com>
2024-09-12 12:02:10 -04:00
hc-github-team-consul-core e9573e3b6f
Backport of Set replication metric to 0 when losing leadership into release/1.19.x (#21687)
* backport of commit d6e0bffbce

* backport of commit 74a6463761

---------

Co-authored-by: Jorge Marey <jorgenw3@gmail.com>
2024-08-29 17:39:58 +00:00
hc-github-team-consul-core b996f99899
Backport of fix: use Envoy's default for validate_clusters to fix breaking routes when some backend clusters don't exist into release/1.19.x (#21621)
---------

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
2024-08-19 23:05:43 -07:00
hc-github-team-consul-core 45abe96d0e
Backport of Fix TestDNS_ServiceLookup_ARecordLimits so that it only creates test agents the minimal amount of time into release/1.19.x (#21612)
* backport of commit 0ca8552d02

* backport of commit ff2713b238

* backport of commit f8c8d0ac13

* backport of commit da2c4499f5

* backport of commit 38e824afb5

* backport of commit 8cf76bd353

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2024-08-15 18:44:19 +00:00
hc-github-team-consul-core b1a65de811
Backport of fix where jwt clusters are generated into release/1.19.x (#21607)
backport of commit 01e9abcddb

Co-authored-by: jm96441n <john.maguire@hashicorp.com>
2024-08-14 20:24:05 +00:00
hc-github-team-consul-core 3e4a572907
Backport of [NET-10719] Fix cluster generation for jwt clusters for external jwt providers into release/1.19.x (#21605)
* backport of commit 62264e4043

* backport of commit b63aa1b8b4

---------

Co-authored-by: jm96441n <john.maguire@hashicorp.com>
2024-08-14 16:04:54 -04:00
hc-github-team-consul-core 81b4be90da
Backport of NET-10685 - Remove dns v2 code into release/1.19.x (#21599)
* backport of commit 9ce9d3c5b0

* backport of commit e75b8148ab

* backport of commit 6f026f18f0

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2024-08-13 18:12:52 -06:00
hc-github-team-consul-core 2225814a4d
Backport of NET-10610 - stop logging no data as errors in DNS lookups into release/1.19.x (#21583)
backport of commit 0493b1e6f5

Co-authored-by: John Murret <john.murret@hashicorp.com>
2024-08-01 18:40:02 +00:00
hc-github-team-consul-core f050396ca9
Backport of [NET-10246] use correct enterprise meta for service name for LinkedService into release/1.19.x (#21533)
* backport of commit f5cec2e623

* backport of commit 8e63265072

---------

Co-authored-by: jm96441n <john.maguire@hashicorp.com>
2024-07-10 15:16:11 +00:00
hc-github-team-consul-core e1dc9ed5a0
Backport of fix(dns): spam ttl logs for prepared queries into release/1.19.x (#21522)
backport of commit 1c7c5be5b0

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2024-07-08 11:07:34 -04:00
hc-github-team-consul-core ad6931c4a8
Backport of fix(txn): validate verbs into release/1.19.x (#21520)
* backport of commit 5807c2c5e1

* backport of commit 8ef60b8add

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2024-07-05 15:18:22 -04:00
hc-github-team-consul-core 1e3ea57e0a
Backport of security: fix AliasCheck panic (update) into release/1.19.x (#21511)
backport of commit 8c3682afd3

Co-authored-by: Kiran Naidoo <kiran@kiran.za.org>
2024-07-03 15:11:59 +00:00
hc-github-team-consul-core 2221e34855
Backport of fix(dns): bug with standard lookup tags not working; SRV questions returning duplicate hostnames into release/1.19.x (#21369)
backport of commit 17425bd174

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2024-06-25 15:36:19 -04:00
hc-github-team-consul-core a1ee7cb38b
Backport of [Security] Close cross scripting vulnerability into release/1.19.x (#21346)
* backport of commit c8cb3349fe

* backport of commit 262efd8f15

---------

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
2024-06-17 16:05:02 -05:00
hc-github-team-consul-core 32960388b2
Backport of security: fix AliasCheck panic into release/1.19.x (#21341)
* backport of commit aeba4bc804

* backport of commit 8c5b157f7b

---------

Co-authored-by: dduzgun-security <deniz.duzgun@hashicorp.com>
2024-06-14 11:48:04 -04:00
hc-github-team-consul-core 421e4b5384
Backport of Use text/template instead of html/template for ACL template policy generation into release/1.19.x (#21306)
backport of commit fa396d0bed

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-06-11 16:56:19 +00:00
hc-github-team-consul-core b1ebefce7d
Backport of security: resolve incorrect type conversions into release/1.19.x (#21257)
* backport of commit 107516c971

* backport of commit 7a2ea4bf92

* backport of commit 442dca74e2

---------

Co-authored-by: dduzgun-security <deniz.duzgun@hashicorp.com>
2024-06-04 22:30:36 -04:00
hc-github-team-consul-core 3c4834a942
Backport of update TestHTTPHandlers_AgentMetrics_LeaderShipMetrics to use 3 servers instead of 2 to allow quorum when leadership flails. into release/1.19.x (#21249)
* backport of commit 5dead1a10e

* backport of commit bcc8abf904

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2024-06-03 12:47:44 -06:00
hc-github-team-consul-core 408ed18246
Backport of dns v2 - both empty string and default should be allowed for namespace and partition in CE into release/1.19.x (#21233)
* backport of commit 8513eda629

* backport of commit 329bdc1345

* backport of commit 0f5d0adebd

* backport of commit 8a1d017999

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2024-05-28 22:39:54 +00:00
Dhia Ayachi 1f4caaedf2
upgrade deep-copy version, upgrade go to 1.22.3 (#21113)
* upgrade deep-copy version, upgrade go to 1.22.3

* add changelog
2024-05-16 13:40:15 -04:00
John Murret 9b9c836915
latest ui files in main (#21119) 2024-05-15 23:52:11 +00:00
John Murret 04940e2c78
additional changes to ensure sameness groups without DefaultForFailover can be used for DNS (#21107) 2024-05-14 15:33:34 -06:00
John Murret 9b2c1be053
NET-5879 - expose sameness group param on service health endpoint and move sameness group health fallback logic into HealthService RPC layer (#21096)
* NET-5879 - move the filter for non-passing to occur in the health RPC layer rather than the callers of the RPC

* fix import of slices

* NET-5879 - expose sameness group param on service health endpoint and move sameness group health fallback logic into HealthService RPC layer

* fixing deepcopy

* fix license headers
2024-05-14 13:32:49 +00:00
John Murret a975b04302
NET-5879 - move the filter for non-passing to occur in the health RPC layer rather than the callers of the RPC (#21098)
* NET-5879 - move the filter for non-passing to occur in the health RPC layer rather than the callers of the RPC

* fix import of slices

* fix test
2024-05-14 07:05:54 -06:00
John Murret 17df32e5cb
NET-9084 - add tests to peering endpoint and blockingquery package to assert blocking works properly. (#21078) 2024-05-09 14:55:13 -04:00
R.B. Boyer 1535844c62
gossip: refactor some gossip related libraries into a central place (#21036)
This refactors and relocates the following packages to live under internal/gossip instead of either in the toplevel lib or agent/consul:

- librtt : related to serf coordinates
- libserf : random serf stuff
2024-05-07 10:30:49 -05:00
Nathan Coleman b5b3a63183
[NET-9098] Narrow scope of peering config on terminating gw filter chain to TCP services (#21054) 2024-05-06 16:21:09 -04:00
Dan Stough 03ab7367a6
feat(dataplane): allow token and tenancy information for proxied DNS (#20899)
* feat(dataplane): allow token and tenancy information for proxied DNS

* changelog
2024-04-22 14:30:43 -04:00
sarahalsmiller 08761f16c8
Net 6820 customize mesh gateway limits (#20945)
* add upstream limits to mesh gateway cluster generation

* changelog

* go mod tidy

* readd changelog data

* undo reversion from rebase

* run codegen

* Update .changelog/20945.txt

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>

* address notes

* gofmt

* clean up

* gofmt

* Update agent/proxycfg/mesh_gateway.go

* gofmt

* nil check

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2024-04-16 10:59:41 -05:00
Nathan Coleman 5e9f02d4be
[NET-8091] Add file-system-certificate config entry for API gateway (#20873)
* Define file-system-certificate config entry

* Collect file-system-certificate(s) referenced by api-gateway onto snapshot

* Add file-system-certificate to config entry kind allow lists

* Remove inapplicable validation

This validation makes sense for inline certificates since Consul server is holding the certificate; however, for file system certificates, Consul server never actually sees the certificate.

* Support file-system-certificate as source for listener TLS certificate

* Add more required mappings for the new config entry type

* Construct proper TLS context based on certificate kind

* Add support or SDS in xdscommon

* Remove unused param

* Adds back verification of certs for inline-certificates

* Undo tangential changes to TLS config consumption

* Remove stray curly braces

* Undo some more tangential changes

* Improve function name for generating API gateway secrets

* Add changelog entry

* Update .changelog/20873.txt

Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>

* Add some nil-checking, remove outdated TODO

* Update test assertions to include file-system-certificate

* Add documentation for file-system-certificate config entry

Add new doc to nav

* Fix grammar mistake

* Rename watchmaps, remove outdated TODO

---------

Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
2024-04-15 16:45:05 -04:00
Michael Zalimeni a8d08e759f
fix: consume ignored entries in CE downgrade via Ent snapshot (#20977)
This operation would previously fail due to unconsumed bytes in the
decoder buffer when reading the Ent snapshot (the first byte of the
record would be misinterpreted as a type indicator, and the remaining
bytes would fail to be deserialized or read as invalid data).

Ensure restore succeeds by decoding the ignored record as an
interface{}, which will consume the record bytes without requiring a
concrete target struct, then moving on to the next record.
2024-04-11 21:08:44 +00:00
Eric Haberkorn e231f0ee9b
Add an agent config option to diable per tenancy usage metrics. (#20976) 2024-04-11 15:20:09 -04:00
John Murret d261a987f1
update go-control-plane envoy dependency to 0.12.0 (#20973)
* update go-control-plane envoy dependency to 0.12.0

* add changelog

* go mod tidy

* fix linting issues

* add agent/grpc-internal to the list of SA1019 ignores
2024-04-10 01:23:04 +00:00
Nathan Coleman 9af713ff17
[NET-5772] Make tcp external service registered on terminating gw reachable from peered cluster (#19881)
* Include SNI + root PEMs from peered cluster on terminating gw filter chain

This allows an external service registered on a terminating gateway to be exported to and reachable from a peered cluster

* Abstract existing logic into re-usable function

* Regenerate golden files w/ new listener logic

* Add changelog entry

* Use peering bundles that are stable across test runs
2024-04-03 12:38:09 -04:00
George Ma 44facc2ea3
chore: remove repetitive words (#20890)
Signed-off-by: availhang <mayangang@outlook.com>
2024-03-28 16:31:55 -07:00
John Murret 39112c7a98
GH-20889 - put conditionals are hcp initialization for consul server (#20926)
* put conditionals are hcp initialization for consul server

* put more things behind configuration flags

* add changelog

* TestServer_hcpManager

* fix TestAgent_scadaProvider
2024-03-28 14:47:11 -06:00
Dan Stough 6026ada0c9
[CE] feat(v2dns): enable v2 dns as default (#20715)
* feat(v2dns): enable v2 dns as default

* changelog
2024-03-25 16:09:01 -04:00
Iryna Shustava d747b51dab
Handle ACL errors consistently when blocking query timeout is reached. (#20876)
Currently, when a client starts a blocking query and an ACL token expires within
that time, Consul will return ACL not found error with a 403 status code. However,
sometimes if an ACL token is invalidated at the same time as the query's deadline is reached,
Consul will instead return an empty response with a 200 status code.

This is because of the events being executed.
1. Client issues a blocking query request with timeout `t`.
2. ACL is deleted.
3. Server detects a change in ACLs and force closes the gRPC stream.
4. Client resubscribes with the same token and resets its state (view).
5. Client sees "ACL not found" error.

If ACL is deleted before step 4, the client is unaware that the stream was closed due to
an ACL error and will return an empty view (from the reset state) with the 200 status code.

To fix this problem, we introduce another state to the subsciption to indicate when a change
to ACLs has occured. If the server sees that there was an error due to ACL change, it will
re-authenticate the request and return an error if the token is no longer valid.

Fixes #20790
2024-03-22 14:59:54 -06:00
Chris S. Kim f3f2175edd
Update go-jose library (#20888) 2024-03-22 10:54:58 -04:00
Derek Menteer ac83ac1343
Fix streaming RPCs for agentless. (#20868)
* Fix streaming RPCs for agentless.

This PR fixes an issue where cross-dc RPCs were unable to utilize
the streaming backend due to having the node name set. The result
of this was the agent-cache being utilized, which would cause high
cpu utilization and memory consumption due to the fact that it
keeps queries alive for 72 hours before purging inactive entries.

This resource consumption is compounded by the fact that each pod
in consul-k8s gets a unique token. Since the agent-cache uses the
token as a component of the key, the same query is duplicated for
each pod that is deployed.

* Add changelog.
2024-03-15 14:44:51 -05:00
Derek Menteer 0ac8ae6c3b
Fix xDS deadlock due to syncLoop termination. (#20867)
* Fix xDS deadlock due to syncLoop termination.

This fixes an issue where agentless xDS streams can deadlock permanently until
a server is restarted. When this issue occurs, no new proxies are able to
successfully connect to the server.

Effectively, the trigger for this deadlock stems from the following return
statement:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L199-L202

When this happens, the entire `syncLoop()` terminates and stops consuming from
the following channel:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L182-L192

Which results in the `ConfigSource.cleanup()` function never receiving a
response and holding a mutex indefinitely:
https://github.com/hashicorp/consul/blob/v1.18.0/agent/proxycfg-sources/catalog/config_source.go#L241-L247

Because this mutex is shared, it effectively deadlocks the server's ability to
process new xDS streams.

----

The fix to this issue involves removing the `chan chan struct{}` used like an
RPC-over-channels pattern and replacing it with two distinct channels:

+ `stopSyncLoopCh` - indicates that the `syncLoop()` should terminate soon.  +
`syncLoopDoneCh` - indicates that the `syncLoop()` has terminated.

Splitting these two concepts out and deferring a `close(syncLoopDoneCh)` in the
`syncLoop()` function ensures that the deadlock above should no longer occur.

We also now evict xDS connections of all proxies for the corresponding
`syncLoop()` whenever it encounters an irrecoverable error. This is done by
hoisting the new `syncLoopDoneCh` upwards so that it's visible to the xDS delta
processing. Prior to this fix, the behavior was to simply orphan them so they
would never receive catalog-registration or service-defaults updates.

* Add changelog.
2024-03-15 13:57:11 -05:00
Derek Menteer eabff257d7
Various bug-fixes and improvements (#20866)
* Shuffle the list of servers returned by `pbserverdiscovery.WatchServers`.

This randomizes the list of servers to help reduce the chance of clients
all connecting to the same server simultaneously. Consul-dataplane is one
such client that does not randomize its own list of servers.

* Fix potential goroutine leak in xDS recv loop.

This commit ensures that the goroutine which receives xDS messages from
proxies will not block forever if the stream's context is cancelled but
the `processDelta()` function never consumes the message (due to being
terminated).

* Add changelog.
2024-03-15 13:10:48 -05:00
sarahalsmiller 262f435800
NET-6821 Disable Terminating Gateway Auto Host Header Rewrite (#20802)
* disable terminating gateway auto host rewrite

* add changelog

* clean up unneeded additional snapshot fields

* add new field to docs

* squash

* fix test
2024-03-12 15:37:20 -05:00
Michael Zalimeni d4761c0ccd
security: upgrade google.golang.org/protobuf to 1.33.0 (#20801)
Resolves CVE-2024-24786.
2024-03-06 23:04:42 +00:00
Matt Keeler abe14f11e6
Remove redundant usage metrics (#20674)
* Remove redundant usage metrics

* Add the changelog

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-specific.mdx

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

---------

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2024-03-05 14:09:47 -05:00
Matt Keeler 5c936fba33
Enable callers to control whether per-tenant usage metrics are included in calls to store.ServiceUsage (#20672)
* Enable callers to control whether per-tenant usage metrics are included in calls to store.ServiceUsage

* Add changelog
2024-03-01 13:44:55 -05:00
John Murret a1c6181677
DNS v2 - split up router into multiple responsibilities & break up router tests into multiple files. (#20688)
* Update agent/dns.go

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>

* PR feedback

* split tests out into multiple files.

* Extract responsibilities from router into discoveryResultsFetcher, messageSerializer, responseGenerator.

* adding recordmaker tests

* add response generator test coverage.

* changing tests case name based on PR feedback

---------

Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>
2024-03-01 15:36:37 +00:00
John Murret a15a957a36
NET-8056 - v2 DNS Testing Improvements (#20710)
* NET-8056 - v2 DNS Testing Improvements

* adding TestDNSServer_Lifecycle

* add license headers to new files.
2024-03-01 05:42:42 -07:00
sarahalsmiller 670ee90a77
Use correct enterprise meta on wildcard service update (#20721)
* use correct enterprise meta on wildcard service update

* changelog

* rename changelog file
2024-02-26 12:03:08 -06:00