Commit Graph

834 Commits (3fc13681ffda5db4d4860b1343d17a14812b5efb)

Author SHA1 Message Date
hc-github-team-consul-core 606452e2fc
Backport of NET-4897 - update comment to include the current issue url from the go team. into release/1.15.x (#18265)
backport of commit 23f0cfc2d6

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-07-24 21:45:33 +00:00
hc-github-team-consul-core a799176bf1
Backport of [NET-4865] Bump golang.org/x/net to 0.12.0 into release/1.15.x (#18188)
Backport of [NET-4865] Bump golang.org/x/net to 0.12.0 into release/1.16.x (#18189)

Bump golang.org/x/net to 0.12.0

While not necessary to directly address CVE-2023-29406 (which should be
handled by using a patched version of Go when building), an
accompanying change to HTTP/2 error handling does impact agent code.

See https://go-review.googlesource.com/c/net/+/506995 for the HTTP/2
change.

Bump this dependency across our submodules as well for the sake of
potential indirect consumers of `x/net/http`.

Manual backport of 84cbf09185.
2023-07-19 16:54:33 +00:00
hc-github-team-consul-core fbfe9c435f
Backport of [NET-4897] net/http host header is now verified and request.host that contains socked now error into release/1.15.x (#18144)
## Backport

This PR is auto-generated from #18129 to be assessed for backporting due
to the inclusion of the label backport/1.15.


🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.

The person who merged in the original PR is:
@jmurret
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.

> merge conflict error: POST
https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict
[]



The below text is copied from the body of the original PR.

---

### Description

This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.

go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.

For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.

[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)

### Testing & Reproduction steps

Check CI tests.

### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern


---

<details>
<summary> Overview of commits </summary>

- 747195f7aa -
516492420b -
f4d6ca19f8 -
a47407115e -
8c03b36e00 -
c50b17c46e -
cc8eaf8213 -
ce10138d07 -
133c7ecbf5 -
b0bd440d8f -
8f223080c0 -
f8578b0749 -
4452224d6a -
19634a4b3b

</details>

---------

Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-07-17 09:36:51 -06:00
hc-github-team-consul-core 0a465dd0d2
Backport of watch: support -filter for consul watch: checks, services, nodes, service into release/1.15.x (#17986)
* backport to 1.15.x

---------

Co-authored-by: cskh <hui.kang@hashicorp.com>
2023-06-30 13:26:31 -07:00
hc-github-team-consul-core 65cc60e87f
Backport of APIGW Normalize Status Conditions into release/1.15.x (#17844)
* APIGW Normalize Status Conditions (#16994)

* normalize status conditions for gateways and routes

* Added tests for checking condition status and panic conditions for
validating combinations, added dummy code for fsm store

* get rid of unneeded gateway condition generator struct

* Remove unused file

* run go mod tidy

* Update tests, add conflicted gateway status

* put back removed status for test

* Fix linting violation, remove custom conflicted status

* Update fsm commands oss

* Fix incorrect combination of type/condition/status

* cleaning up from PR review

* Change "invalidCertificate" to be of accepted status

* Move status condition enums into api package

* Update gateways controller and generated code

* Update conditions in fsm oss tests

* run go mod tidy on consul-container module to fix linting

* Fix type for gateway endpoint test

* go mod tidy from changes to api

* go mod tidy on troubleshoot

* Fix route conflicted reason

* fix route conflict reason rename

* Fix text for gateway conflicted status

* Add valid certificate ref condition setting

* Revert change to resolved refs to be handled in future PR

* Resolve sneaky merge conflicts

---------

Co-authored-by: John Maguire <john.maguire@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-06-23 14:05:11 -04:00
hc-github-team-consul-core 6a833cd2cb
Backport of [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' into release/1.15.x (#17737)
* backport of commit 2f94024b35

* backport of commit 7626d0992d

* backport of commit 79aabc958a

* backport of commit 44eee4168b

* merge commit fa40654

* fix merge conf

* [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* nits

* updated docs

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* [NET-3865] [Supportability] Additional Information in the output of 'consul operator raft list-peers' (#17582)

* init

* fix tests

* added -detailed in docs

* added change log

* fix doc

* checking for entry in map

* fix tests

* removed detailed flag

* removed detailed flag

* revert unwanted changes

* removed unwanted changes

* updated change log

* pr review comment changes

* pr comment changes single API instead of two

* fix change log

* fix tests

* fix tests

* fix test operator raft endpoint test

* Update .changelog/17582.txt

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* nits

* updated docs

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* resolve diff

* fix merge conf

* fix merge conf

* removed detailed flag

---------

Co-authored-by: Ashesh Vidyut <ashesh.vidyut@hashicorp.com>
Co-authored-by: Ashesh Vidyut <134911583+absolutelightning@users.noreply.github.com>
Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-06-18 04:29:38 +00:00
hc-github-team-consul-core 08a106c400
Backport of [API Gateway] Fix rate limiting for API gateways into release/1.15.x (#17634)
* backport of commit fb2f3b6100

* backport of commit 178abb8495

* backport of commit 77b3998774

* backport of commit a245b326ac

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-06-09 12:43:20 +00:00
Michael Zalimeni 66f7d7b207
Manual backport 1.15.x of Disable remote proxy patching except AWS Lambda (#17432)
backport of commit ba18381ab2
2023-05-24 22:30:25 -04:00
Ashvitha 87d3f45ab5
Rename hcp-metrics-collector to consul-telemetry-collector (#17327) (#17412)
* Rename hcp-metrics-collector to consul-telemetry-collector

* Fix docs

* Fix doc comment

---------

Co-authored-by: Connor <connor.kelly@hashicorp.com>
2023-05-19 11:33:36 -04:00
malizz 734d7a27f4
backport PR 15979 to 1.15.x (#17208)
* backport PR 15979 to 1.15.x

* fix test file

* fix proto generated file

* update unit tests

* fix merge conflict

* fix test
2023-05-09 17:25:50 -07:00
Freddy 02f8ed4ca2
Backport of Allow HCP metrics collection for Envoy proxies into release/1.15.x (#16611)
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: Ashvitha <ashvitha297@gmail.com>
2023-03-10 21:50:30 +00:00
Semir Patel 4697059463 Bump consul/sdk to v0.13.1 2023-03-07 11:45:36 -06:00
hc-github-team-consul-core f51d12c952
Backport of Add ServiceResolver RequestTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable into release/1.15.x (#16520)
* backport of commit 29768f27aa

* backport of commit 1f26c2a6b4

* Add RequestTimeout field

* Add changelog entry

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-03-03 11:13:25 -05:00
hc-github-team-consul-core da95252505
backport of commit ed697687c4 (#16402)
Co-authored-by: skpratt <sarah.pratt@hashicorp.com>
2023-02-23 19:39:12 +00:00
cskh bda302b2fa
Revert "feat: envoy extension - http local rate limit (#16196)" (#16373)
This reverts commit e91bc9c058.
2023-02-22 14:59:53 -05:00
hc-github-team-consul-core fca4b563d4
Backport of [API Gateway] Various fixes for Config Entry fields into release/1.15.x (#16354)
* [API Gateway] Various fixes for Config Entry fields

* simplify logic per PR review

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-21 23:36:25 -05:00
hc-github-team-consul-core 1241135e68
Backport of [API Gateway] Validate listener name is not empty into release/1.15.x (#16341)
* backport of commit cabcb052fd

* backport of commit 89baddca25

---------

Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-02-21 19:34:04 +00:00
hc-github-team-consul-core a3a9b05e14
backport of commit 748e2a76cb (#16328)
Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-18 20:17:58 +00:00
Andrew Stucki 9ce2cff600
Backport of Add stricter validation and some normalization code for API Gateway ConfigEntries into release/1.15.x (#16317)
* Add stricter validation and some normalization code for API Gateway ConfigEntries

* Switch to certs with valid hostnames in them
2023-02-17 21:03:30 +00:00
hc-github-team-consul-core 0e89df2b7a
Backport of [OSS] security: update go to 1.20.1 into release/1.15.x (#16314)
* backport of commit 2a8cdf1747

* backport of commit 716d162048

* backport of commit 2c757fac4f

* backport of commit f81c5d4933

* backport of commit ff4f0596f3

* backport of commit 31bbca9fe4

* backport of commit 2f54191353

* backport of commit c263147180

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-17 21:00:25 +00:00
hc-github-team-consul-core 0a32425202
Backport of Fix infinite recursion in inline-certificate config entry into release/1.15.x (#16281)
* backport of commit ea4dd2bb4c

* backport of commit 637d13a0a9

* backport of commit 18f19cecd1

* backport of commit 3e6807bdbc

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-15 20:03:29 +00:00
Andrew Stucki 9bb0ecfc18
[API Gateway] Add integration test for HTTP routes (#16236)
* [API Gateway] Add integration test for conflicted TCP listeners

* [API Gateway] Update simple test to leverage intentions and multiple listeners

* Fix broken unit test

* [API Gateway] Add integration test for HTTP routes
2023-02-13 14:18:05 -05:00
skpratt db2bd404bf
Synthesize anonymous token pre-bootstrap when needed (#16200)
* add bootstrapping detail for acl errors

* error detail improvements

* update acl bootstrapping test coverage

* update namespace errors

* update test coverage

* consolidate error message code and update changelog

* synthesize anonymous token

* Update token language to distinguish Accessor and Secret ID usage (#16044)

* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* revert naming change

* add testing

* revert naming change

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-09 20:34:02 +00:00
Andrew Stucki 99cf421e7b
Add some fixes to allow for registering via consul connect envoy -gateway api (#16219)
* Add some fixes to allow for registering via consul connect envoy -gateway api

* Fix infinite recursion

---------

Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
2023-02-09 20:28:04 +00:00
skpratt 6f0b226b0d
ACL error improvements: incomplete bootstrapping and non-existent token (#16105)
* add bootstrapping detail for acl errors

* error detail improvements

* update acl bootstrapping test coverage

* update namespace errors

* update test coverage

* add changelog

* update message for unbootstrapped error

* consolidate error message code and update changelog

* logout message change
2023-02-08 23:49:44 +00:00
Kyle Havlovitz 898e59b13c
Add the `operator usage instances` command and api endpoint (#16205)
This endpoint shows total services, connect service instances and
billable service instances in the local datacenter or globally. Billable
instances = total service instances - connect services - consul server instances.
2023-02-08 12:07:21 -08:00
Andrew Stucki df03b45bbc
Add additional controller implementations (#16188)
* Add additional controller implementations

* remove additional interface

* Fix comparison checks and mark unused contexts

* Switch to time.Now().UTC()

* Add a pointer helper for shadowing loop variables

* Extract anonymous functions for readability

* clean up logging

* Add Type to the Condition proto

* Update some comments and add additional space for readability

* Address PR feedback

* Fix up dirty checks and change to pointer receiver
2023-02-08 14:50:17 -05:00
cskh e91bc9c058
feat: envoy extension - http local rate limit (#16196)
- http local rate limit
- Apply rate limit only to local_app
- unit test and integ test
2023-02-07 21:56:15 -05:00
skpratt 9199e99e21
Update token language to distinguish Accessor and Secret ID usage (#16044)
* remove legacy tokens

* remove lingering legacy token references from docs

* update language and naming for token secrets and accessor IDs

* updates all tokenID references to clarify accessorID

* remove token type references and lookup tokens by accessorID index

* remove unnecessary constants

* replace additional tokenID param names

* Add warning info for deprecated -id parameter

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* Update field comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-02-07 12:26:30 -06:00
Dhia Ayachi e42ab7e429
Remove empty tags 2 (#16113)
* Add support for RemoveEmptyTags in API client

* Add changelog

---------

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
2023-02-06 11:12:43 -08:00
skpratt a010902978
Remove legacy acl policies (#15922)
* remove legacy tokens

* remove legacy acl policies

* flatten test policies to *_prefix

* address oss feedback re: phrasing and tests
2023-02-06 15:35:52 +00:00
Derek Menteer 2f149d60cc
[OSS] Add Peer field to service-defaults upstream overrides (#15956)
* Add Peer field to service-defaults upstream overrides.

* add api changes, compat mode for service default overrides

* Fixes based on testing

---------

Co-authored-by: DanStough <dan.stough@hashicorp.com>
2023-02-03 10:51:53 -05:00
skpratt 997c44d1a7
add missing field to oss struct (#16094) 2023-01-28 03:52:52 +00:00
skpratt ad43846755
Remove legacy acl tokens (#15947)
* remove legacy tokens

* Update test comment

Co-authored-by: Paul Glass <pglass@hashicorp.com>

* fix imports

* update docs for additional CLI changes

* add test case for anonymous token

* set deprecated api fields to json ignore and fix patch errors

* update changelog to breaking-change

* fix import

* update api docs to remove legacy reference

* fix docs nav data

---------

Co-authored-by: Paul Glass <pglass@hashicorp.com>
2023-01-27 09:17:07 -06:00
Thomas Eckert 13da1a5285
Native API Gateway Config Entries (#15897)
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)

Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
2023-01-18 22:14:34 +00:00
Paul Glass f5231b9157
Add new config_file_service_registration token (#15828) 2023-01-10 10:24:02 -06:00
Thomas Eckert c708e4d5dd
Update the README for the Consul API (#15936) 2023-01-06 21:10:56 +00:00
Eric Haberkorn 8d923c1789
Add the Lua Envoy extension (#15906) 2023-01-06 12:13:40 -05:00
Florian Apolloner 077b0a48a3
Allow Operator Generated bootstrap token (#14437)
Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520
2023-01-04 20:19:33 +00:00
Dan Stough b3bd3a6586
[OSS] feat: access logs for listeners and listener filters (#15864)
* feat: access logs for listeners and listener filters

* changelog

* fix integration test
2022-12-22 15:18:15 -05:00
Nitya Dhanushkodi 24f01f96b1
add extensions for local service to GetExtensionConfigurations (#15871)
This gets the extensions information for the local service into the snapshot and ExtensionConfigurations for a proxy. It grabs the extensions from config entries and puts them in structs.NodeService.Proxy field, which already is copied into the config snapshot.

Also:
* add EnvoyExtensions to api.AgentService so that it matches structs.NodeService
2022-12-22 10:03:33 -08:00
Nitya Dhanushkodi d382ca0aec
extensions: refactor serverless plugin to use extensions from config entry fields (#15817)
docs: update config entry docs and the Lambda manual registration docs

Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
Co-authored-by: Eric <eric@haberkorn.co>
2022-12-19 12:19:37 -08:00
cskh 252a08e903
Change field to pointer so it will be parsed as nil (#15831) 2022-12-19 10:25:36 -05:00
Dan Stough 233dbcb67f
feat: add access logging API to proxy defaults (#15780) 2022-12-13 14:52:18 -05:00
cskh 04bf24c8c1
feat(ingress-gateway): support outlier detection of upstream service for ingress gateway (#15614)
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway

* changelog

Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
2022-12-13 11:51:37 -05:00
Dao Thanh Tung b890c40ce4
Fixing CLI ACL token processing unexpected precedence (#15274)
* Fixing CLI ACL token processing unexpected precedence

* Minor flow format and add Changelog

* Fixed failed tests and improve error logging message

* Add unit test cases and minor changes from code review

* Unset env var once the test case finishes running

* remove label FINISH
2022-12-02 12:19:52 -05:00
James Oulman 7e78fb7818
Add support for configuring Envoys route idle_timeout (#14340)
* Add idleTimeout

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2022-11-29 17:43:15 -05:00
Derek Menteer 8079686bf0
Add 1.14.1 release updates. (#15514)
Add post-release changes for 1.14.1 updates.
2022-11-21 13:35:30 -06:00
Derek Menteer 6fa8fa4fca
Fix issue with connect Envoy choosing incorrect TLS settings. (#15466)
This commit fixes a situation where the API TLS configuration
incorrectly influences the GRPC port TLS configuration for XDS.
2022-11-18 14:36:20 -06:00
Derek Menteer f52f3c5afc
Fix SDK to support older versions of Consul. (#15423)
This change was necessary, because the configuration was always
generated with a gRPC TLS port, which did not exist in Consul 1.13,
and would result in the server failing to launch with an error.

This code checks the version of Consul and conditionally adds the
gRPC TLS port, only if the version number is greater than 1.14.
2022-11-18 10:32:01 -06:00