## Backport
This PR is auto-generated from #18129 to be assessed for backporting due
to the inclusion of the label backport/1.15.
🚨
>**Warning** automatic cherry-pick of commits failed. If the first
commit failed,
you will see a blank no-op commit below. If at least one commit
succeeded, you
will see the cherry-picked commits up to, _not including_, the commit
where
the merge conflict occurred.
The person who merged in the original PR is:
@jmurret
This person should manually cherry-pick the original PR into a new
backport PR,
and close this one when the manual backport PR is merged in.
> merge conflict error: POST
https://api.github.com/repos/hashicorp/consul/merges: 409 Merge conflict
[]
The below text is copied from the body of the original PR.
---
### Description
This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.
go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.
[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)
### Testing & Reproduction steps
Check CI tests.
### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
---
<details>
<summary> Overview of commits </summary>
- 747195f7aa -
516492420b -
f4d6ca19f8 -
a47407115e -
8c03b36e00 -
c50b17c46e -
cc8eaf8213 -
ce10138d07 -
133c7ecbf5 -
b0bd440d8f -
8f223080c0 -
f8578b0749 -
4452224d6a -
19634a4b3b
</details>
---------
Co-authored-by: temp <temp@hashicorp.com>
Co-authored-by: John Murret <john.murret@hashicorp.com>
This endpoint shows total services, connect service instances and
billable service instances in the local datacenter or globally. Billable
instances = total service instances - connect services - consul server instances.
* Fixing CLI ACL token processing unexpected precedence
* Minor flow format and add Changelog
* Fixed failed tests and improve error logging message
* Add unit test cases and minor changes from code review
* Unset env var once the test case finishes running
* remove label FINISH
Adds a new query param merge-central-config for use with the below endpoints:
/catalog/service/:service
/catalog/connect/:service
/health/service/:service
/health/connect/:service
If set on the request, the response will include a fully resolved service definition which is merged with the proxy-defaults/global and service-defaults/:service config entries (on-demand style). This is useful to view the full service definition for a mesh service (connect-proxy kind or gateway kind) which might not be merged before being written into the catalog (example: in case of services in the agentless model).
* add queryBackend to the api query meta.
* add a changelog
* use string type instead of int
* Apply suggestions from code review
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* add require http codes in api and use in operator_autopilot health check
* add StatusError type in api package
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
This PR updates the tags that we generate for Envoy stats.
Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
Header is: X-Consul-Default-ACL-Policy=<allow|deny>
This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
The api client should never rever to HTTP if the user explicitly
requested TLS. This change broke some tests because the tests always use
an non-TLS http server, but some tests explicitly enable TLS.
This PR adds the option to set in-memory certificates to the API client instead of requiring the certificate to be stored on disk in a file.
This allows us to define API client TLS options per Consul secret backend in Vault.
Related issue hashicorp/vault#4800
Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic).
HTTP API: GET /operator/keyring?local-only=true
CLI: consul keyring -list --local-only
Sending the local-only flag with any non-GET/list request will result in an error.
Roles are named and can express the same bundle of permissions that can
currently be assigned to a Token (lists of Policies and Service
Identities). The difference with a Role is that it not itself a bearer
token, but just another entity that can be tied to a Token.
This lets an operator potentially curate a set of smaller reusable
Policies and compose them together into reusable Roles, rather than
always exploding that same list of Policies on any Token that needs
similar permissions.
This also refactors the acl replication code to be semi-generic to avoid
3x copypasta.
Fixes: #4222
# Data Filtering
This PR will implement filtering for the following endpoints:
## Supported HTTP Endpoints
- `/agent/checks`
- `/agent/services`
- `/catalog/nodes`
- `/catalog/service/:service`
- `/catalog/connect/:service`
- `/catalog/node/:node`
- `/health/node/:node`
- `/health/checks/:service`
- `/health/service/:service`
- `/health/connect/:service`
- `/health/state/:state`
- `/internal/ui/nodes`
- `/internal/ui/services`
More can be added going forward and any endpoint which is used to list some data is a good candidate.
## Usage
When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of:
```
<selector> == <value>
<selector> != <value>
<value> in <selector>
<value> not in <selector>
<selector> contains <value>
<selector> not contains <value>
<selector> is empty
<selector> is not empty
not <other expression>
<expression 1> and <expression 2>
<expression 1> or <expression 2>
```
Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library
## Other changes
Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.
This endpoint aggregates all checks related to <service id> on the agent
and return an appropriate http code + the string describing the worst
check.
This allows to cleanly expose service status to other component, hiding
complexity of multiple checks.
This is especially useful to use consul to feed a load balancer which
would delegate health checking to consul agent.
Exposing this endpoint on the agent is necessary to avoid a hit on
consul servers and avoid decreasing resiliency (this endpoint will work
even if there is no consul leader in the cluster).
* Plumb xDS server and proxyxfg into the agent startup
* Add `consul connect envoy` command to allow running Envoy as a connect sidecar.
* Add test for help tabs; typos and style fixups from review
* Add cache types for catalog/services and health/services and basic test that caching works
* Support non-blocking cache types with Cache-Control semantics.
* Update API docs to include caching info for every endpoint.
* Comment updates per PR feedback.
* Add note on caching to the 10,000 foot view on the architecture page to make the new data path more clear.
* Document prepared query staleness quirk and force all background requests to AllowStale so we can spread service discovery load across servers.
Deniz Onur Duzgun