diff --git a/.changelog/15654.txt b/.changelog/15654.txt new file mode 100644 index 0000000000..169191377b --- /dev/null +++ b/.changelog/15654.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: Adds new command - `consul services export` - for exporting a service to a peer or partition +``` diff --git a/.changelog/16845.txt b/.changelog/16845.txt new file mode 100644 index 0000000000..7181e319e3 --- /dev/null +++ b/.changelog/16845.txt @@ -0,0 +1,3 @@ +```release-note:improvement +systemd: set service type to notify. +``` diff --git a/.changelog/17038.txt b/.changelog/17038.txt new file mode 100644 index 0000000000..b3a47f98a7 --- /dev/null +++ b/.changelog/17038.txt @@ -0,0 +1,3 @@ +```release-note:improvement +agent: add new metrics to track cpu disk and memory usage for server hosts (defaults to: enabled) +``` diff --git a/.changelog/17055.txt b/.changelog/17055.txt new file mode 100644 index 0000000000..9300c41121 --- /dev/null +++ b/.changelog/17055.txt @@ -0,0 +1,3 @@ +```release-note:bug +gateways: Fix an bug where targeting a virtual service defined by a service-resolver was broken for HTTPRoutes. +``` diff --git a/.changelog/17317.txt b/.changelog/17317.txt deleted file mode 100644 index 76c86a0a28..0000000000 --- a/.changelog/17317.txt +++ /dev/null @@ -1,3 +0,0 @@ -```release-note:bug -connect: fix a bug with Envoy potentially starting with incomplete configuration by not waiting enough for initial xDS configuration. -``` diff --git a/.changelog/17452.txt b/.changelog/17452.txt new file mode 100644 index 0000000000..2f40095dcd --- /dev/null +++ b/.changelog/17452.txt @@ -0,0 +1,3 @@ +```release-note:feature +mesh: Support configuring JWT authentication in Envoy. +``` diff --git a/.changelog/17505.txt b/.changelog/17505.txt new file mode 100644 index 0000000000..28ed8d0ed9 --- /dev/null +++ b/.changelog/17505.txt @@ -0,0 +1,3 @@ +```release-note:feature +xds: Add a built-in Envoy extension that inserts Wasm network filters. +``` diff --git a/.changelog/17513.txt b/.changelog/17513.txt new file mode 100644 index 0000000000..a87557d08c --- /dev/null +++ b/.changelog/17513.txt @@ -0,0 +1,3 @@ +```release-note:security +Update to UBI base image to 9.2. +``` diff --git a/.changelog/17525.txt b/.changelog/17525.txt new file mode 100644 index 0000000000..66bdcb8d90 --- /dev/null +++ b/.changelog/17525.txt @@ -0,0 +1,3 @@ +```release-note:improvement +http: accept query parameters `datacenter`, `ap` (enterprise-only), and `namespace` (enterprise-only). Both short-hand and long-hand forms of these query params are now supported via the HTTP API (dc/datacenter, ap/partition, ns/namespace). +``` \ No newline at end of file diff --git a/.changelog/17546.txt b/.changelog/17546.txt new file mode 100644 index 0000000000..6f40255539 --- /dev/null +++ b/.changelog/17546.txt @@ -0,0 +1,3 @@ +```release-note:improvement +connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2 +``` diff --git a/.changelog/17566.txt b/.changelog/17566.txt new file mode 100644 index 0000000000..f15718bd76 --- /dev/null +++ b/.changelog/17566.txt @@ -0,0 +1,3 @@ +```release-note:bug +xds: Fixed a bug where modifying ACLs on a token being actively used for an xDS connection caused all xDS updates to fail. +``` diff --git a/.changelog/17577.txt b/.changelog/17577.txt new file mode 100644 index 0000000000..3699d52611 --- /dev/null +++ b/.changelog/17577.txt @@ -0,0 +1,3 @@ +```release-note:improvement +fix metric names in /docs/agent/telemetry +``` \ No newline at end of file diff --git a/.changelog/17581.txt b/.changelog/17581.txt new file mode 100644 index 0000000000..9277dbcd3e --- /dev/null +++ b/.changelog/17581.txt @@ -0,0 +1,3 @@ +```release-note:bug +gateways: **(Enterprise only)** Fixed a bug in API gateways where gateway configuration objects in non-default partitions did not reconcile properly. +``` diff --git a/.changelog/17593.txt b/.changelog/17593.txt new file mode 100644 index 0000000000..1f84e75f57 --- /dev/null +++ b/.changelog/17593.txt @@ -0,0 +1,3 @@ +```release-note:bug +docs: fix list of telemetry metrics +``` diff --git a/.changelog/17596.txt b/.changelog/17596.txt new file mode 100644 index 0000000000..1058df1ea3 --- /dev/null +++ b/.changelog/17596.txt @@ -0,0 +1,3 @@ +```release-note:improvement + debug: change default setting of consul debug command. now default duration is 5ms and default log level is 'TRACE' + ``` \ No newline at end of file diff --git a/.changelog/17609.txt b/.changelog/17609.txt new file mode 100644 index 0000000000..cbace1f8c7 --- /dev/null +++ b/.changelog/17609.txt @@ -0,0 +1,4 @@ +```release-note:bug +gateways: Fixed a bug in API gateways where binding a route that only targets a service imported from a peer results +in the programmed gateway having no routes. +``` diff --git a/.changelog/17631.txt b/.changelog/17631.txt new file mode 100644 index 0000000000..b24b7461ec --- /dev/null +++ b/.changelog/17631.txt @@ -0,0 +1,3 @@ +```release-note:bug +gateways: Fixed a bug where API gateways were not being taken into account in determining xDS rate limits. +``` diff --git a/.changelog/4633.txt b/.changelog/4633.txt new file mode 100644 index 0000000000..ffacfccced --- /dev/null +++ b/.changelog/4633.txt @@ -0,0 +1,3 @@ +```release-note:feature +server: **(Enterprise Only)** added server side RPC requests IP based read/write rate-limiter. +``` diff --git a/.changelog/5102.txt b/.changelog/5102.txt new file mode 100644 index 0000000000..97d8c7bf8b --- /dev/null +++ b/.changelog/5102.txt @@ -0,0 +1,3 @@ +```release-note:feature +server: **(Enterprise Only)** allow automatic license utilization reporting. +``` \ No newline at end of file diff --git a/.changelog/_5669.txt b/.changelog/_5669.txt new file mode 100644 index 0000000000..6528460e69 --- /dev/null +++ b/.changelog/_5669.txt @@ -0,0 +1,3 @@ +```release-note:improvement +audit-logging: **(Enterprise only)** enable error response and request body logging +``` \ No newline at end of file diff --git a/.changelog/_5740.txt b/.changelog/_5740.txt new file mode 100644 index 0000000000..4f1d6f6448 --- /dev/null +++ b/.changelog/_5740.txt @@ -0,0 +1,3 @@ +```release-note:feature +api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt. +``` \ No newline at end of file diff --git a/.changelog/_5750.txt b/.changelog/_5750.txt new file mode 100644 index 0000000000..027753c721 --- /dev/null +++ b/.changelog/_5750.txt @@ -0,0 +1,3 @@ +```release-note:feature +cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt. +``` \ No newline at end of file diff --git a/.changelog/_5805.txt b/.changelog/_5805.txt new file mode 100644 index 0000000000..786effc310 --- /dev/null +++ b/.changelog/_5805.txt @@ -0,0 +1,3 @@ +```release-note:security +audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges. +``` \ No newline at end of file diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 8f017a3f89..66b5a14a73 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -34,16 +34,16 @@ jobs: run: | CONSUL_DATE=$(build-support/scripts/build-date.sh) ## TODO: This assumes `make version` outputs 1.1.1+ent-prerel - echo "::set-output name=product-date::${CONSUL_DATE}" + echo "product-date=${CONSUL_DATE}" >> "$GITHUB_OUTPUT" - name: Set shared -ldflags id: shared-ldflags run: | T="github.com/hashicorp/consul/version" - echo "::set-output name=shared-ldflags::-X ${T}.GitCommit=${GITHUB_SHA::8} \ + echo "shared-ldflags=-X ${T}.GitCommit=${GITHUB_SHA::8} \ -X ${T}.GitDescribe=${{ steps.set-product-version.outputs.product-version }} \ -X ${T}.BuildDate=${{ steps.get-product-version.outputs.product-date }} \ - " + " >> "$GITHUB_OUTPUT" validate-outputs: needs: set-product-version runs-on: ubuntu-latest diff --git a/.github/workflows/check-legacy-links-format.yml b/.github/workflows/check-legacy-links-format.yml deleted file mode 100644 index 85dbb0e70f..0000000000 --- a/.github/workflows/check-legacy-links-format.yml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright (c) HashiCorp, Inc. -# SPDX-License-Identifier: MPL-2.0 - -name: Legacy Link Format Checker - -on: - push: - paths: - - "website/content/**/*.mdx" - - "website/data/*-nav-data.json" - -jobs: - check-links: - uses: hashicorp/dev-portal/.github/workflows/docs-content-check-legacy-links-format.yml@475289345d312552b745224b46895f51cc5fc490 - with: - repo-owner: "hashicorp" - repo-name: "consul" - commit-sha: ${{ github.sha }} - mdx-directory: "website/content" - nav-data-directory: "website/data" diff --git a/.github/workflows/nightly-test-1.12.x.yaml b/.github/workflows/nightly-test-1.16.x.yaml similarity index 98% rename from .github/workflows/nightly-test-1.12.x.yaml rename to .github/workflows/nightly-test-1.16.x.yaml index 0f016075e2..c30ed6811c 100644 --- a/.github/workflows/nightly-test-1.12.x.yaml +++ b/.github/workflows/nightly-test-1.16.x.yaml @@ -1,7 +1,7 @@ # Copyright (c) HashiCorp, Inc. # SPDX-License-Identifier: MPL-2.0 -name: Nightly Test 1.12.x +name: Nightly Test 1.16.x on: schedule: - cron: '0 4 * * *' @@ -9,8 +9,8 @@ on: env: EMBER_PARTITION_TOTAL: 4 # Has to be changed in tandem with the matrix.partition - BRANCH: "release/1.12.x" - BRANCH_NAME: "release-1.12.x" # Used for naming artifacts + BRANCH: "release/1.16.x" + BRANCH_NAME: "release-1.16.x" # Used for naming artifacts jobs: frontend-test-workspace-node: diff --git a/.github/workflows/test-integrations.yml b/.github/workflows/test-integrations.yml index 260044aee8..38b6a44cbe 100644 --- a/.github/workflows/test-integrations.yml +++ b/.github/workflows/test-integrations.yml @@ -239,7 +239,7 @@ jobs: # this is further going to multiplied in envoy-integration tests by the # other dimensions in the matrix. Currently TOTAL_RUNNERS would be # multiplied by 8 based on these values: - # envoy-version: ["1.23.8", "1.24.6", "1.25.4", "1.26.0"] + # envoy-version: ["1.23.10", "1.24.8", "1.25.7", "1.26.2"] # xds-target: ["server", "client"] TOTAL_RUNNERS: 4 JQ_SLICER: '[ inputs ] | [_nwise(length / $runnercount | floor)]' @@ -273,7 +273,7 @@ jobs: strategy: fail-fast: false matrix: - envoy-version: ["1.23.8", "1.24.6", "1.25.4", "1.26.0"] + envoy-version: ["1.23.10", "1.24.8", "1.25.7", "1.26.2"] xds-target: ["server", "client"] test-cases: ${{ fromJSON(needs.generate-envoy-job-matrices.outputs.envoy-matrix) }} env: diff --git a/.gitignore b/.gitignore index 9bb786209c..a48d19b74c 100644 --- a/.gitignore +++ b/.gitignore @@ -66,3 +66,5 @@ override.tf.json # Ignore CLI configuration files .terraformrc terraform.rc +/go.work +/go.work.sum diff --git a/.golangci.yml b/.golangci.yml index 245f04d7a9..bac9b716a3 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -88,6 +88,9 @@ linters-settings: - github.com/hashicorp/go-msgpack: recommendations: - github.com/hashicorp/consul-net-rpc/go-msgpack + - github.com/golang/protobuf: + recommendations: + - google.golang.org/protobuf depguard: list-type: denylist @@ -101,7 +104,9 @@ linters-settings: # Default: [] packages-with-error-message: - net/rpc: "only use forked copy in github.com/hashicorp/consul-net-rpc/net/rpc" + - github.com/golang/protobuf: "only use google.golang.org/protobuf" run: timeout: 10m concurrency: 4 + skip-dirs-use-default: false diff --git a/.release/linux/package/usr/lib/systemd/system/consul.service b/.release/linux/package/usr/lib/systemd/system/consul.service index 1bbf51a7a1..65eca696e1 100644 --- a/.release/linux/package/usr/lib/systemd/system/consul.service +++ b/.release/linux/package/usr/lib/systemd/system/consul.service @@ -6,6 +6,7 @@ After=network-online.target ConditionFileNotEmpty=/etc/consul.d/consul.hcl [Service] +Type=notify EnvironmentFile=-/etc/consul.d/consul.env User=consul Group=consul diff --git a/CHANGELOG.md b/CHANGELOG.md index bd3379535a..ef4edc7004 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,128 @@ +## 1.16.0-rc1 (June 12, 2023) + +BREAKING CHANGES: + +* api: The `/v1/health/connect/` and `/v1/health/ingress/` endpoints now immediately return 403 "Permission Denied" errors whenever a token with insufficient `service:read` permissions is provided. Prior to this change, the endpoints returned a success code with an empty result list when a token with insufficient permissions was provided. [[GH-17424](https://github.com/hashicorp/consul/issues/17424)] +* peering: Removed deprecated backward-compatibility behavior. +Upstream overrides in service-defaults will now only apply to peer upstreams when the `peer` field is provided. +Visit the 1.16.x [upgrade instructions](https://developer.hashicorp.com/consul/docs/upgrading/upgrade-specific) for more information. [[GH-16957](https://github.com/hashicorp/consul/issues/16957)] + +SECURITY: + +* audit-logging: **(Enterprise only)** limit `v1/operator/audit-hash` endpoint to ACL token with `operator:read` privileges. + +FEATURES: + +* api: (Enterprise only) Add `POST /v1/operator/audit-hash` endpoint to calculate the hash of the data used by the audit log hash function and salt. +* cli: (Enterprise only) Add a new `consul operator audit hash` command to retrieve and compare the hash of the data used by the audit log hash function and salt. +* cli: Adds new command - `consul services export` - for exporting a service to a peer or partition [[GH-15654](https://github.com/hashicorp/consul/issues/15654)] +* connect: **(Consul Enterprise only)** Implement order-by-locality failover. +* mesh: Add new permissive mTLS mode that allows sidecar proxies to forward incoming traffic unmodified to the application. This adds `AllowEnablingPermissiveMutualTLS` setting to the mesh config entry and the `MutualTLSMode` setting to proxy-defaults and service-defaults. [[GH-17035](https://github.com/hashicorp/consul/issues/17035)] +* mesh: Support configuring JWT authentication in Envoy. [[GH-17452](https://github.com/hashicorp/consul/issues/17452)] +* server: **(Enterprise Only)** added server side RPC requests IP based read/write rate-limiter. [[GH-4633](https://github.com/hashicorp/consul/issues/4633)] +* server: **(Enterprise Only)** allow automatic license utilization reporting. [[GH-5102](https://github.com/hashicorp/consul/issues/5102)] +* server: added server side RPC requests global read/write rate-limiter. [[GH-16292](https://github.com/hashicorp/consul/issues/16292)] +* xds: Add `property-override` built-in Envoy extension that directly patches Envoy resources. [[GH-17487](https://github.com/hashicorp/consul/issues/17487)] +* xds: Add a built-in Envoy extension that inserts External Authorization (ext_authz) network and HTTP filters. [[GH-17495](https://github.com/hashicorp/consul/issues/17495)] +* xds: Add a built-in Envoy extension that inserts Wasm HTTP filters. [[GH-16877](https://github.com/hashicorp/consul/issues/16877)] +* xds: Add a built-in Envoy extension that inserts Wasm network filters. [[GH-17505](https://github.com/hashicorp/consul/issues/17505)] + +IMPROVEMENTS: + +* * api: Support filtering for config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)] +* * cli: Add `-filter` option to `consul config list` for filtering config entries. [[GH-17183](https://github.com/hashicorp/consul/issues/17183)] +* api: Enable setting query options on agent force-leave endpoint. [[GH-15987](https://github.com/hashicorp/consul/issues/15987)] +* audit-logging: (Enterprise only) enable error response and request body logging [[GH-5669](https://github.com/hashicorp/consul/issues/5669)] +* audit-logging: **(Enterprise only)** enable error response and request body logging +* ca: automatically set up Vault's auto-tidy setting for tidy_expired_issuers when using Vault as a CA provider. [[GH-17138](https://github.com/hashicorp/consul/issues/17138)] +* ca: support Vault agent auto-auth config for Vault CA provider using AliCloud authentication. [[GH-16224](https://github.com/hashicorp/consul/issues/16224)] +* ca: support Vault agent auto-auth config for Vault CA provider using AppRole authentication. [[GH-16259](https://github.com/hashicorp/consul/issues/16259)] +* ca: support Vault agent auto-auth config for Vault CA provider using Azure MSI authentication. [[GH-16298](https://github.com/hashicorp/consul/issues/16298)] +* ca: support Vault agent auto-auth config for Vault CA provider using JWT authentication. [[GH-16266](https://github.com/hashicorp/consul/issues/16266)] +* ca: support Vault agent auto-auth config for Vault CA provider using Kubernetes authentication. [[GH-16262](https://github.com/hashicorp/consul/issues/16262)] +* command: Adds ACL enabled to status output on agent startup. [[GH-17086](https://github.com/hashicorp/consul/issues/17086)] +* command: Allow creating ACL Token TTL with greater than 24 hours with the -expires-ttl flag. [[GH-17066](https://github.com/hashicorp/consul/issues/17066)] +* connect: **(Enterprise Only)** Add support for specifying "Partition" and "Namespace" in Prepared Queries failover rules. +* connect: update supported envoy versions to 1.23.10, 1.24.8, 1.25.7, 1.26.2 [[GH-17546](https://github.com/hashicorp/consul/issues/17546)] +* connect: update supported envoy versions to 1.23.8, 1.24.6, 1.25.4, 1.26.0 [[GH-5200](https://github.com/hashicorp/consul/issues/5200)] +* fix metric names in /docs/agent/telemetry [[GH-17577](https://github.com/hashicorp/consul/issues/17577)] +* gateway: Change status condition reason for invalid certificate on a listener from "Accepted" to "ResolvedRefs". [[GH-17115](https://github.com/hashicorp/consul/issues/17115)] +* http: accept query parameters `datacenter`, `ap` (enterprise-only), and `namespace` (enterprise-only). Both short-hand and long-hand forms of these query params are now supported via the HTTP API (dc/datacenter, ap/partition, ns/namespace). [[GH-17525](https://github.com/hashicorp/consul/issues/17525)] +* systemd: set service type to notify. [[GH-16845](https://github.com/hashicorp/consul/issues/16845)] +* ui: Update alerts to Hds::Alert component [[GH-16412](https://github.com/hashicorp/consul/issues/16412)] +* ui: Update to use Hds::Toast component to show notifications [[GH-16519](https://github.com/hashicorp/consul/issues/16519)] +* ui: update from