68 Commits (9f90060b0fa3b183245fe9388b4debbd12359e73)

Author	SHA1	Message	Date
Matt Keeler	efe279f802	Retry lint fixes (#19151) ... * Add a make target to run lint-consul-retry on all the modules * Cleanup sdk/testutil/retry * Fix a bunch of retry.Run* usage to not use the outer testing.T * Fix some more recent retry lint issues and pin to v1.4.0 of lint-consul-retry * Fix codegen copywrite lint issues * Don’t perform cleanup after each retry attempt by default. * Use the common testutil.TestingTB interface in test-integ/tenancy * Fix retry tests * Update otel access logging extension test to perform requests within the retry block	12 months ago
John Murret	2591318c82	Skip tests with p95 greater than 30 seconds outside of main and release branches. (#19628) ... Skip tests with p95 greater than 30 seconds.	1 year ago
Chris S. Kim	9d00b13140	Vault CA bugfixes (#19285) ... * Re-add retry logic to Vault token renewal * Fix goroutine leak * Add test for detecting goroutine leak * Add changelog * Rename tests * Add comment	1 year ago
Chris Thain	dcdf2fc6ba	Update Vault CA provider namespace configuration (#19095)	1 year ago
Chris S. Kim	4dfca64ded	Vault CA provider clean up previous default issuers (#18773)	1 year ago
hashicorp-copywrite[bot]	5fb9df1640	[COMPLIANCE] License changes (#18443) ... * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at <Blog URL>, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * add missing license headers * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	1 year ago
Chris S. Kim	747a4c73c1	Fix bug with Vault CA provider (#18112) ... Updating RootPKIPath but not IntermediatePKIPath would not update leaf signing certs with the new root. Unsure if this happens in practice but manual testing showed it is a bug that would break mesh and agent connections once the old root is pruned.	1 year ago
Chris S. Kim	a4653de8da	CA provider doc updates and Vault provider minor update (#17831) ... Update CA provider docs Clarify that providers can differ between primary and secondary datacenters Provide a comparison chart for consul vs vault CA providers Loosen Vault CA provider validation for RootPKIPath Update Vault CA provider documentation	1 year ago
John Eikenberry	bd76fdeaeb	enable auto-tidy expired issuers in vault (as CA) ... When using vault as a CA and generating the local signing cert, try to enable the PKI endpoint's auto-tidy feature with it set to tidy expired issuers.	2 years ago
Chris S. Kim	a5397b1f23	Connect CA Primary Provider refactor (#16749) ... * Rename Intermediate cert references to LeafSigningCert Within the Consul CA subsystem, the term "Intermediate" is confusing because the meaning changes depending on provider and datacenter (primary vs secondary). For example, when using the Consul CA the "ActiveIntermediate" may return the root certificate in a primary datacenter. At a high level, we are interested in knowing which CA is responsible for signing leaf certs, regardless of its position in a certificate chain. This rename makes the intent clearer. * Move provider state check earlier * Remove calls to GenerateLeafSigningCert GenerateLeafSigningCert (formerly known as GenerateIntermediate) is vestigial in non-Vault providers, as it simply returns the root certificate in primary datacenters. By folding Vault's intermediate cert logic into `GenerateRoot` we can encapsulate the intermediate cert handling within `newCARoot`. * Move GenerateLeafSigningCert out of PrimaryProvidder Now that the Vault Provider calls GenerateLeafSigningCert within GenerateRoot, we can remove the method from all other providers that never used it in a meaningful way. * Add test for IntermediatePEM * Rename GenerateRoot to GenerateCAChain "Root" was being overloaded in the Consul CA context, as different providers and configs resulted in a single root certificate or a chain originating from an external trusted CA. Since the Vault provider also generates intermediates, it seems more accurate to call this a CAChain.	2 years ago
Ronald	94ec4eb2f4	copyright headers for agent folder (#16704) ... * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files	2 years ago
John Eikenberry	f5641ffccc	support vault auth config for alicloud ca provider ... Add support for using existing vault auto-auth configurations as the provider configuration when using Vault's CA provider with AliCloud. AliCloud requires 2 extra fields to enable it to use STS (it's preferred auth setup). Our vault-plugin-auth-alicloud package contained a method to help generate them as they require you to make an http call to a faked endpoint proxy to get them (url and headers base64 encoded).	2 years ago
John Eikenberry	56ffee6d42	add provider ca support for approle auth-method ... Adds support for the approle auth-method. Only handles using the approle role/secret to auth and it doesn't support the agent's extra management configuration options (wrap and delete after read) as they are not required as part of the auth (ie. they are vault agent things).	2 years ago
John Eikenberry	e8eec1fa80	add provider ca auth support for kubernetes ... Adds support for Kubernetes jwt/token file based auth. Only needs to read the file and save the contents as the jwt/token.	2 years ago
John Eikenberry	4211069080	add provider ca support for jwt file base auth ... Adds support for a jwt token in a file. Simply reads the file and sends the read in jwt along to the vault login. It also supports a legacy mode with the jwt string being passed directly. In which case the path is made optional.	2 years ago
John Eikenberry	4f2d9a91e5	add provider ca auth-method support for azure ... Does the required dance with the local HTTP endpoint to get the required data for the jwt based auth setup in Azure. Keeps support for 'legacy' mode where all login data is passed on via the auth methods parameters. Refactored check for hardcoded /login fields.	2 years ago
John Eikenberry	ed7367b6f4	remove redundant vault api retry logic (#16143) ... remove redundant vault api retry logic We upgraded Vault API module version to a version that has built-in retry logic. So this code is no longer necessary. Also add mention of re-configuring the provider in comments.	2 years ago
John Eikenberry	5c836f2aa9	fix goroutine leak in renew testing (#16142) ... fix goroutine leak in renew testing Test overwrote the stopWatcher() function variable for the test without keeping and calling the original value. The original value is the function that stops the goroutine... so it needs to be called.	2 years ago
Chris Thain	2f4c8e50f2	Support Vault agent auth config for AWS/GCP CA provider auth (#15970)	2 years ago
R.B. Boyer	900584ca82	connect: ensure all vault connect CA tests use limited privilege tokens (#15669) ... All of the current integration tests where Vault is the Connect CA now use non-root tokens for the test. This helps us detect privilege changes in the vault model so we can keep our guides up to date. One larger change was that the RenewIntermediate function got refactored slightly so it could be used from a test, rather than the large duplicated function we were testing in a test which seemed error prone.	2 years ago
R.B. Boyer	4940a728ab	Detect Vault 1.11+ import in secondary datacenters and update default issuer (#15661) ... The fix outlined and merged in #15253 fixed the issue as it occurs in the primary DC. There is a similar issue that arises when vault is used as the Connect CA in a secondary datacenter that is fixed by this PR. Additionally: this PR adds support to run the existing suite of vault related integration tests against the last 4 versions of vault (1.9, 1.10, 1.11, 1.12)	2 years ago
Chris S. Kim	c9ec9fa320	Fix Vault managed intermediate PKI bug (#15525)	2 years ago
Alexander Scheel	2b90307f6d	Detect Vault 1.11+ import, update default issuer (#15253) ... Consul used to rely on implicit issuer selection when calling Vault endpoints to issue new CSRs. Vault 1.11+ changed that behavior, which caused Consul to check the wrong (previous) issuer when renewing its Intermediate CA. This patch allows Consul to explicitly set a default issuer when it detects that the response from Vault is 1.11+. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com>	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
Kyle Havlovitz	d122108992	Warn instead of returning an error when intermediate mount tune permission is missing	2 years ago
Kyle Havlovitz	d67bccd210	Update intermediate pki mount/role when reconfiguring Vault provider	2 years ago
Connor	922619dfc3	Fix leaked Vault LifetimeRenewers (#12607) ... * Fix leaked Vault LifetimeRenewers When the Vault CA Provider is reconfigured we do not stop the LifetimeRenewers which can cause them to leak until the Consul processes recycles. On Configure execute stopWatcher if it exists and is not nil before starting a new renewal * Add jitter before restarting the LifetimeWatcher If we fail to login to Vault or our token is no longer valid we can overwhelm a Vault instance with many requests very quickly by restarting the LifetimeWatcher. Before restarting the LifetimeWatcher provide a backoff time of 1 second or less. * Use a retry.Waiter instead of RandomStagger * changelog * gofmt'd * Swap out bool for atomic.Unit32 in test * Provide some extra clarification in comment and changelog	3 years ago
Daniel Nephin	9b7468f99e	ca/provider: remove ActiveRoot from Provider	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script ... set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
Iryna Shustava	0ee456649f	connect: Support auth methods for the vault connect CA provider (#11573) ... * Support vault auth methods for the Vault connect CA provider * Rotate the token (re-authenticate to vault using auth method) when the token can no longer be renewed	3 years ago
Connor	efe4b21287	Support Vault Namespaces explicitly in CA config (#11477) ... * Support Vault Namespaces explicitly in CA config If there is a Namespace entry included in the Vault CA configuration, set it as the Vault Namespace on the Vault client Currently the only way to support Vault namespaces in the Consul CA config is by doing one of the following: 1) Set the VAULT_NAMESPACE environment variable which will be picked up by the Vault API client 2) Prefix all Vault paths with the namespace Neither of these are super pleasant. The first requires direct access and modification to the Consul runtime environment. It's possible and expected, not super pleasant. The second requires more indepth knowledge of Vault and how it uses Namespaces and could be confusing for anyone without that context. It also infers that it is not supported * Add changelog * Remove fmt.Fprint calls * Make comment clearer * Add next consul version to website docs * Add new test for default configuration * go mod tidy * Add skip if vault not present * Tweak changelog text	3 years ago
FFMMM	4ddf973a31	add root_cert_ttl option for consul connect, vault ca providers (#11428) ... * add root_cert_ttl option for consul connect, vault ca providers Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Chris S. Kim <ckim@hashicorp.com> * add changelog, pr feedback Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> * Update .changelog/11428.txt, more docs Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> * Update website/content/docs/agent/options.mdx Co-authored-by: Kyle Havlovitz <kylehav@gmail.com> Co-authored-by: Chris S. Kim <ckim@hashicorp.com> Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>	3 years ago
Daniel Nephin	fc14f5ab14	ca: move provider creation into CAManager ... This further decouples the CAManager from Server. It reduces the interface between them and removes the need for the SetLogger method on providers.	3 years ago
Dhia Ayachi	9b45107c1e	Format certificates properly (rfc7468) with a trailing new line (#10411) ... * trim carriage return from certificates when inserting rootCA in the inMemDB * format rootCA properly when returning the CA on the connect CA endpoint * Fix linter warnings * Fix providers to trim certs before returning it * trim newlines on write when possible * add changelog * make sure all provider return a trailing newline after the root and intermediate certs * Fix endpoint to return trailing new line * Fix failing test with vault provider * make test more robust * make sure all provider return a trailing newline after the leaf certs * Check for suffix before removing newline and use function * Add comment to consul provider * Update change log Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> * fix typo * simplify code callflow Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> * extract requireNewLine as shared func * remove dependency to testify in testing file * remove extra newline in vault provider * Add cert newline fix to envoy xds * remove new line from mock provider * Remove adding a new line from provider and fix it when the cert is read * Add a comment to explain the fix * Add missing for leaf certs * fix missing new line * fix missing new line in leaf certs * remove extra new line in test * updage changelog Co-authored-by: Daniel Nephin <dnephin@hashicorp.com> * fix in vault provider and when reading cache (RPC call) * fix AWS provider * fix failing test in the provider * remove comments and empty lines * add check for empty cert in test * fix linter warnings * add new line for leaf and private key * use string concat instead of Sprintf * fix new lines for leaf signing * preallocate slice and remove append * Add new line to `SignIntermediate` and `CrossSignCA` Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com> Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>	3 years ago
Matt Keeler	d9d4c492ab	Ensure that CA initialization does not block leader election. ... After fixing that bug I uncovered a couple more: Fix an issue where we might try to cross sign a cert when we never had a valid root. Fix a potential issue where reconfiguring the CA could cause either the Vault or AWS PCA CA providers to delete resources that are still required by the new incarnation of the CA.	4 years ago
Kyle Havlovitz	01ce9f5b18	Update CI for leader renew CA test using Vault	4 years ago
Kyle Havlovitz	2ec94b027e	connect: Enable renewing the intermediate cert in the primary DC	4 years ago
Kyle Havlovitz	49056fe70f	Clean up Vault renew tests and shutdown	4 years ago
Kyle Havlovitz	411b6537ef	Add a test for token renewal	4 years ago
Hans Hasselberg	5281cb74db	Setup intermediate_pki_path on secondary when using vault (#8001) ... Make sure to mount vault backend for intermediate_pki_path on secondary dc.	5 years ago
Daniel Nephin	61ec7aa5c9	ci: Run all connect/ca tests from the integration suite ... To reduce the chance of some tests not being run because it does not match the regex passed to '-run'. Also document why some tests are allowed to be skipped on CI.	5 years ago
Daniel Nephin	f4a35dfd84	ci: Do not skip tests because of missing binaries on CI ... If the CI environment is not correct for running tests the tests should fail, so that we don't accidentally stop running some tests because of a change to our CI environment. Also removed a duplicate delcaration from init. I believe one was overriding the other as they are both in the same package.	5 years ago
R.B. Boyer	e2eb9f0585	test: ensure we don't ask vault to sign a leaf that outlives its CA when acting as a secondary (#7100)	5 years ago
Paul Banks	d7329097b2	Change CA Configure struct to pass Datacenter through (#6775) ... * Change CA Configure struct to pass Datacenter through * Remove connect/ca/plugin as we don't have immediate plans to use it. We still intend to one day but there are likely to be several changes to the CA provider interface before we do so it's better to rebuild from history when we do that work properly. * Rename PrimaryDC; fix endpoint in secondary DCs	5 years ago
Paul Banks	45d57ca601	connect: Allow CA Providers to store small amount of state (#6751) ... * pass logger through to provider * test for proper operation of NeedsLogger * remove public testServer function * Ooops actually set the logger in all the places we need it - CA config set wasn't and causing segfault * Fix all the other places in tests where we set the logger * Allow CA Providers to persist some state * Update CA provider plugin interface * Fix plugin stubs to match provider changes * Update agent/connect/ca/provider.go Co-Authored-By: R.B. Boyer <rb@hashicorp.com> * Cleanup review comments	5 years ago
Todd Radel	29b5253154	connect: Implement NeedsLogger interface for CA providers (#6556) ... * add NeedsLogger to Provider interface * implements NeedsLogger in default provider * pass logger through to provider * test for proper operation of NeedsLogger * remove public testServer function * Switch test to actually assert on logging output rather than reflection. --amend * Ooops actually set the logger in all the places we need it - CA config set wasn't and causing segfault * Fix all the other places in tests where we set the logger * Add TODO comment	5 years ago
Todd Radel	54f92e2924	Make all Connect Cert Common Names valid FQDNs (#6423)	5 years ago
Paul Banks	87699eca2f	Fix support for RSA CA keys in Connect. (#6638) ... * Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs. * Ensure key type ad bits are populated from CA cert and clean up tests * Add integration test and fix error when initializing secondary CA with RSA key. * Add more tests, fix review feedback * Update docs with key type config and output * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com>	5 years ago
R.B. Boyer	796de297c8	connect: intermediate CA certs generated with the vault provider lack URI SANs (#6491) ... This only affects vault versions >=1.1.1 because the prior code accidentally relied upon a bug that was fixed in https://github.com/hashicorp/vault/pull/6505 The existing tests should have caught this, but they were using a vendored copy of vault version 0.10.3. This fixes the tests by running an actual copy of vault instead of an in-process copy. This has the added benefit of changing the dependency on vault to just vault/api. Also update VaultProvider to use similar SetIntermediate validation code as the ConsulProvider implementation.	5 years ago
Alvin Huang	c516fabfac	revert commits on master (#6413)	5 years ago