60 Commits (8572ac135c839a2590a733bcc048b1adbf1158f6)

Author	SHA1	Message	Date
Paul Banks	b621910618	Support Connect CAs that can't cross sign (#6726) ... * Support Connect CAs that can't cross sign * revert spurios mod changes from make tools * Add log warning when forcing CA rotation * Fixup SupportsCrossSigning to report errors and work with Plugin interface (fixes tests) * Fix failing snake_case test * Remove misleading comment * Revert "Remove misleading comment" This reverts commit bc4db9cabed8ad5d0e39b30e1fe79196d248349c. * Remove misleading comment * Regen proto files messed up by rebase	5 years ago
Paul Banks	45d57ca601	connect: Allow CA Providers to store small amount of state (#6751) ... * pass logger through to provider * test for proper operation of NeedsLogger * remove public testServer function * Ooops actually set the logger in all the places we need it - CA config set wasn't and causing segfault * Fix all the other places in tests where we set the logger * Allow CA Providers to persist some state * Update CA provider plugin interface * Fix plugin stubs to match provider changes * Update agent/connect/ca/provider.go Co-Authored-By: R.B. Boyer <rb@hashicorp.com> * Cleanup review comments	5 years ago
Todd Radel	29b5253154	connect: Implement NeedsLogger interface for CA providers (#6556) ... * add NeedsLogger to Provider interface * implements NeedsLogger in default provider * pass logger through to provider * test for proper operation of NeedsLogger * remove public testServer function * Switch test to actually assert on logging output rather than reflection. --amend * Ooops actually set the logger in all the places we need it - CA config set wasn't and causing segfault * Fix all the other places in tests where we set the logger * Add TODO comment	5 years ago
Todd Radel	54f92e2924	Make all Connect Cert Common Names valid FQDNs (#6423)	5 years ago
Paul Banks	87699eca2f	Fix support for RSA CA keys in Connect. (#6638) ... * Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs. * Ensure key type ad bits are populated from CA cert and clean up tests * Add integration test and fix error when initializing secondary CA with RSA key. * Add more tests, fix review feedback * Update docs with key type config and output * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com>	5 years ago
Matt Keeler	28221f66f2	Use encoding/json instead of jsonpb even for protobuf types (#6572) ... This only works so long as we use simplistic protobuf types. Constructs such as oneof or Any types that require type annotations for decoding properly will fail hard but that is by design. If/when we want to use any of that we will probably need to consider a v2 API.	5 years ago
Matt Keeler	abed91d069	Generate JSON and Binary Marshalers for Protobuf Types (#6564) ... * Add JSON and Binary Marshaler Generators for Protobuf Types * Generate files with the correct version of gogo/protobuf I have pinned the version in the makefile so when you run make tools you get the right version. This pulls the version out of go.mod so it should remain up to date. The version at the time of this commit we are using is v1.2.1 * Fixup some shell output * Update how we determine the version of gogo This just greps the go.mod file instead of expecting the go mod cache to already be present * Fixup vendoring and remove no longer needed json encoder functions	5 years ago
R.B. Boyer	af01d397a5	connect: don't colon-hex-encode the AuthorityKeyId and SubjectKeyId fields in connect certs (#6492) ... The fields in the certs are meant to hold the original binary representation of this data, not some ascii-encoded version. The only time we should be colon-hex-encoding fields is for display purposes or marshaling through non-TLS mediums (like RPC).	5 years ago
R.B. Boyer	796de297c8	connect: intermediate CA certs generated with the vault provider lack URI SANs (#6491) ... This only affects vault versions >=1.1.1 because the prior code accidentally relied upon a bug that was fixed in https://github.com/hashicorp/vault/pull/6505 The existing tests should have caught this, but they were using a vendored copy of vault version 0.10.3. This fixes the tests by running an actual copy of vault instead of an in-process copy. This has the added benefit of changing the dependency on vault to just vault/api. Also update VaultProvider to use similar SetIntermediate validation code as the ConsulProvider implementation.	5 years ago
Matt Keeler	51dcd126b7	Add support for implementing new requests with protobufs instea… (#6502) ... * Add build system support for protobuf generation This is done generically so that we don’t have to keep updating the makefile to add another proto generation. Note: anything not in the vendor directory and with a .proto extension will be run through protoc if the corresponding namespace.pb.go file is not up to date. If you want to rebuild just a single proto file you can do so with: make proto-rebuild PROTOFILES=<list of proto files to rebuild> Providing the PROTOFILES var will override the default behavior of finding all the .proto files. * Start adding types to the agent/proto package These will be needed for some other work and are by no means comprehensive. * Add ability to resolve/fixup the agentpb.ACLLinks structure in the state store. * Use protobuf marshalling of raft requests instead of msgpack for protoc generated types. This does not change any encoding of existing types. * Removed structs package automatically encoding with protobuf marshalling Instead the caller of raftApply that wants to opt-in to protobuf encoding will have to call `raftApplyProtobuf` * Run update-vendor to fixup modules.txt Nothing changed as far as dependencies go but the ordering of modules in that file depends on the time they are first seen and its not alphabetical. * Rename some things and implement the structs.RPCInfo interface bits agentpb.QueryOptions and agentpb.WriteRequest implement 3 of the 4 RPCInfo funcs and the new TargetDatacenter message type implements the fourth. * Use the right encoding function. * Renamed agent/proto package to agent/agentpb to prevent package name conflicts * Update modules.txt to fix ordering * Change blockingQuery to take in interfaces for the query options and meta * Add %T to error output. * Add/Update some comments	5 years ago
Alvin Huang	c516fabfac	revert commits on master (#6413)	5 years ago
tradel	9b1ac4e7ef	add subject names to issued certs	5 years ago
tradel	82ae7caf3e	Added DC and domain args to Configure method	5 years ago
R.B. Boyer	561b2fe606	connect: generate the full SNI names for discovery targets in the compiler rather than in the xds package (#6340)	5 years ago
Paul Banks	e87cef2bb8	Revert "connect: support AWS PCA as a CA provider" (#6251) ... This reverts commit 3497b7c00d.	5 years ago
Todd Radel	3497b7c00d	connect: support AWS PCA as a CA provider (#6189) ... Port AWS PCA provider from consul-ent	5 years ago
Todd Radel	2552f4a11a	connect: Support RSA keys in addition to ECDSA (#6055) ... Support RSA keys in addition to ECDSA	5 years ago
Christian Muehlhaeuser	7753b97cc7	Simplified code in various places (#6176) ... All these changes should have no side-effects or change behavior: - Use bytes.Buffer's String() instead of a conversion - Use time.Since and time.Until where fitting - Drop unnecessary returns and assignment	5 years ago
Hans Hasselberg	33a7df3330	tls: auto_encrypt enables automatic RPC cert provisioning for consul clients (#5597)	6 years ago
R.B. Boyer	f4a3b9d518	fix typos reported by golangci-lint:misspell (#5434)	6 years ago
R.B. Boyer	409c901f8e	test: fix concurrent map access when setting up test vault	6 years ago
R.B. Boyer	c7067645dd	fix a few leap-year related clock math inaccuracies and failing tests	6 years ago
Kyle Havlovitz	29e4c17b07	connect/ca: fix a potential panic in the Consul provider	6 years ago
Kyle Havlovitz	a28ba4687d	connect/ca: return a better error message if the CA isn't fully initialized when signing	6 years ago
Hans Hasselberg	067027230b	connect: add tls config for vault connect ca provider (#5125) ... * add tlsconfig for vault connect ca provider. * add options to the docs * add tests for new configuration	6 years ago
Mitchell Hashimoto	f76022fa63	CA Provider Plugins (#4751) ... This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet. ## Why? We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security. ### go-plugin The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself. The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin. `go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature. ## Implementation Details ### `map[string]interface{}` The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software. Note that this is just the transport protocol, the end software views it fully decoded. ### `x509.Certificate` and `CertificateRequest` We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire. ### Testing `go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC. ### Vendoring This PR doesn't introduce vendoring for two reasons: 1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict. 2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors). ## Next Steps With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.	6 years ago
Kyle Havlovitz	e8dd89359a	agent: fix formatting	6 years ago
Aestek	25f04fbd21	[Security] Add finer control over script checks (#4715) ... * Add -enable-local-script-checks options These options allow for a finer control over when script checks are enabled by giving the option to only allow them when they are declared from the local file system. * Add documentation for the new option * Nitpick doc wording	6 years ago
Kyle Havlovitz	57deb28ade	connect/ca: tighten up the intermediate signing verification	6 years ago
Kyle Havlovitz	2919519665	connect/ca: add intermediate functions to Vault ca provider	6 years ago
Kyle Havlovitz	52e8652ac5	connect/ca: add intermediate functions to Consul CA provider	6 years ago
Kyle Havlovitz	d515d25856	Merge pull request #4644 from hashicorp/ca-refactor ... connect/ca: rework initialization/root generation in providers	6 years ago
Paul Banks	74f2a80a42	Fix CA pruning when CA config uses string durations. (#4669) ... * Fix CA pruning when CA config uses string durations. The tl;dr here is: - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported - Most of our tests managed to escape this by defining them as time.Duration directly - Out actual default value is a string - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC). - msgpack decode leaves the string as a `[]uint8` - Some of our parsers required string and failed - So after 1 hour, a default configured server would throw an error about pruning old CAs - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA. - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding. tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them. This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time! * Typo fix * Fix bad Uint8 usage	6 years ago
Kyle Havlovitz	5c7fbc284d	connect/ca: hash the consul provider ID and include isRoot	6 years ago
Kyle Havlovitz	c112a72880	connect/ca: some cleanup and reorganizing of the new methods	6 years ago
Kyle Havlovitz	546bdf8663	connect/ca: add Configure/GenerateRoot to provider interface	6 years ago
Siva Prasad	288d350a73	Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix." (#4497) ... * Revert "BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472)" This reverts commit cec5d72396. * Revert "CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493)" This reverts commit 589b589b53.	6 years ago
Siva Prasad	589b589b53	CA initialization while boostrapping and TestLeader_ChangeServerID fix. (#4493) ... * connect: fix an issue with Consul CA bootstrapping being interrupted * streamline change server id test	6 years ago
Kyle Havlovitz	f67a4d59c0	connect/ca: simplify passing of leaf cert TTL	6 years ago
Kyle Havlovitz	ce10de036e	connect/ca: check LeafCertTTL when rotating expired roots	6 years ago
Kyle Havlovitz	d6ca015a42	connect/ca: add configurable leaf cert TTL	6 years ago
Matt Keeler	677d6dac80	Remove x509 name constraints ... These were only added as SPIFFE intends to use the in the future but currently does not mandate their usage due to patch support in common TLS implementations and some ambiguity over how to use them with URI SAN certificates. We included them because until now everything seem fine with it, however we've found the latest version of `openssl` (1.1.0h) fails to validate our certificats if its enabled. LibreSSL as installed on OS X by default doesn’t have these issues. For now it's most compatible not to have them and later we can find ways to add constraints with wider compatibility testing.	7 years ago
Kyle Havlovitz	8c2c9705d9	connect/ca: use weak type decoding in the Vault config parsing	7 years ago
Kyle Havlovitz	050da22473	connect/ca: undo the interface changes and use sign-self-issued in Vault	7 years ago
Kyle Havlovitz	914d9e5e20	connect/ca: add leaf verify check to cross-signing tests	7 years ago
Kyle Havlovitz	bc997688e3	connect/ca: update Consul provider to use new cross-sign CSR method	7 years ago
Kyle Havlovitz	8a70ea64a6	connect/ca: update Vault provider to add cross-signing methods	7 years ago
Kyle Havlovitz	6a2fc00997	connect/ca: add URI SAN support to the Vault provider	7 years ago
Kyle Havlovitz	226a59215d	connect/ca: fix vault provider URI SANs and test	7 years ago
Kyle Havlovitz	1a8ac686b2	connect/ca: add the Vault CA provider	7 years ago