303 Commits (9f7afb56b2e7551d6bdf2fb5f3d65244a2d8c13f)

Author	SHA1	Message	Date
Freddy	9eb1867fbb	Terminating gateway discovery (#7571) ... * Enable discovering terminating gateways * Add TerminatingGatewayServices to state store * Use GatewayServices RPC endpoint for ingress/terminating	5 years ago
Freddy	aae14b3951	Add decode rules for Expose cfg in service-defaults (#7611)	5 years ago
Matt Keeler	0e7d3d93b3	Enable filtering language support for the v1/connect/intentions… (#7593) ... * Enable filtering language support for the v1/connect/intentions listing API * Update website for filtering of Intentions * Update website/source/api/connect/intentions.html.md	5 years ago
Freddy	90576060bc	Add config entry for terminating gateways (#7545) ... This config entry will be used to configure terminating gateways. It accepts the name of the gateway and a list of services the gateway will represent. For each service users will be able to specify: its name, namespace, and additional options for TLS origination. Co-authored-by: Kyle Havlovitz <kylehav@gmail.com> Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>	5 years ago
Kyle Havlovitz	c911174327	Add config entry/state for Ingress Gateways (#7483) ... * Add Ingress gateway config entry and other relevant structs * Add api package tests for ingress gateways * Embed EnterpriseMeta into ingress service struct * Add namespace fields to api module and test consul config write decoding * Don't require a port for ingress gateways * Add snakeJSON and camelJSON cases in command test * Run Normalize on service's ent metadata Sadly cannot think of a way to test this in OSS. * Every protocol requires at least 1 service * Validate ingress protocols * Update agent/structs/config_entry_gateways.go Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Freddy	18d356899c	Enable CLI to register terminating gateways (#7500) ... * Enable CLI to register terminating gateways * Centralize gateway proxy configuration	5 years ago
Alejandro Baez	bafa69bb69	Add PolicyReadByName for API (#6615)	5 years ago
R.B. Boyer	85a08bf8ed	server: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#7419) ... Fixes #7414	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884) ... This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the *agent* and the results of that operation for primary mesh gateways are needed in the *server* there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Matt Keeler	7584dfe8c8	Fix session backwards incompatibility with 1.6.x and earlier.	5 years ago
Matt Keeler	e231d62bc9	Make the config entry and leaf cert cache types ns aware (#7256)	5 years ago
Hans Hasselberg	6739fe6e83	connect: add validations around intermediate cert ttl (#7213)	5 years ago
Akshay Ganeshen	8beb716414	feat: support sending body in HTTP checks (#6602)	5 years ago
Matt Keeler	d0cd092e3b	Catalog + Namespace OSS changes. (#7219) ... * Various Prepared Query + Namespace things * Last round of OSS changes for a namespaced catalog	5 years ago
R.B. Boyer	8c596953b0	agent: ensure that we always use the same settings for msgpack (#7245) ... We set RawToString=true so that []uint8 => string when decoding an interface{}. We set the MapType so that map[interface{}]interface{} decodes to map[string]interface{}. Add tests to ensure that this doesn't break existing usages. Fixes #7223	5 years ago
Matt Keeler	9e5fd7f925	OSS Changes for various config entry namespacing bugs (#7226)	5 years ago
Matt Keeler	dfb0177dbc	Testing updates to support namespaced testing of the agent/xds… (#7185) ... * Various testing updates to support namespaced testing of the agent/xds package * agent/proxycfg package updates to support better namespace testing	5 years ago
Matt Keeler	6855a778c2	Updates to the Txn API for namespaces (#7172) ... * Updates to the Txn API for namespaces * Update agent/consul/txn_endpoint.go Co-Authored-By: R.B. Boyer <rb@hashicorp.com> Co-authored-by: R.B. Boyer <public@richardboyer.net>	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130) ... * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Matt Keeler	c09693e545	Updates to Config Entries and Connect for Namespaces (#7116)	5 years ago
Hans Hasselberg	804eb17094	connect: check if intermediate cert needs to be renewed. (#6835) ... Currently when using the built-in CA provider for Connect, root certificates are valid for 10 years, however secondary DCs get intermediates that are valid for only 1 year. There is no mechanism currently short of rotating the root in the primary that will cause the secondary DCs to renew their intermediates. This PR adds a check that renews the cert if it is half way through its validity period. In order to be able to test these changes, a new configuration option was added: IntermediateCertTTL which is set extremely low in the tests.	5 years ago
Aestek	ba8fd8296f	Add support for dual stack IPv4/IPv6 network (#6640) ... * Use consts for well known tagged adress keys * Add ipv4 and ipv6 tagged addresses for node lan and wan * Add ipv4 and ipv6 tagged addresses for service lan and wan * Use IPv4 and IPv6 address in DNS	5 years ago
Matt Keeler	663cf1e9a8	AuthMethod updates to support alternate namespace logins (#7029)	5 years ago
Matt Keeler	8bd34e126f	Intentions ACL enforcement updates (#7028) ... * Renamed structs.IntentionWildcard to structs.WildcardSpecifier * Refactor ACL Config Get rid of remnants of enterprise only renaming. Add a WildcardName field for specifying what string should be used to indicate a wildcard. * Add wildcard support in the ACL package For read operations they can call anyAllowed to determine if any read access to the given resource would be granted. For write operations they can call allAllowed to ensure that write access is granted to everything. * Make v1/agent/connect/authorize namespace aware * Update intention ACL enforcement This also changes how intention:read is granted. Before the Intention.List RPC would allow viewing an intention if the token had intention:read on the destination. However Intention.Match allowed viewing if access was allowed for either the source or dest side. Now Intention.List and Intention.Get fall in line with Intention.Matches previous behavior. Due to this being done a few different places ACL enforcement for a singular intention is now done with the CanRead and CanWrite methods on the intention itself. * Refactor Intention.Apply to make things easier to follow.	5 years ago
R.B. Boyer	10f04a8c4a	connect: derive connect certificate serial numbers from a memdb index instead of the provider table max index (#7011)	5 years ago
Matt Keeler	fa2003d7cb	Move Session.CheckIDs into OSS only code. (#6993)	5 years ago
Matt Keeler	a78f7d7a34	OSS changes for implementing token based namespace inferencing ... remove debug log	5 years ago
Matt Keeler	80d13d500b	Miscellaneous acl package cleanup ... • Renamed EnterpriseACLConfig to just Config • Removed chained_authorizer_oss.go as it was empty • Renamed acl.go to errors.go to more closely describe its contents	5 years ago
Matt Keeler	0b346616e9	Rename EnterpriseAuthorizerContext -> AuthorizerContext	5 years ago
Matt Keeler	5934f803bf	Sync of OSS changes to support namespaces (#6909)	5 years ago
rerorero	34649b8820	[ci] fix: go-fmt fails on master branch (#6906)	5 years ago
Matt Keeler	2343413bf0	Fix the TestAPI_CatalogRegistration test	5 years ago
Matt Keeler	8f0ab0129e	Miscellaneous Fixes (#6896) ... Ensure we close the Sentinel Evaluator so as not to leak go routines Fix a bunch of test logging so that various warnings when starting a test agent go to the ltest logger and not straight to stdout. Various canned ent meta types always return a valid pointer (no more nils). This allows us to blindly deref + assign in various places. Update ACL index tracking to ensure oss -> ent upgrades will work as expected. Update ent meta parsing to include function to disallow wildcarding.	5 years ago
Matt Keeler	a704ebe639	Add Namespace support to the API module and the CLI commands (#6874) ... Also update the Docs and fixup the HTTP API to return proper errors when someone attempts to use Namespaces with an OSS agent. Add Namespace HTTP API docs Make all API endpoints disallow unknown fields	5 years ago
Matt Keeler	deb91f3d3c	[Feature] API: Add a internal endpoint to query for ACL authori… (#6888) ... * Implement endpoint to query whether the given token is authorized for a set of operations * Updates to allow for remote ACL authorization via RPC This is only used when making an authorization request to a different datacenter.	5 years ago
Matt Keeler	b069d6777b	OSS KV Modifications to Support Namespaces	5 years ago
Matt Keeler	7b471f6bf8	OSS Modifications necessary for sessions namespacing	5 years ago
Paul Banks	cd1b613352	connect: Add AWS PCA provider (#6795) ... * Update AWS SDK to use PCA features. * Add AWS PCA provider * Add plumbing for config, config validation tests, add test for inheriting existing CA resources created by user * Unparallel the tests so we don't exhaust PCA limits * Merge updates * More aggressive polling; rate limit pass through on sign; Timeout on Sign and CA create * Add AWS PCA docs * Fix Vault doc typo too * Doc typo * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com> Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Doc fixes; tests for erroring if State is modified via API * More review cleanup * Uncomment tests! * Minor suggested clean ups	5 years ago
Paul Banks	b621910618	Support Connect CAs that can't cross sign (#6726) ... * Support Connect CAs that can't cross sign * revert spurios mod changes from make tools * Add log warning when forcing CA rotation * Fixup SupportsCrossSigning to report errors and work with Plugin interface (fixes tests) * Fix failing snake_case test * Remove misleading comment * Revert "Remove misleading comment" This reverts commit bc4db9cabed8ad5d0e39b30e1fe79196d248349c. * Remove misleading comment * Regen proto files messed up by rebase	5 years ago
Paul Banks	45d57ca601	connect: Allow CA Providers to store small amount of state (#6751) ... * pass logger through to provider * test for proper operation of NeedsLogger * remove public testServer function * Ooops actually set the logger in all the places we need it - CA config set wasn't and causing segfault * Fix all the other places in tests where we set the logger * Allow CA Providers to persist some state * Update CA provider plugin interface * Fix plugin stubs to match provider changes * Update agent/connect/ca/provider.go Co-Authored-By: R.B. Boyer <rb@hashicorp.com> * Cleanup review comments	5 years ago
Matt Keeler	ab5a05f71d	Fix type name (#6728)	5 years ago
Matt Keeler	825e19bc5f	Add DirEntry method to fill enterprise authz context	5 years ago
Paul Banks	87699eca2f	Fix support for RSA CA keys in Connect. (#6638) ... * Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs. * Ensure key type ad bits are populated from CA cert and clean up tests * Add integration test and fix error when initializing secondary CA with RSA key. * Add more tests, fix review feedback * Update docs with key type config and output * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com>	5 years ago
Matt Keeler	5d687ce6a9	Fix the Synthetic Policy Tests (#6715)	5 years ago
Sarah Adams	78ad8203a4	Use encoding/json as JSON decoder instead of mapstructure (#6680) ... Fixes #6147	5 years ago
Matt Keeler	79f78632e1	Update the ACL Resolver to allow for Consul Enterprise specific hooks. (#6687)	5 years ago
Matt Keeler	e4ea9b0a96	Updates to allow for Namespacing ACL resources in Consul Enterp… (#6675) ... Main Changes: • method signature updates everywhere to account for passing around enterprise meta. • populate the EnterpriseAuthorizerContext for all ACL related authorizations. • ACL resource listings now operate like the catalog or kv listings in that the returned entries are filtered down to what the token is allowed to see. With Namespaces its no longer all or nothing. • Modified the acl.Policy parsing to abstract away basic decoding so that enterprise can do it slightly differently. Also updated method signatures so that when parsing a policy it can take extra ent metadata to use during rules validation and policy creation. Secondary Changes: • Moved protobuf encoding functions out of the agentpb package to eliminate circular dependencies. • Added custom JSON unmarshalers for a few ACL resource types (to support snake case and to get rid of mapstructure) • AuthMethod validator cache is now an interface as these will be cached per-namespace for Consul Enterprise. • Added checks for policy/role link existence at the RPC API so we don’t push the request through raft to have it fail internally. • Forward ACL token delete request to the primary datacenter when the secondary DC doesn’t have the token. • Added a bunch of ACL test helpers for inserting ACL resource test data.	5 years ago
Freddy	60f6ec0c2f	Store check type in catalog (#6561)	5 years ago
Matt Keeler	973341a592	ACL Authorizer overhaul (#6620) ... * ACL Authorizer overhaul To account for upcoming features every Authorization function can now take an extra *acl.EnterpriseAuthorizerContext. These are unused in OSS and will always be nil. Additionally the acl package has received some thorough refactoring to enable all of the extra Consul Enterprise specific authorizations including moving sentinel enforcement into the stubbed structs. The Authorizer funcs now return an acl.EnforcementDecision instead of a boolean. This improves the overall interface as it makes multiple Authorizers easily chainable as they now indicate whether they had an authoritative decision or should use some other defaults. A ChainedAuthorizer was added to handle this Authorizer enforcement chain and will never itself return a non-authoritative decision. * Include stub for extra enterprise rules in the global management policy * Allow for an upgrade of the global-management policy	5 years ago
PHBourquin	039615641e	Checks to passing/critical only after reaching a consecutive success/failure threshold (#5739) ... A check may be set to become passing/critical only if a specified number of successive checks return passing/critical in a row. Status will stay identical as before until the threshold is reached. This feature is available for HTTP, TCP, gRPC, Docker & Monitor checks.	5 years ago