consul

Commit Graph

Author	SHA1	Message	Date
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884 ) This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the agent and the results of that operation for primary mesh gateways are needed in the server there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Matt Keeler	80ed304e04	Run make update-vendor and fixup various go.sum files go mod tidy removes these lines because we have a replace directive	5 years ago
Matt Keeler	77074be9b8	Bump sdk module version to 0.4.0	5 years ago
Akshay Ganeshen	8beb716414	feat: support sending body in HTTP checks (#6602 )	5 years ago
Matt Keeler	d0cd092e3b	Catalog + Namespace OSS changes. (#7219 ) * Various Prepared Query + Namespace things * Last round of OSS changes for a namespaced catalog	5 years ago
Fouad Zaryouh	ef6399928d	api: add replace-existing-checks param to the api package (#7136 )	5 years ago
Matt Keeler	6855a778c2	Updates to the Txn API for namespaces (#7172 ) * Updates to the Txn API for namespaces * Update agent/consul/txn_endpoint.go Co-Authored-By: R.B. Boyer <rb@hashicorp.com> Co-authored-by: R.B. Boyer <public@richardboyer.net>	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130 ) * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Michel Vocks	6681be918a	api: add option to set TLS options in-memory for API client (#7093 ) This PR adds the option to set in-memory certificates to the API client instead of requiring the certificate to be stored on disk in a file. This allows us to define API client TLS options per Consul secret backend in Vault. Related issue hashicorp/vault#4800	5 years ago
Matt Keeler	b57c2b78fd	Unflake the TestAPI_AgentConnectCALeaf test (#7142 ) * Unflake the TestAPI_AgentConnectCALeaf test * Modify the WaitForActiveCARoot to actually verify that at least one root exists Also verify that the active root id field is set	5 years ago
Matt Keeler	c09693e545	Updates to Config Entries and Connect for Namespaces (#7116 )	5 years ago
Matt Keeler	bbc2eb1951	Add the v1/catalog/node-services/:node endpoint (#7115 ) The backing RPC already existed but the endpoint will be useful for other service syncing processes such as consul-k8s as this endpoint can return all services registered with a node regardless of namespacing.	5 years ago
R.B. Boyer	4af5c2d6dd	fix the submodule go.mod and go.sum files (#7098 )	5 years ago
Paul Banks	1807af552e	Fix TestAPI_DiscoveryChain_Get flake (#7082 )	5 years ago
Hans Hasselberg	804eb17094	connect: check if intermediate cert needs to be renewed. (#6835 ) Currently when using the built-in CA provider for Connect, root certificates are valid for 10 years, however secondary DCs get intermediates that are valid for only 1 year. There is no mechanism currently short of rotating the root in the primary that will cause the secondary DCs to renew their intermediates. This PR adds a check that renews the cert if it is half way through its validity period. In order to be able to test these changes, a new configuration option was added: IntermediateCertTTL which is set extremely low in the tests.	5 years ago
Aestek	ba8fd8296f	Add support for dual stack IPv4/IPv6 network (#6640 ) * Use consts for well known tagged adress keys * Add ipv4 and ipv6 tagged addresses for node lan and wan * Add ipv4 and ipv6 tagged addresses for service lan and wan * Use IPv4 and IPv6 address in DNS	5 years ago
neillyt	a2ab9b4aee	Fix Namespace Typo (#7030 )	5 years ago
Matt Keeler	34e6e23d43	Change how namespaces are specified for the CLI (#6960 )	5 years ago
Freddy	37dfe6d112	Add kv meta to namespaces api module (#6958 )	5 years ago
R.B. Boyer	7ac47d6f61	build: fix makefile and CI 'fmt' and 'vet' tasks (#6937 )	5 years ago
ogidow	dfa8d919f7	agent: fix watch event behavior (#5265 ) Use an equality match on the last event id, not a small value, to correct the random nature of the trigger	5 years ago
Matt Keeler	5934f803bf	Sync of OSS changes to support namespaces (#6909 )	5 years ago
Matt Keeler	2343413bf0	Fix the TestAPI_CatalogRegistration test	5 years ago
Matt Keeler	a704ebe639	Add Namespace support to the API module and the CLI commands (#6874 ) Also update the Docs and fixup the HTTP API to return proper errors when someone attempts to use Namespaces with an OSS agent. Add Namespace HTTP API docs Make all API endpoints disallow unknown fields	5 years ago
Mike Morris	7484b42b93	test: wait for Serf check in TestServicesWatch (#6867 ) Following example from TestNodesWatch * test: WaitForSerfCheck all the places in api/watch_funcs_test.go	5 years ago
Sarah Adams	aed5cb7669	give feedback to CLI user on forceleave command if node does not exist (#6841 )	5 years ago
Matt Keeler	b069d6777b	OSS KV Modifications to Support Namespaces	5 years ago
Paul Banks	cd1b613352	connect: Add AWS PCA provider (#6795 ) * Update AWS SDK to use PCA features. * Add AWS PCA provider * Add plumbing for config, config validation tests, add test for inheriting existing CA resources created by user * Unparallel the tests so we don't exhaust PCA limits * Merge updates * More aggressive polling; rate limit pass through on sign; Timeout on Sign and CA create * Add AWS PCA docs * Fix Vault doc typo too * Doc typo * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com> Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Doc fixes; tests for erroring if State is modified via API * More review cleanup * Uncomment tests! * Minor suggested clean ups	5 years ago
Matt Keeler	f6af75ae74	Refactor the Read Namespace requests to not use the query funct… (#6720 ) The query function doesn’t handle non-200 return codes properly so this ended up still trying to decode the body which resulted in weird error messages.	5 years ago
Blake Covarrubias	0aa025df1c	docs: Miscellaneous docs cleanup (#6742 ) Fix spelling errors, API doc inconsistencies, and formatting issues. * Fix several spelling errors. * Prepend / to v1/event/list path in Watches. * Rename script handlers to match Watch type. * Remove /v1 path prefix on service health API endpoints. Makes request path consistent with the rest of the HTTP API documentation which does not include the /v1 prefix. * Fix bracket formatting issue on Telemetry page. The HTML codes used for brackets inside of the code block are not interpolated, and are shown as literal strings. Replace the numeric HTML codes with the intended character value to fix display formatting. Also placed variable reference on agent/options.html inside code block for consistency with the presentation of other options on the page. * Add missing word to Coordinate.Node docstring. Resolves #6014	5 years ago
Sarah Adams	78ad8203a4	Use encoding/json as JSON decoder instead of mapstructure (#6680 ) Fixes #6147	5 years ago
Sarah Christoff	5e1c6e907b	Set MinQuorum variable in Autopilot (#6654 ) * Add MinQuorum to Autopilot	5 years ago
Freddy	60f6ec0c2f	Store check type in catalog (#6561 )	5 years ago
Matt Keeler	4a5ebb57b0	Add support for the Namespace HTTP API in the API Client (#6581 )	5 years ago
Sarah Christoff	5e26971864	Prune Unhealthy Agents (#6571 ) * Add -prune flag to ForceLeave	5 years ago
Freddy	2274f64cab	Add Namespace as an api query/write option (#6551 )	5 years ago
Freddy	fdd10dd8b8	Expose HTTP-based paths through Connect proxy (#6446 ) Fixes: #5396 This PR adds a proxy configuration stanza called expose. These flags register listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only listening on the loopback interface, while still accepting traffic from non Connect-enabled services. Under expose there is a boolean checks flag that would automatically expose all registered HTTP and gRPC check paths. This stanza also accepts a paths list to expose individual paths. The primary use case for this functionality would be to expose paths for third parties like Prometheus or the kubelet. Listeners for requests to exposed paths are be configured dynamically at run time. Any time a proxy, or check can be registered, a listener can also be created. In this initial implementation requests to these paths are not authenticated/encrypted.	5 years ago
R.B. Boyer	92fd596d32	api/watch: try to avoid more flakes in this package (#6538 )	5 years ago
R.B. Boyer	af01d397a5	connect: don't colon-hex-encode the AuthorityKeyId and SubjectKeyId fields in connect certs (#6492 ) The fields in the certs are meant to hold the original binary representation of this data, not some ascii-encoded version. The only time we should be colon-hex-encoding fields is for display purposes or marshaling through non-TLS mediums (like RPC).	5 years ago
R.B. Boyer	f74244dde8	api/watch: reduce timing dependence on tests of watch behavior Also for debugging purposes send the stdout/stderr streams from consul processes spawned for API tests to testing.T.Logf	5 years ago
R.B. Boyer	ea6231ce35	api: bump to sdk v0.2.0	5 years ago
R.B. Boyer	fd1c62ee8b	connect: ensure time.Duration fields retain their human readable forms in the API (#6348 ) This applies for both config entries and the compiled discovery chain. Also omit some other config entries fields when empty.	5 years ago
R.B. Boyer	561b2fe606	connect: generate the full SNI names for discovery targets in the compiler rather than in the xds package (#6340 )	5 years ago
R.B. Boyer	ae79cdab1b	connect: introduce ExternalSNI field on service-defaults (#6324 ) Compiling this will set an optional SNI field on each DiscoveryTarget. When set this value should be used for TLS connections to the instances of the target. If not set the default should be used. Setting ExternalSNI will disable mesh gateway use for that target. It also disables several service-resolver features that do not make sense for an external service.	5 years ago
Matt Keeler	69c7d64701	Add missing LicenseReset API function (#6311 )	5 years ago
Sarah Adams	8ff1f481fe	add flag to allow /operator/keyring requests to only hit local servers (#6279 ) Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic). HTTP API: GET /operator/keyring?local-only=true CLI: consul keyring -list --local-only Sending the local-only flag with any non-GET/list request will result in an error.	5 years ago
Matt Keeler	6a1e0dfed8	Update the v1/agent/service/:service endpoint to output tagged… (#6304 )	5 years ago
Mike Morris	65be58703c	connect: remove managed proxies (#6220 ) * connect: remove managed proxies implementation and all supporting config options and structs * connect: remove deprecated ProxyDestination * command: remove CONNECT_PROXY_TOKEN env var * agent: remove entire proxyprocess proxy manager * test: remove all managed proxy tests * test: remove irrelevant managed proxy note from TestService_ServerTLSConfig * test: update ContentHash to reflect managed proxy removal * test: remove deprecated ProxyDestination test * telemetry: remove managed proxy note * http: remove /v1/agent/connect/proxy endpoint * ci: remove deprecated test exclusion * website: update managed proxies deprecation page to note removal * website: remove managed proxy configuration API docs * website: remove managed proxy note from built-in proxy config * website: add note on removing proxy subdirectory of data_dir	5 years ago
R.B. Boyer	9753c77a79	api: un-deprecate api.DecodeConfigEntry (#6278 ) Add clarifying commentary about when it is not safe to use it. Also add tests.	5 years ago
R.B. Boyer	c395affc93	connect: expose an API endpoint to compile the discovery chain (#6248 ) In addition to exposing compilation over the API cleaned up the structures that would be exchanged to be cleaner and easier to support and understand. Also removed ability to configure the envoy OverprovisioningFactor.	5 years ago

1 2 3 4 5 ...

489 Commits (c5b7c072fde4caae06fa705269e430745654374c)