226 Commits (4275e8fa82b3b54ee1ed04373fa8a659cce2e484)

Author	SHA1	Message	Date
R.B. Boyer	398b766532	xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658) ... - Also add support for envoy 1.17.0	4 years ago
R.B. Boyer	3b6ffc447b	xds: remove deprecated usages of xDS (#9602) ... Note that this does NOT upgrade to xDS v3. That will come in a future PR. Additionally: - Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated. - Shuffled some agent/xds imports in advance of a later xDS v3 upgrade. - Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR. Fixes #8425	4 years ago
R.B. Boyer	6eeccc93ce	connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737)	4 years ago
Kyle Havlovitz	7dac583863	connect/ca: Allow ForceWithoutCrossSigning for all providers ... This allows setting ForceWithoutCrossSigning when reconfiguring the CA for any provider, in order to forcibly move to a new root in cases where the old provider isn't reachable or able to cross-sign for whatever reason.	4 years ago
R.B. Boyer	b5f9a293c9	chore: regenerate envoy golden files (#9634)	4 years ago
Daniel Nephin	b9e60c0775	testing: skip slow tests with -short ... Add a skip condition to all tests slower than 100ms. This change was made using `gotestsum tool slowest` with data from the last 3 CI runs of master. See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests With this change: ``` $ time go test -count=1 -short ./agent ok github.com/hashicorp/consul/agent 0.743s real 0m4.791s $ time go test -count=1 -short ./agent/consul ok github.com/hashicorp/consul/agent/consul 4.229s real 0m8.769s ```	4 years ago
R.B. Boyer	7c7a3e5165	command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229) ... Fixes #9215	4 years ago
Freddy	fe728855ed	Add DC and NS support for Envoy metrics (#9207) ... This PR updates the tags that we generate for Envoy stats. Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.	4 years ago
Mike Morris	6396042ba7	connect: switch the default gateway port from 443 to 8443 (#9116) ... * test: update ingress gateway golden file to port 8443 * test: update Envoy flags_test to port 8443 Co-authored-by: R.B. Boyer <rb@hashicorp.com>	4 years ago
R.B. Boyer	8baf158ea8	Revert "Add namespace support for metrics (OSS) (#9117)" (#9124) ... This reverts commit 06b3b017d3.	4 years ago
Freddy	06b3b017d3	Add namespace support for metrics (OSS) (#9117)	4 years ago
R.B. Boyer	a2c50d3303	connect: add support for envoy 1.16.0, drop support for 1.12.x, and bump point releases as well (#8944) ... Supported versions will be: "1.16.0", "1.15.2", "1.14.5", "1.13.6"	4 years ago
R.B. Boyer	9fbcb2e68d	command: remove conditional envoy bootstrap generation for versions <=1.10.0 since those are not supported (#8855)	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834) ... - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
Tim Arenz	a1fe711390	Add support for -ca-path option in the connect envoy command (#8606) ... * Add support for -ca-path option in the connect envoy command * Adding changelog entry	4 years ago
Daniel Nephin	33c401a16e	logging: Setup accept io.Writer instead of []io.Writer ... Also accept a non-pointer Config, since the config is not modified	4 years ago
Daniel Nephin	d68edcecf4	testing: Remove all the defer os.Removeall ... Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage the removal	4 years ago
R.B. Boyer	397019d970	xds: revert setting set_node_on_first_message_only to true when generating envoy bootstrap config (#8440) ... When consul is restarted and an envoy that had already sent DiscoveryRequests to the previous consul process sends a request to the new process it doesn't respect the setting and never populates DiscoveryRequest.Node for the life of the new consul process due to this bug: https://github.com/envoyproxy/envoy/issues/9682 Fixes #8430	4 years ago
Daniel Nephin	0420d91cdd	Remove LogOutput from Agent ... Now that it is no longer used, we can remove this unnecessary field. This is a pre-step in cleanup up RuntimeConfig->Consul.Config, which is a pre-step to adding a gRPCHandler component to Server for streaming. Removing this field also allows us to remove one of the return values from logging.Setup.	4 years ago
R.B. Boyer	c599a2f5f4	xds: add support for envoy 1.15.0 and drop support for 1.11.x (#8424) ... Related changes: - hard-fail the xDS connection attempt if the envoy version is known to be too old to be supported - remove the RouterMatchSafeRegex proxy feature since all supported envoy versions have it - stop using --max-obj-name-len (due to: envoyproxy/envoy#11740)	4 years ago
Chris Piraino	7c4cc71131	Fix envoy bootstrap logic to not append multiple self_admin clusters (#8371) ... Previously, the envoy bootstrap config would blindly copy the self_admin cluster into the list of static clusters when configuring either ReadyBindAddr, PrometheusBindAddr, or StatsBindAddr. Since ingress gateways always configure the ReadyBindAddr property, users ran into this case much more often than previously.	4 years ago
Hans Hasselberg	496fb5fc5b	add support for envoy 1.14.4, 1.13.4, 1.12.6 (#8216)	4 years ago
R.B. Boyer	1eef096dfe	xds: version sniff envoy and switch regular expressions from 'regex' to 'safe_regex' on newer envoy versions (#8222) ... - cut down on extra node metadata transmission - split the golden file generation to compare all envoy version	4 years ago
R.B. Boyer	462f0f37ed	connect: various changes to make namespaces for intentions work more like for other subsystems (#8194) ... Highlights: - add new endpoint to query for intentions by exact match - using this endpoint from the CLI instead of the dump+filter approach - enforcing that OSS can only read/write intentions with a SourceNS or DestinationNS field of "default". - preexisting OSS intentions with now-invalid namespace fields will delete those intentions on initial election or for wildcard namespaces an attempt will be made to downgrade them to "default" unless one exists. - also allow the '-namespace' CLI arg on all of the intention subcommands - update lots of docs	4 years ago
Matt Keeler	3dbbd2d37d	Implement Client Agent Auto Config ... There are a couple of things in here. First, just like auto encrypt, any Cluster.AutoConfig RPC will implicitly use the less secure RPC mechanism. This drastically modifies how the Consul Agent starts up and moves most of the responsibilities (other than signal handling) from the cli command and into the Agent.	5 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
Hans Hasselberg	e62a43c6cf	Support envoy 1.14.2, 1.13.2, 1.12.4 (#8057)	5 years ago
Kyle Havlovitz	6fd3b25313	Fix a CLI test failure with namespaces in enterprise	5 years ago
Kyle Havlovitz	e3a725c4e0	Always allow updating the exposed service and differentiate by namespace	5 years ago
Kyle Havlovitz	edab5588d8	Add -host flag to expose command	5 years ago
Kyle Havlovitz	5958328552	Allow multiple listeners per service via expose command	5 years ago
Kyle Havlovitz	acae044df4	Document the namespace format for expose CLI command	5 years ago
Kyle Havlovitz	b874c8ef0c	Add connect expose CLI command	5 years ago
Daniel Nephin	c88fae0aac	ci: Add staticcheck and fix most errors ... Three of the checks are temporarily disabled to limit the size of the diff, and allow us to enable all the other checks in CI. In a follow up we can fix the issues reported by the other checks one at a time, and enable them.	5 years ago
Kyle Havlovitz	b14696e32a	Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924) ... * Standardize support for Tagged and BindAddresses in Ingress Gateways This updates the TaggedAddresses and BindAddresses behavior for Ingress to match Mesh/Terminating gateways. The `consul connect envoy` command now also allows passing an address without a port for tagged/bind addresses. * Update command/connect/envoy/envoy.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * PR comments * Check to see if address is an actual IP address * Update agent/xds/listeners.go Co-authored-by: Freddy <freddygv@users.noreply.github.com> * fix whitespace Co-authored-by: Chris Piraino <cpiraino@hashicorp.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	5 years ago
Daniel Nephin	c662f0f0de	Fix a number of problems found by staticcheck ... Some of these problems are minor (unused vars), but others are real bugs (ignored errors). Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago
Freddy	ccd0822539	Use proxy-id in gateway auto-registration (#7845)	5 years ago
Chris Piraino	3d2de925d8	Add support for ingress-gateway in CLI command (#7618) ... * Add support for ingress-gateway in CLI command - Supports -register command - Creates a static Envoy listener that exposes only the /ready API so that we can register a TCP healthcheck against the ingress gateway itself - Updates ServiceAddressValue.String() to be more in line with Value()	5 years ago
Daniel Nephin	25b585d0bf	Fix golden file for envoy tests ... The envoy version was updated after the PR which added this test was opened, and merged before the test was merged, so it ended up with the wrong version.	5 years ago
Daniel Nephin	6b860c926f	Merge pull request #7608 from hashicorp/dnephin/grpc-default-scheme ... command/envoy: enable TLS when CONSUL_HTTP_ADDR=https://...	5 years ago
Hans Hasselberg	66415be90e	connect: support envoy 1.14.1 (#7624)	5 years ago
Daniel Nephin	8b6861518f	Fix CONSUL_HTTP_ADDR=https not enabling TLS ... Use the config instead of attempting to reparse the env var.	5 years ago
Daniel Nephin	0888c6575b	Step 3: fix a bug in api.NewClient and fix the tests ... The api client should never rever to HTTP if the user explicitly requested TLS. This change broke some tests because the tests always use an non-TLS http server, but some tests explicitly enable TLS.	5 years ago
Daniel Nephin	1a8ffec6a7	Step 2: extract the grpc address logic and a new type ... The new grpcAddress function contains all of the logic to translate the command line options into the values used in the template. The new type has two advantages. 1. It introduces a logical grouping of values in the BootstrapTplArgs struct which is exceptionally large. This grouping makes the struct easier to understand because each set of nested values can be seen as a single entity. 2. It gives us a reasonable return value for this new function.	5 years ago
Daniel Nephin	830b4a15f6	Step 1: move all the grpcAddr logic into the same spot ... There is no reason a reader should have to jump around to find this value. It is only used in 1 place	5 years ago
Freddy	b61214ef24	Fix regression with gateway registration and update docs (#7582)	5 years ago
Daniel Nephin	475659a132	Remove name from NewTestAgent ... Using: git grep -l 'NewTestAgent(t, t.Name(),' | \ xargs sed -i -e 's/NewTestAgent(t, t.Name(),/NewTestAgent(t,/g'	5 years ago
Daniel Nephin	09d0876b6c	command: remove unused logOutput field	5 years ago
Freddy	18d356899c	Enable CLI to register terminating gateways (#7500) ... * Enable CLI to register terminating gateways * Centralize gateway proxy configuration	5 years ago
Daniel Nephin	e5d6273a48	command/envoy: Refactor flag parsing/validation (#7504)	5 years ago
Daniel Nephin	a95974cf79	Remove unnecessary methods ... They call only a single method and add no additional functionality	5 years ago
Daniel Nephin	8df3746927	cmd: use env vars as defaults ... Insted of setting them afterward in Run. This change required a small re-ordering of the test to patch the environment before calling New()	5 years ago
Daniel Nephin	6e10616b13	Fix tests failing on master ... The default version was changed in https://github.com/hashicorp/consul/pull/7452 which caused these tests to fail.	5 years ago
Hans Hasselberg	d5f4b8c3a3	envoy: default to 1.13.1 (#7452)	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884) ... This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the *agent* and the results of that operation for primary mesh gateways are needed in the *server* there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Chris Piraino	d7fdb249fc	Fix -mesh-gateway flag help text (#7265)	5 years ago
Hans Hasselberg	6739fe6e83	connect: add validations around intermediate cert ttl (#7213)	5 years ago
Hans Hasselberg	9cb7adb304	add envoy version 1.12.2 and 1.13.0 to the matrix (#7240) ... * add 1.12.2 * add envoy 1.13.0 * Introduce -envoy-version to get 1.10.0 passing. * update old version and fix consul-exec case * add envoy_version and fix check * Update Envoy CLI tests to account for the 1.13 compatibility changes. Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago
Paschalis Tsilias	a335aa57c5	Expose Envoy's /stats for statsd agents (#7173) ... * Expose Envoy /stats for statsd agents; Add testcases * Remove merge conflict leftover * Add support for prefix instead of path; Fix docstring to mirror these changes * Add new config field to docs; Add testcases to check that /stats/prometheus is exposed as well * Parametrize matchType (prefix or path) and value * Update website/source/docs/connect/proxies/envoy.md Co-Authored-By: Paul Banks <banks@banksco.de> Co-authored-by: Paul Banks <banks@banksco.de>	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130) ... * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Anthony Scalisi	beb928f8de	fix spelling errors (#7135)	5 years ago
Matt Keeler	c09693e545	Updates to Config Entries and Connect for Namespaces (#7116)	5 years ago
Hans Hasselberg	804eb17094	connect: check if intermediate cert needs to be renewed. (#6835) ... Currently when using the built-in CA provider for Connect, root certificates are valid for 10 years, however secondary DCs get intermediates that are valid for only 1 year. There is no mechanism currently short of rotating the root in the primary that will cause the secondary DCs to renew their intermediates. This PR adds a check that renews the cert if it is half way through its validity period. In order to be able to test these changes, a new configuration option was added: IntermediateCertTTL which is set extremely low in the tests.	5 years ago
Aestek	ba8fd8296f	Add support for dual stack IPv4/IPv6 network (#6640) ... * Use consts for well known tagged adress keys * Add ipv4 and ipv6 tagged addresses for node lan and wan * Add ipv4 and ipv6 tagged addresses for service lan and wan * Use IPv4 and IPv6 address in DNS	5 years ago
Hans Hasselberg	ab2f2484ad	connect: use inline_string instead for envoy ca(#7024)	5 years ago
Hans Hasselberg	1661041046	envoy: replace ca filename with inline_bytes. (#6822)	5 years ago
Aestek	6817676223	Fix consul connect token env variable doc (#5942) ... The cli documentation for consul connect commands incorrectly indicated to use CONSUL_TOKEN instead of CONSUL_HTTP_TOKEN env var.	5 years ago
R.B. Boyer	72207256b9	xds: improve how envoy metrics are emitted (#6312) ... Since generated envoy clusters all are named using (mostly) SNI syntax we can have envoy read the various fields out of that structure and emit it as stats labels to the various telemetry backends. I changed the delimiter for the 'customization hash' from ':' to '~' because ':' is always reencoded by envoy as '_' when generating metrics keys.	5 years ago
Mike Morris	65be58703c	connect: remove managed proxies (#6220) ... * connect: remove managed proxies implementation and all supporting config options and structs * connect: remove deprecated ProxyDestination * command: remove CONNECT_PROXY_TOKEN env var * agent: remove entire proxyprocess proxy manager * test: remove all managed proxy tests * test: remove irrelevant managed proxy note from TestService_ServerTLSConfig * test: update ContentHash to reflect managed proxy removal * test: remove deprecated ProxyDestination test * telemetry: remove managed proxy note * http: remove /v1/agent/connect/proxy endpoint * ci: remove deprecated test exclusion * website: update managed proxies deprecation page to note removal * website: remove managed proxy configuration API docs * website: remove managed proxy note from built-in proxy config * website: add note on removing proxy subdirectory of data_dir	5 years ago
Alvin Huang	37ea271eb7	fix grpc-addr-config hosts template	5 years ago
Alvin Huang	206b2016a4	Merge remote-tracking branch 'origin/master' into release/1-6	5 years ago
Sarah Adams	896749d585	fix 'consul connect envoy' to try to use previously-configured grpc port (#6245) ... fix 'consul connect envoy' to try to use previously-configured grpc port on running agent before defaulting to 8502 Fixes #5011	5 years ago
Matt Keeler	1fdda51839	Fix envoy canBind (#6238) ... * Fix envoy cli canBind function The string form of an Addr was including the CIDR causing the str equals to not match. * Remove debug prints	5 years ago
Matt Keeler	59454c7edc	Set --max-obj-name-len 256 when execing Envoy (#6202) ... * Pass -max-obj-name-len 256 to envoy * Update test expectations. * Add a note about requireing the max-obj-name-len option to be set	5 years ago
hashicorp-ci	fa20c7db97	Merge Consul OSS branch 'master' at commit 95dbb7f2f1	5 years ago
Freddy	d219e31db8	Update retries that weren't using retry.R (#6146)	5 years ago
Matt Keeler	6e65811db2	Envoy CLI bind addresses (#6107) ... * Ensure we MapWalk the proxy config in the NodeService and ServiceNode structs This gets rid of some json encoder errors in the catalog endpoints * Allow passing explicit bind addresses to envoy * Move map walking to the ConnectProxyConfig struct Any place where this struct gets JSON encoded will benefit as opposed to having to implement it everywhere. * Fail when a non-empty address is provided and not bindable * camel case * Update command/connect/envoy/envoy.go Co-Authored-By: Paul Banks <banks@banksco.de>	5 years ago
Michael Schurter	b5aab27c21	connect: allow overriding envoy listener bind_address (#6033) ... * connect: allow overriding envoy listener bind_address * Update agent/xds/config.go Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com> * connect: allow overriding envoy listener bind_port * envoy: support unix sockets for grpc in bootstrap Add AgentSocket BootstrapTplArgs which if set overrides the AgentAddress and AgentPort to generate a bootstrap which points Envoy to a unix socket file instead of an ip:port. * Add a test for passing the consul addr as a unix socket * Fix config formatting for envoy bootstrap tests * Fix listeners test cases for bind addr/port * Update website/source/docs/connect/proxies/envoy.md	5 years ago
Matt Keeler	8d953f5840	Implement Mesh Gateways ... This includes both ingress and egress functionality.	5 years ago
Paul Banks	9f656a2dc8	Fix envoy 1.10 exec (#5964) ... * Make exec test assert Envoy version - it was not rebuilding before and so often ran against wrong version. This makes 1.10 fail consistenty. * Switch Envoy exec to use a named pipe rather than FD magic since Envoy 1.10 doesn't support that. * Refactor to use an internal shim command for piping the bootstrap through. * Fmt. So sad that vscode golang fails so often these days. * go mod tidy * revert go mod tidy changes * Revert "ignore consul-exec tests until fixed (#5986)" This reverts commit 683262a686. * Review cleanups	6 years ago
Hans Hasselberg	4d9116d759	connect: provide -admin-access-log-path for envoy (#5858)	6 years ago
Paul Banks	0cfb6051ea	Add integration test for central config; fix central config WIP (#5752) ... * Add integration test for central config; fix central config WIP * Add integration test for central config; fix central config WIP * Set proxy protocol correctly and begin adding upstream support * Add upstreams to service config cache key and start new notify watcher if they change. This doesn't update the tests to pass though. * Fix some merging logic get things working manually with a hack (TODO fix properly) * Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway * Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does * Fix up service manageer and API test failures * Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally * Remove version.go hack - will make integration test fail until release * Remove unused code from commands and upstream merge * Re-bump version to 1.5.0	6 years ago
Matt Keeler	3145bf5230	Centralized Config CLI (#5731) ... * Add HTTP endpoints for config entry management * Finish implementing decoding in the HTTP Config entry apply endpoint * Add CAS operation to the config entry apply endpoint Also use this for the bootstrapping and move the config entry decoding function into the structs package. * First pass at the API client for the config entries * Fixup some of the ConfigEntry APIs Return a singular response object instead of a list for the ConfigEntry.Get RPC. This gets plumbed through the HTTP API as well. Dont return QueryMeta in the JSON response for the config entry listing HTTP API. Instead just return a list of config entries. * Minor API client fixes * Attempt at some ConfigEntry api client tests These don’t currently work due to weak typing in JSON * Get some of the api client tests passing * Implement reflectwalk magic to correct JSON encoding a ProxyConfigEntry Also added a test for the HTTP endpoint that exposes the problem. However, since the test doesn’t actually do the JSON encode/decode its still failing. * Move MapWalk magic into a binary marshaller instead of JSON. * Add a MapWalk test * Get rid of unused func * Get rid of unused imports * Fixup some tests now that the decoding from msgpack coerces things into json compat types * Stub out most of the central config cli Fully implement the config read command. * Basic config delete command implementation * Implement config write command * Implement config list subcommand Not entirely sure about the output here. Its basically the read output indented with a line specifying the kind/name of each type which is also duplicated in the indented output. * Update command usage * Update some help usage formatting * Add the connect enable helper cli command * Update list command output * Rename the config entry API client methods. * Use renamed apis * Implement config write tests Stub the others with the noTabs tests. * Change list output format Now just simply output 1 line per named config * Add config read tests * Add invalid args write test. * Add config delete tests * Add config list tests * Add connect enable tests * Update some CLI commands to use CAS ops This also modifies the HTTP API for a write op to return a boolean indicating whether the value was written or not. * Fix up the HTTP API CAS tests as I realized they weren’t testing what they should. * Update config entry rpc tests to properly test CAS * Fix up a few more tests * Fix some tests that using ConfigEntries.Apply * Update config_write_test.go * Get rid of unused import	6 years ago
R.B. Boyer	b4727fbe18	cli: actually allow the 'connect envoy' and 'watch' subcommands to work with -token-file (#5733)	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
R.B. Boyer	e47d7eeddb	acl: adding support for kubernetes auth provider login (#5600) ... * auth providers * binding rules * auth provider for kubernetes * login/logout	6 years ago
Alvin Huang	8ceca2ace3	Add fmt and vet (#5671) ... * add go fmt and vet * go fmt fixes	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Matt Keeler	766d771017	Pass a testing.T into NewTestAgent and TestAgent.Start (#5342) ... This way we can avoid unnecessary panics which cause other tests not to run. This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent	6 years ago
Grégoire Seux	4f62a3b528	Implement /v1/agent/health/service/<service name> endpoint (#3551) ... This endpoint aggregates all checks related to <service id> on the agent and return an appropriate http code + the string describing the worst check. This allows to cleanly expose service status to other component, hiding complexity of multiple checks. This is especially useful to use consul to feed a load balancer which would delegate health checking to consul agent. Exposing this endpoint on the agent is necessary to avoid a hit on consul servers and avoid decreasing resiliency (this endpoint will work even if there is no consul leader in the cluster).	6 years ago
Hans Hasselberg	acc458d7a4	Builtin tls helper (#5078) ... * command: add tls subcommand * website: update docs and guide	6 years ago
Dhi Aurrahman	649082ab35	connect: Fix comment DYNAMIC_DNS to LOGICAL_DNS (#4799) ... LOGICAL_DNS is one of the supported service discovery types [1]. [1] https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/service_discovery#arch-overview-service-discovery-types	6 years ago
Paul Banks	c9217c958e	merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in	6 years ago
Paul Banks	f9c0f00abb	cli: envoy command default gRPC port (#4768) ... * Default gRPC port; Start on some basic tests for argument and ENV handling; Make Exec test less platform-dependent. * Allow hot-restarts * Remove debug	6 years ago
R.B. Boyer	c310451b2b	cli: avoid passing envoy bootstrap configuration as arguments (#4747) ... Play a trick with CLOEXEC to pass the envoy bootstrap configuration as an open file descriptor to the exec'd envoy process. The file only briefly touches disk before being unlinked. We convince envoy to read from this open file descriptor by using the /dev/fd/$FDNUMBER mechanism to read the open file descriptor as a file. Because the filename no longer has an extension envoy's sniffing logic falls back on JSON instead of YAML, so the bootstrap configuration must be generated as JSON instead.	6 years ago
Paul Banks	dca1303d05	Connect Envoy Command (#4735) ... * Plumb xDS server and proxyxfg into the agent startup * Add `consul connect envoy` command to allow running Envoy as a connect sidecar. * Add test for help tabs; typos and style fixups from review	6 years ago
Paul Banks	e812f5516a	Add -sidecar-for and new /agent/service/:service_id endpoint (#4691) ... - A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it: - **supports hash-based blocking** and so; - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config. - It's not proxy specific and so works for any service. - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested). - The built-in proxy now uses the new endpoint exclusively for it's config - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered. - Several fixes for edge cases for SidecarService - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.	6 years ago
Paul Banks	b83bbf248c	Add Proxy Upstreams to Service Definition (#4639) ... * Refactor Service Definition ProxyDestination. This includes: - Refactoring all internal structs used - Updated tests for both deprecated and new input for: - Agent Services endpoint response - Agent Service endpoint response - Agent Register endpoint - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Register - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Services endpoint response - Catalog Node endpoint response - Catalog Service endpoint response - Updated API tests for all of the above too (both deprecated and new forms of register) TODO: - config package changes for on-disk service definitions - proxy config endpoint - built-in proxy support for new fields * Agent proxy config endpoint updated with upstreams * Config file changes for upstreams. * Add upstream opaque config and update all tests to ensure it works everywhere. * Built in proxy working with new Upstreams config * Command fixes and deprecations * Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts... TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct. * Fix translated keys in API registration. ≈ * Fixes from docs - omit some empty undocumented fields in API - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally. * Documentation updates for Upstreams in service definition * Fixes for tests broken by many refactors. * Enable travis on f-connect branch in this branch too. * Add consistent Deprecation comments to ProxyDestination uses * Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs	6 years ago
Paul Banks	b06ddc9187	Rename proxy package (re-run of #4550) (#4638) ... * Rename agent/proxy package to reflect that it is limited to managed proxy processes Rationale: we have several other components of the agent that relate to Connect proxies for example the ProxyConfigManager component needed for Envoy work. Those things are pretty separate from the focus of this package so far which is only concerned with managing external proxy processes so it's nota good fit to put code for that in here, yet there is a naming clash if we have other packages related to proxy functionality that are not in the `agent/proxy` package. Happy to bikeshed the name. I started by calling it `managedproxy` but `managedproxy.Manager` is especially unpleasant. `proxyprocess` seems good in that it's more specific about purpose but less clearly connected with the concept of "managed proxies". The names in use are cleaner though e.g. `proxyprocess.Manager`. This rename was completed automatically using golang.org/x/tools/cmd/gomvpkg. Depends on #4541 * Fix missed windows tagged files	6 years ago
Pierre Souchay	1a906ef34e	Fix more unstable tests in agent and command	6 years ago
Pierre Souchay	92acdaa94c	Fixed flaky tests (#4626)	6 years ago
Pierre Souchay	3f9d1370b7	Fix unstable test TestRegisterMonitor_heartbeat (#4568)	6 years ago
Pierre Souchay	cec5d72396	BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472) ... - Improve resilience of testrpc.WaitForLeader() - Add additionall retry to CI - Increase "go test" timeout to 8m - Add wait for cluster leader to several tests in the agent package - Add retry to some tests in the api and command packages	6 years ago
Mitchell Hashimoto	e39dfc919a	command/connect/proxy: ignore check doesn't exist on -register	7 years ago
Paul Banks	d1c67d90bc	Fixs a few issues that stopped this working in real life but not caught by tests: ... - Dev mode assumed no persistence of services although proxy state is persisted which caused proxies to be killed on startup as their services were no longer registered. Fixed. - Didn't snapshot the ProxyID which meant that proxies were adopted OK from snapshot but failed to restart if they died since there was no proxyID in the ENV on restart - Dev mode with no persistence just kills all proxies on shutdown since it can't recover them later - Naming things	7 years ago
Mitchell Hashimoto	caae034f3b	command/connect/proxy: can specify prepared query upstream types	7 years ago
Paul Banks	cdc7cfaa36	Abandon daemonize for simpler solution (preserving history): ... Reverts: - bdb274852ae469c89092d6050697c0ff97178465 - 2c689179c4f61c11f0016214c0fc127a0b813bfe - d62e25c4a7ab753914b6baccd66f88ffd10949a3 - c727ffbcc98e3e0bf41e1a7bdd40169bd2d22191 - 31b4d18933fd0acbe157e28d03ad59c2abf9a1fb - 85c3f8df3eabc00f490cd392213c3b928a85aa44	7 years ago
Paul Banks	ba0fb58a72	Make daemoinze an option on test binary without hacks. Misc fixes for racey or broken tests. Still failing on several though.	7 years ago
Paul Banks	2b377dc624	Run daemon processes as a detached child. ... This turns out to have a lot more subtelty than we accounted for. The test suite is especially prone to races now we can only poll the child and many extra levels of indirectoin are needed to correctly run daemon process without it becoming a Zombie. I ran this test suite in a loop with parallel enabled to verify for races (-race doesn't find any as they are logical inter-process ones not actual data races). I made it through ~50 runs before hitting an error due to timing which is much better than before. I want to go back and see if we can do better though. Just getting this up.	7 years ago
Kyle Havlovitz	61d7e38148	Clarify CA commands' help text	7 years ago
Kyle Havlovitz	6ca7543ea6	Add CA CLI commands for getting/setting config	7 years ago
Mitchell Hashimoto	7d24ecd700	command/connect/proxy: register monitor tests	7 years ago
Mitchell Hashimoto	eb2a6952ba	address comment feedback	7 years ago
Mitchell Hashimoto	db10240bea	command/connect/proxy: register monitor and -register flag	7 years ago
Mitchell Hashimoto	1db42050bd	command/connect/proxy: output information when starting similar to agent	7 years ago
Mitchell Hashimoto	55c6d34ea0	command/connect/proxy: detailed help	7 years ago
Mitchell Hashimoto	b1d709fc9f	command/connect/proxy: -service-addr required for -listen	7 years ago
Mitchell Hashimoto	83f2509c6f	command/connect/proxy: can set public listener from flags	7 years ago
Mitchell Hashimoto	c93ba83310	command/connect/proxy: tests for configuration	7 years ago
Mitchell Hashimoto	42ee214c8a	command/connect/proxy: accept -service and -upstream	7 years ago
Mitchell Hashimoto	8c713e6104	connect/proxy: don't require proxy ID	7 years ago
Mitchell Hashimoto	5d969e3cbb	command/connect/proxy: set ACL token based on proxy token flag	7 years ago
Mitchell Hashimoto	867db89303	command/connect/proxy: set proxy ID from env var if set	7 years ago
Paul Banks	10db79c8ae	Rework connect/proxy and command/connect/proxy. End to end demo working again	7 years ago
Paul Banks	69d5efdbbd	Original proxy and connect.Client implementation. Working end to end.	7 years ago