113 Commits (f526dfd0ac65e5d3cbdf533b044ee8315f4101d6)

Author	SHA1	Message	Date
Matt Keeler	766d771017	Pass a testing.T into NewTestAgent and TestAgent.Start (#5342) ... This way we can avoid unnecessary panics which cause other tests not to run. This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791) ... This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Paul Banks	0f27ffd163	Proxy Config Manager (#4729) ... * Proxy Config Manager This component watches for local state changes on the agent and ensures that each service registered locally with Kind == connect-proxy has it's state being actively populated in the cache. This serves two purposes: 1. For the built-in proxy, it ensures that the state needed to accept connections is available in RAM shortly after registration and likely before the proxy actually starts accepting traffic. 2. For (future - next PR) xDS server and other possible future proxies that require _push_ based config discovery, this provides a mechanism to subscribe and be notified about updates to a proxy instance's config including upstream service discovery results. * Address review comments * Better comments; Better delivery of latest snapshot for slow watchers; Embed Config * Comment typos * Add upstream Stringer for funsies	6 years ago
Paul Banks	e812f5516a	Add -sidecar-for and new /agent/service/:service_id endpoint (#4691) ... - A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it: - **supports hash-based blocking** and so; - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config. - It's not proxy specific and so works for any service. - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested). - The built-in proxy now uses the new endpoint exclusively for it's config - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered. - Several fixes for edge cases for SidecarService - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.	6 years ago
Paul Banks	b83bbf248c	Add Proxy Upstreams to Service Definition (#4639) ... * Refactor Service Definition ProxyDestination. This includes: - Refactoring all internal structs used - Updated tests for both deprecated and new input for: - Agent Services endpoint response - Agent Service endpoint response - Agent Register endpoint - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Register - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Services endpoint response - Catalog Node endpoint response - Catalog Service endpoint response - Updated API tests for all of the above too (both deprecated and new forms of register) TODO: - config package changes for on-disk service definitions - proxy config endpoint - built-in proxy support for new fields * Agent proxy config endpoint updated with upstreams * Config file changes for upstreams. * Add upstream opaque config and update all tests to ensure it works everywhere. * Built in proxy working with new Upstreams config * Command fixes and deprecations * Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts... TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct. * Fix translated keys in API registration. ≈ * Fixes from docs - omit some empty undocumented fields in API - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally. * Documentation updates for Upstreams in service definition * Fixes for tests broken by many refactors. * Enable travis on f-connect branch in this branch too. * Add consistent Deprecation comments to ProxyDestination uses * Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs	6 years ago
Pierre Souchay	eddcf228ea	Implementation of Weights Data structures (#4468) ... * Implementation of Weights Data structures Adding this datastructure will allow us to resolve the issues #1088 and #4198 This new structure defaults to values: ``` { Passing: 1, Warning: 0 } ``` Which means, use weight of 0 for a Service in Warning State while use Weight 1 for a Healthy Service. Thus it remains compatible with previous Consul versions. * Implemented weights for DNS SRV Records * DNS properly support agents with weight support while server does not (backwards compatibility) * Use Warning value of Weights of 1 by default When using DNS interface with only_passing = false, all nodes with non-Critical healthcheck used to have a weight value of 1. While having weight.Warning = 0 as default value, this is probably a bad idea as it breaks ascending compatibility. Thus, we put a default value of 1 to be consistent with existing behaviour. * Added documentation for new weight field in service description * Better documentation about weights as suggested by @banks * Return weight = 1 for unknown Check states as suggested by @banks * Fixed typo (of -> or) in error message as requested by @mkeeler * Fixed unstable unit test TestRetryJoin * Fixed unstable tests * Fixed wrong Fatalf format in `testrpc/wait.go` * Added notes regarding DNS SRV lookup limitations regarding number of instances * Documentation fixes and clarification regarding SRV records with weights as requested by @banks * Rephrase docs	6 years ago
Siva Prasad	b1a34f899f	TestAgentAntiEntropy: Wait until Consul service is up on the agent. (#4591) ... * Anti-Entropy test wait for Consul service added * Reverted some tests back to using WaitForLeader	6 years ago
Pierre Souchay	cec5d72396	BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472) ... - Improve resilience of testrpc.WaitForLeader() - Add additionall retry to CI - Increase "go test" timeout to 8m - Add wait for cluster leader to several tests in the agent package - Add retry to some tests in the api and command packages	6 years ago
Mitchell Hashimoto	b3854fdd28	agent/local: silly spacing on select statements	6 years ago
Mitchell Hashimoto	8c72bb0cdf	agent/local: address remaining test feedback	6 years ago
Mitchell Hashimoto	9f128e40d6	agent/local: don't use time.After in test since notify is instant	6 years ago
Mitchell Hashimoto	7543d270e2	agent/local: support local alias checks	6 years ago
Paul Banks	df2cb30b01	Make tests pass and clean proxy persistence. No detached child changes yet. ... This is a good state for persistence stuff to re-start the detached child work that got mixed up last time.	7 years ago
Paul Banks	cdc7cfaa36	Abandon daemonize for simpler solution (preserving history): ... Reverts: - bdb274852ae469c89092d6050697c0ff97178465 - 2c689179c4f61c11f0016214c0fc127a0b813bfe - d62e25c4a7ab753914b6baccd66f88ffd10949a3 - c727ffbcc98e3e0bf41e1a7bdd40169bd2d22191 - 31b4d18933fd0acbe157e28d03ad59c2abf9a1fb - 85c3f8df3eabc00f490cd392213c3b928a85aa44	7 years ago
Paul Banks	ba0fb58a72	Make daemoinze an option on test binary without hacks. Misc fixes for racey or broken tests. Still failing on several though.	7 years ago
Mitchell Hashimoto	3d3eee2f6e	agent: resolve some conflicts and fix tests	7 years ago
Mitchell Hashimoto	2bd39a84a6	agent/local: add Notify mechanism for proxy changes	7 years ago
Mitchell Hashimoto	476ea7b04a	agent: start/stop proxies	7 years ago
Paul Banks	e0e12e165b	TLS watching integrated into Service with some basic tests. ... There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.	7 years ago
Paul Banks	730da74369	Fix various test failures and vet warnings. ... Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.	7 years ago
Paul Banks	2a69663448	Agent Connect Proxy config endpoint with hash-based blocking	7 years ago
Paul Banks	e6071051cf	Added connect proxy config and local agent state setup on boot.	7 years ago
Mitchell Hashimoto	9781cb1ace	agent/local: anti-entropy for connect proxy services	7 years ago
James Phillips	2937656f8e	Adds a longer retry period for the AE deferred output test. ... There's some justification in the comments about this and a TODO to improve this later. Fixes #3668	7 years ago
Frank Schroeder	1cb8b0ffe3	local state: fix go vet issue	7 years ago
Frank Schroeder	7335c34c32	local state: remove stale comment	7 years ago
Frank Schroeder	3d547e30c7	local state: make test more robust	7 years ago
Frank Schroeder	58d52ac580	local state: rename Add{Check,Service}State to Set{Check,Service}State	7 years ago
Frank Schroeder	de57b16d99	local state: address review comments ... * move non-blocking notification mechanism into ae.Trigger * move Pause/Resume into separate type	7 years ago
Frank Schroeder	5c77c59501	local state: refactor TestAgentAntiEntropy_EnableTagOverride ... Make intent clearer by being more explicit and adding some comments. Use verify.Values to compare service entries.	7 years ago
Frank Schroeder	524981f367	local state: fix TestAgentAntiEntropy_EnableTagOverride ... The test had a race condition where it relied on the first service to be synced to the remote catalog which sometimes failed.	7 years ago
Frank Schroeder	1181aaee6a	local state: rename tests	7 years ago
Frank Schroeder	1602ac56b5	local state: drop retry loops from tests ... Since the tests are now using synchronous calls for state syncing we no longer need to use retry loops to wait for the changes to propagate.	7 years ago
Frank Schroeder	71c74e62c7	local state: fix anti-entropy state tests ... The anti-entropy tests relied on the side-effect of the StartSync() method to perform a full sync instead of a partial sync. This lead to multiple anti-entropy go routines being started unnecessary retry loops. This change changes the behavior to perform synchronous full syncs when necessary removing the need for all of the time.Sleep and most of the retry loops.	7 years ago
Frank Schroeder	37b95ef98e	local state: fix test with updated error message	7 years ago
Frank Schroeder	b803bf3091	local state: tests compile	7 years ago
Frank Schroeder	6027a9e2a5	local state: move to separate package ... This patch moves the local state to a separate package to further decouple it from the agent code. The code compiles but the tests do not yet.	7 years ago
Frank Schroeder	67a0689f71	Revert "local state: move to separate package" ... This reverts commit d447e823c6.	7 years ago
Frank Schroeder	9ed4b2d631	Revert "local state: tests compile" ... This reverts commit 1af52bf7be.	7 years ago
Frank Schroeder	5ae4c52ca6	Revert "local state: fix test with updated error message" ... This reverts commit e9149f64d9.	7 years ago
Frank Schroeder	67cdfc038e	Revert "local state: fix anti-entropy state tests" ... This reverts commit f8e20cd996.	7 years ago
Frank Schroeder	556bf3f85d	Revert "local state: drop retry loops from tests" ... This reverts commit 2bdba8ab06.	7 years ago
Frank Schroeder	a3aa864d5b	Revert "local state: rename tests" ... This reverts commit ff62eaf063.	7 years ago
Frank Schroeder	39615cb57b	Revert "local state: fix TestAgentAntiEntropy_EnableTagOverride" ... This reverts commit 86f7ea6013.	7 years ago
Frank Schroeder	9fdea75d99	Revert "local state: refactor TestAgentAntiEntropy_EnableTagOverride" ... This reverts commit c28e23eac8.	7 years ago
Frank Schroeder	46641e44d9	Revert "local state: address review comments" ... This reverts commit 1d315075b1.	7 years ago
Frank Schroeder	fa45a6a547	Revert "local state: rename Add{Check,Service}State to Set{Check,Service}State" ... This reverts commit 9280841a80.	7 years ago
Frank Schroeder	42af4cdc70	Revert "local state: make test more robust" ... This reverts commit f9267380db.	7 years ago
Frank Schroeder	4ae8317fbf	Revert "local state: remove stale comment" ... This reverts commit 35f4acdddc.	7 years ago
Frank Schroeder	223fae7d92	Revert "fix go vet issue" ... This reverts commit 078e50b635.	7 years ago
Frank Schroeder	078e50b635	fix go vet issue	7 years ago
Frank Schroeder	35f4acdddc	local state: remove stale comment	7 years ago
Frank Schroeder	f9267380db	local state: make test more robust	7 years ago
Frank Schroeder	9280841a80	local state: rename Add{Check,Service}State to Set{Check,Service}State	7 years ago
Frank Schroeder	1d315075b1	local state: address review comments ... * move non-blocking notification mechanism into ae.Trigger * move Pause/Resume into separate type	7 years ago
Frank Schroeder	c28e23eac8	local state: refactor TestAgentAntiEntropy_EnableTagOverride ... Make intent clearer by being more explicit and adding some comments. Use verify.Values to compare service entries.	7 years ago
Frank Schroeder	86f7ea6013	local state: fix TestAgentAntiEntropy_EnableTagOverride ... The test had a race condition where it relied on the first service to be synced to the remote catalog which sometimes failed.	7 years ago
Frank Schroeder	ff62eaf063	local state: rename tests	7 years ago
Frank Schroeder	2bdba8ab06	local state: drop retry loops from tests ... Since the tests are now using synchronous calls for state syncing we no longer need to use retry loops to wait for the changes to propagate.	7 years ago
Frank Schroeder	f8e20cd996	local state: fix anti-entropy state tests ... The anti-entropy tests relied on the side-effect of the StartSync() method to perform a full sync instead of a partial sync. This lead to multiple anti-entropy go routines being started unnecessary retry loops. This change changes the behavior to perform synchronous full syncs when necessary removing the need for all of the time.Sleep and most of the retry loops.	7 years ago
Frank Schroeder	e9149f64d9	local state: fix test with updated error message	7 years ago
Frank Schroeder	1af52bf7be	local state: tests compile	7 years ago
Frank Schroeder	d447e823c6	local state: move to separate package ... This patch moves the local state to a separate package to further decouple it from the agent code. The code compiles but the tests do not yet.	7 years ago