consul

Commit Graph

Author	SHA1	Message	Date
Daniel Nephin	068b43df90	Enable gofmt simplify Code changes done automatically with 'gofmt -s -w'	5 years ago
Chris Piraino	91ab89dd48	Move ingress param to a new endpoint (#8081 ) In discussion with team, it was pointed out that query parameters tend to be filter mechanism, and that semantically the "/v1/health/connect" endpoint should return "all healthy connect-enabled endpoints (e.g. could be side car proxies or native instances) for this service so I can connect with mTLS". That does not fit an ingress gateway, so we remove the query parameter and add a new endpoint "/v1/health/ingress" that semantically means "all the healthy ingress gateway instances that I can connect to to access this connect-enabled service without mTLS"	5 years ago
Chris Piraino	3c037d9b96	Add ?ingress query parameter on /v1/health/connect Refactor boolean query parameter logic from ?passing value to re-use with ingress	5 years ago
Daniel Nephin	475659a132	Remove name from NewTestAgent Using: git grep -l 'NewTestAgent(t, t.Name(),' \| \ xargs sed -i -e 's/NewTestAgent(t, t.Name(),/NewTestAgent(t,/g'	5 years ago
Chris Piraino	7d4d416a4d	test: unflake two TestHealthServiceNode_* tests Replaces WaitForLeader with WaitForTestAgent. This waits to make sure that the node itself is correctly registered in the catalog before attempting additional registrations.	5 years ago
Freddy	60f6ec0c2f	Store check type in catalog (#6561 )	5 years ago
Freddy	5873c56a03	Flaky test overhaul (#6100 )	5 years ago
Matt Keeler	f3d9b999ee	Add tagged addresses for services (#5965 ) This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579 ) Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568 ) * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517 ) * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Matt Keeler	766d771017	Pass a testing.T into NewTestAgent and TestAgent.Start (#5342 ) This way we can avoid unnecessary panics which cause other tests not to run. This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent	6 years ago
Paul Banks	b83bbf248c	Add Proxy Upstreams to Service Definition (#4639 ) * Refactor Service Definition ProxyDestination. This includes: - Refactoring all internal structs used - Updated tests for both deprecated and new input for: - Agent Services endpoint response - Agent Service endpoint response - Agent Register endpoint - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Register - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Services endpoint response - Catalog Node endpoint response - Catalog Service endpoint response - Updated API tests for all of the above too (both deprecated and new forms of register) TODO: - config package changes for on-disk service definitions - proxy config endpoint - built-in proxy support for new fields * Agent proxy config endpoint updated with upstreams * Config file changes for upstreams. * Add upstream opaque config and update all tests to ensure it works everywhere. * Built in proxy working with new Upstreams config * Command fixes and deprecations * Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts... TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct. * Fix translated keys in API registration. ≈ * Fixes from docs - omit some empty undocumented fields in API - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally. * Documentation updates for Upstreams in service definition * Fixes for tests broken by many refactors. * Enable travis on f-connect branch in this branch too. * Add consistent Deprecation comments to ProxyDestination uses * Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs	6 years ago
Paul Banks	88388d760d	Support Agent Caching for Service Discovery Results (#4541 ) * Add cache types for catalog/services and health/services and basic test that caching works * Support non-blocking cache types with Cache-Control semantics. * Update API docs to include caching info for every endpoint. * Comment updates per PR feedback. * Add note on caching to the 10,000 foot view on the architecture page to make the new data path more clear. * Document prepared query staleness quirk and force all background requests to AllowStale so we can spread service discovery load across servers.	6 years ago
Pierre Souchay	1a906ef34e	Fix more unstable tests in agent and command	6 years ago
Pierre Souchay	22500f242e	Fix unstable tests in agent, api, and command/watch	6 years ago
Pierre Souchay	eddcf228ea	Implementation of Weights Data structures (#4468 ) * Implementation of Weights Data structures Adding this datastructure will allow us to resolve the issues #1088 and #4198 This new structure defaults to values: ``` { Passing: 1, Warning: 0 } ``` Which means, use weight of 0 for a Service in Warning State while use Weight 1 for a Healthy Service. Thus it remains compatible with previous Consul versions. * Implemented weights for DNS SRV Records * DNS properly support agents with weight support while server does not (backwards compatibility) * Use Warning value of Weights of 1 by default When using DNS interface with only_passing = false, all nodes with non-Critical healthcheck used to have a weight value of 1. While having weight.Warning = 0 as default value, this is probably a bad idea as it breaks ascending compatibility. Thus, we put a default value of 1 to be consistent with existing behaviour. * Added documentation for new weight field in service description * Better documentation about weights as suggested by @banks * Return weight = 1 for unknown Check states as suggested by @banks * Fixed typo (of -> or) in error message as requested by @mkeeler * Fixed unstable unit test TestRetryJoin * Fixed unstable tests * Fixed wrong Fatalf format in `testrpc/wait.go` * Added notes regarding DNS SRV lookup limitations regarding number of instances * Documentation fixes and clarification regarding SRV records with weights as requested by @banks * Rephrase docs	6 years ago
Pierre Souchay	cec5d72396	BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472 ) - Improve resilience of testrpc.WaitForLeader() - Add additionall retry to CI - Increase "go test" timeout to 8m - Add wait for cluster leader to several tests in the agent package - Add retry to some tests in the api and command packages	6 years ago
Mitchell Hashimoto	4d852e62a3	agent: address PR feedback	7 years ago
Mitchell Hashimoto	368137b81b	agent: /v1/health/connect/:service	7 years ago
Frank Schröder	12216583a1	New config parser, HCL support, multiple bind addrs (#3480 ) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt	7 years ago
Frank Schroeder	1acff3533e	agent: move agent/consul/structs to agent/structs	7 years ago
Frank Schroeder	1c75cf1af5	pkg refactor command/agent/* -> agent/* command/consul/* -> agent/consul/* command/agent/command{,_test}.go -> command/agent{,_test}.go command/base/command.go -> command/base.go command/base/* -> command/* commands.go -> command/commands.go The script which did the refactor is: ( cd $GOPATH/src/github.com/hashicorp/consul git mv command/agent/command.go command/agent.go git mv command/agent/command_test.go command/agent_test.go git mv command/agent/flag_slice_value{,_test}.go command/ git mv command/agent . git mv command/base/command.go command/base.go git mv command/base/config_util{,_test}.go command/ git mv commands.go command/ git mv consul agent rmdir command/base/ gsed -i -e 's\|package agent\|package command\|' command/agent{,_test}.go gsed -i -e 's\|package agent\|package command\|' command/flag_slice_value{,_test}.go gsed -i -e 's\|package base\|package command\|' command/base.go command/config_util{,_test}.go gsed -i -e 's\|package main\|package command\|' command/commands.go gsed -i -e 's\|base.Command\|BaseCommand\|' command/commands.go gsed -i -e 's\|agent.Command\|AgentCommand\|' command/commands.go gsed -i -e 's\|\tCommand:\|\tBaseCommand:\|' command/commands.go gsed -i -e 's\|base\.\|\|' command/commands.go gsed -i -e 's\|command\.\|\|' command/commands.go gsed -i -e 's\|command\|c\|' main.go gsed -i -e 's\|range Commands\|range command.Commands\|' main.go gsed -i -e 's\|Commands: Commands\|Commands: command.Commands\|' main.go gsed -i -e 's\|base\.BoolValue\|BoolValue\|' command/operator_autopilot_set.go gsed -i -e 's\|base\.DurationValue\|DurationValue\|' command/operator_autopilot_set.go gsed -i -e 's\|base\.StringValue\|StringValue\|' command/operator_autopilot_set.go gsed -i -e 's\|base\.UintValue\|UintValue\|' command/operator_autopilot_set.go gsed -i -e 's\|\bCommand\b\|BaseCommand\|' command/base.go gsed -i -e 's\|BaseCommand Options\|Command Options\|' command/base.go gsed -i -e 's\|base.Command\|BaseCommand\|' command/.go gsed -i -e 's\|c\.Command\|c.BaseCommand\|g' command/.go gsed -i -e 's\|\tCommand:\|\tBaseCommand:\|' command/_test.go gsed -i -e 's\|base\.\|\|' command/_test.go gsed -i -e 's\|\bCommand\b\|AgentCommand\|' command/agent{,_test}.go gsed -i -e 's\|cmd.AgentCommand\|cmd.BaseCommand\|' command/agent.go gsed -i -e 's\|cli.AgentCommand = new(Command)\|cli.Command = new(AgentCommand)\|' command/agent_test.go gsed -i -e 's\|exec.AgentCommand\|exec.Command\|' command/agent_test.go gsed -i -e 's\|exec.BaseCommand\|exec.Command\|' command/agent_test.go gsed -i -e 's\|NewTestAgent\|agent.NewTestAgent\|' command/agent_test.go gsed -i -e 's\|= TestConfig\|= agent.TestConfig\|' command/agent_test.go gsed -i -e 's\|: RetryJoin\|: agent.RetryJoin\|' command/agent_test.go gsed -i -e 's\|\.\./\.\./\|../\|' command/config_util_test.go gsed -i -e 's\|\bverifyUniqueListeners\|VerifyUniqueListeners\|' agent/config{,_test}.go command/agent.go gsed -i -e 's\|\bserfLANKeyring\b\|SerfLANKeyring\|g' agent/{agent,keyring,testagent}.go command/agent.go gsed -i -e 's\|\bserfWANKeyring\b\|SerfWANKeyring\|g' agent/{agent,keyring,testagent}.go command/agent.go gsed -i -e 's\|\bNewAgent\b\|agent.New\|g' command/agent{,_test}.go gsed -i -e 's\|\bNewAgent\|New\|' agent/{acl_test,agent,testagent}.go gsed -i -e 's\|\bAgent\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bBool\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bConfig\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bDefaultConfig\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bDevConfig\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bMergeConfig\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bReadConfigPaths\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bParseMetaPair\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bSerfLANKeyring\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|\bSerfWANKeyring\b\|agent.&\|g' command/agent{,_test}.go gsed -i -e 's\|circonus\.agent\|circonus\|g' command/agent{,_test}.go gsed -i -e 's\|logger\.agent\|logger\|g' command/agent{,_test}.go gsed -i -e 's\|metrics\.agent\|metrics\|g' command/agent{,_test}.go gsed -i -e 's\|// agent.Agent\|// agent\|' command/agent{,_test}.go gsed -i -e 's\|a\.agent\.Config\|a.Config\|' command/agent{,_test}.go gsed -i -e 's\|agent\.AppendSliceValue\|AppendSliceValue\|' command/{configtest,validate}.go gsed -i -e 's\|consul/consul\|agent/consul\|' GNUmakefile gsed -i -e 's\|\.\./test\|../../test\|' agent/consul/server_test.go # fix imports f=$(grep -rl 'github.com/hashicorp/consul/command/agent' * \| grep '\.go') gsed -i -e 's\|github.com/hashicorp/consul/command/agent\|github.com/hashicorp/consul/agent\|' $f goimports -w $f f=$(grep -rl 'github.com/hashicorp/consul/consul' * \| grep '\.go') gsed -i -e 's\|github.com/hashicorp/consul/consul\|github.com/hashicorp/consul/agent/consul\|' $f goimports -w $f goimports -w command/*.go main.go )	8 years ago
Seth Vargo	ee1b5d5024	Update comment	8 years ago
Seth Vargo	532f8d1435	Parse values given to ?passing in the API This PR fixes GH-2212 in the most backwards-compatible way I can think of. If the user does not pass a value for `?passing`, it's assumed to be true, which mirrors the current behavior. However, if the user passes any value for passing, that value is parsed as a bool using strconv. It's important to note that this is technically a breaking change. Previously using `?passing=false` would return only passing nodes. While this behavior is obviously incorrect, it was the previous behavior. We should call this out very clearly in the CHANGELOG.	8 years ago
Frank Schroeder	141efd9588	test: rename *Config vars to 'cfg' 'c' is ambigous since Command also uses this and we want to use 'config' as a package name.	8 years ago
Frank Schroeder	0be63d7060	test: refactor httpTest with TestAgent	8 years ago
Frank Schroeder	308f9929b3	test: run agent tests in parallel This brings down the test run from 108 sec to 15 sec. There is an occasional port conflict because of the nature the next port is chosen. So far it seems rare enough to live with it.	8 years ago
Frank Schroeder	23a6ff383c	agent: refactor tests for TestAgent Refactored tests that use * makeAgentXXX * makeDNSServerXXX * makeHTTPServerXXX	8 years ago
Frank Schroeder	82650f73e3	agent: move http/dns endpoints into agent Move the HTTP and DNS endpoints into the agent and control their lifespan via the agent. This removes the requirement to manage HTTP and DNS servers indpendent of the agent since the agent is mostly useless without an endpoint and the endpoints without the agent.	8 years ago
Frank Schroeder	1e89692cc1	test: drop error check on http.NewRequest Most URLs are static so the error check is redundant. The subsequent test wouldn't work if the url is wrong.	8 years ago
Frank Schroeder	2794db5aac	test: Fix badly formatted retry.Run tests	8 years ago
James Phillips	ddfa57765c	retry: Removes the description parameter.	8 years ago
James Phillips	2de1cd5a4a	test: Turns off ACLs for catalog and health WAN translation tests. Since this was doing registration to a foreign DC, it needs extra time for the route to the ACL datacenter to be set up. ACLs aren't part of this test, so by disabling them we make this more reliable and converge faster than if we had added a retry.	8 years ago
Frank Schroeder	21a82a0a16	test: Refactor WaitForResult tests with retry Refactor tests that use testutil.WaitForResult to use retry. Since this requires refactoring the test functions in general this patch also shows the use of the github.com/pascaldekloe/goe/verify library which provides a good mechanism for comparing nested data structures. Instead of just converting the tests from testutil.WaitForResult to retry the tests that performing a nested comparison of data structures are converted to the verify library at the same time.	8 years ago
Frank Schroeder	48fa2962eb	Revert "test: Run command/agent tests in parallel" This reverts commit `17be40a733`.	8 years ago
Frank Schroeder	17be40a733	test: Run command/agent tests in parallel	8 years ago
Frank Schroeder	eddb1af603	Remove duplicate constants This patch removes duplicate internal copies of constants in the structs package which are also defined in the api package. The api.KVOp type with all its values for the TXN endpoint and the api.HealthXXX constants are now used throughout the codebase. This resulted in some circular dependencies in the testutil package which have been resolved by copying code and constants and moving the WaitForLeader function into a separate testrpc package.	8 years ago
Seth Vargo	0170a28ec9	Use new APIs	8 years ago
Kyle Havlovitz	5acd69b4fc	Add node metadata filtering to remaining health/catalog endpoints	8 years ago
Kyle Havlovitz	3add830b3a	Test fixes (#2434 ) * Test fixes in health_endpoint_test.go, agent_endpoint_test.go and rtt_test.go * Don't reuse the same agent config in TestAgent_ReconnectConfigSettings	8 years ago
James Phillips	55e83c9e1c	Tweaks the WAN address translation unit tests.	8 years ago
David van Geest	cdeff022dd	Translate Address to tagged WAN address in HTTP API when appropriate.	8 years ago
James Phillips	cd7b3d4b49	Widens coordinate update sleeps in unit tests.	9 years ago
James Phillips	c248b0017a	Fixes nil slices from HTTP endpoints. These would manifest in the HTTP output as Javascript nulls instead of empty lists, so we had unintentionally changed the interface while porting to the new state store. We added code to each HTTP endpoint to convert nil slices to empty ones so they JSON-ify properly, and we added tests to catch this in the future.	9 years ago
James Phillips	33e3505aea	Adds endpoints for raw network coordinates.	9 years ago
James Phillips	9caa5b3653	Adds distance sorting to health endpoint. Cleans up unit tests.	9 years ago
Armon Dadgar	135c409573	agent: Fixing passing filter. Fixes #241	11 years ago
William Tisäter	3f361972ac	Remove TODO comments	11 years ago
Armon Dadgar	1d26ea41e1	agent: Improving test reliability	11 years ago

24 Commits (262fb3a0c382ff5434d2787dbbce007942532b47)