Commit Graph

148 Commits (3df68751f5da251f32f4cada0648ac0598de476a)

Author SHA1 Message Date
R.B. Boyer ef559dfdd4
agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
Daniel Nephin b4b85bd83a agent: fix agent logging
Remove the leading whitespace on every log line. This was causing problems for
a customer because their logging system was interpretting the logs as a single
multi-line log.
2021-06-02 17:15:12 -04:00
Daniel Nephin e573641995 cmd: remove unnecessary args to agent.New
The version args are static and passed in from the caller. Instead read
the static values in New.

The shutdownCh was never closed, so did nothing. Remove it as a field
and an arg.
2021-06-02 16:29:29 -04:00
Daniel Nephin b9e60c0775 testing: skip slow tests with -short
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
2020-12-07 13:42:55 -05:00
Daniel Nephin d68edcecf4 testing: Remove all the defer os.Removeall
Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage
the removal
2020-08-14 19:58:53 -04:00
Matt Keeler 3dbbd2d37d
Implement Client Agent Auto Config
There are a couple of things in here.

First, just like auto encrypt, any Cluster.AutoConfig RPC will implicitly use the less secure RPC mechanism.

This drastically modifies how the Consul Agent starts up and moves most of the responsibilities (other than signal handling) from the cli command and into the Agent.
2020-06-17 16:49:46 -04:00
Daniel Nephin 85e0338136 config: remove Args field from Flags
This field was populated for one reason, to test that it was empty.
Of all the callers, only a single one used this functionality. The rest
constructed a `Flags{}` struct which did not set Args.

I think this shows that the logic was in the wrong place. Only the agent
command needs to care about validating the args.

This commit removes the field, and moves the logic to the one caller
that cares.

Also fix some comments.
2020-06-16 12:49:53 -04:00
Daniel Nephin 475659a132 Remove name from NewTestAgent
Using:

git grep -l 'NewTestAgent(t, t.Name(),' | \
    xargs sed -i -e 's/NewTestAgent(t, t.Name(),/NewTestAgent(t,/g'
2020-03-31 16:13:44 -04:00
Chris Piraino 401221de58
Allow users to configure either unstructured or JSON logging (#7130)
* hclog Allow users to choose between unstructured and JSON logging
2020-01-28 17:50:41 -06:00
Freddy 5873c56a03
Flaky test overhaul (#6100) 2019-07-12 09:52:26 -06:00
Jeff Mitchell 4243c3ae42
Move internal/ to sdk/ (#5568)
* Move internal/ to sdk/

* Add a readme to the SDK folder
2019-03-27 08:54:56 -04:00
Jeff Mitchell 47c390025b
Convert to Go Modules (#5517)
* First conversion

* Use serf 0.8.2 tag and associated updated deps

* * Move freeport and testutil into internal/

* Make internal/ its own module

* Update imports

* Add replace statements so API and normal Consul code are
self-referencing for ease of development

* Adapt to newer goe/values

* Bump to new cleanhttp

* Fix ban nonprintable chars test

* Update lock bad args test

The error message when the duration cannot be parsed changed in Go 1.12
(ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test.

* Update another test as well

* Bump travis

* Bump circleci

* Bump go-discover and godo to get rid of launchpad dep

* Bump dockerfile go version

* fix tar command

* Bump go-cleanhttp
2019-03-26 17:04:58 -04:00
Matt Keeler 766d771017
Pass a testing.T into NewTestAgent and TestAgent.Start (#5342)
This way we can avoid unnecessary panics which cause other tests not to run.

This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent
2019-02-14 10:59:14 -05:00
Pierre Souchay eddcf228ea Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Frank Schroeder 9872475167
agent: fix TestRetryJoin 2017-10-24 20:35:37 +02:00
Frank Schroeder 7d3467176b
agent: fix TestRetryJoinFail 2017-10-24 20:35:37 +02:00
Frank Schroeder 96a584dee2
agent: fix TestRetryJoinWanFail 2017-10-24 20:35:36 +02:00
Frank Schroeder b97ab367f4
config: return error on extra command line arguments (#3397)
The `consul agent` command was ignoring extra command line arguments
which can lead to confusion when the user has for example forgotten to
add a dash in front of an argument or is not using an `=` when setting
boolean flags to `true`. `-bootstrap true` is not the same as
`-bootstrap=true`, for example.

Since all command line flags are known and we don't expect unparsed
arguments we can return an error. However, this may make it slightly
more difficult in the future if we ever wanted to have these kinds of
arguments.

Fixes #3397
2017-10-23 08:07:48 +02:00
Frank Schroeder 1b026de1db commands: run all tests in parallel (again) 2017-10-18 00:08:45 +02:00
Frank Schroeder d54a5ef8d8 commands: do not run cmd tests in parallel
Package level parallelization is sufficient.
2017-10-18 00:08:45 +02:00
Frank Schroeder 31d5fb7f47 commands: move agent command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 1c75cf1af5 pkg refactor
command/agent/*                  -> agent/*
    command/consul/*                 -> agent/consul/*
    command/agent/command{,_test}.go -> command/agent{,_test}.go
    command/base/command.go          -> command/base.go
    command/base/*                   -> command/*
    commands.go                      -> command/commands.go

The script which did the refactor is:

(
	cd $GOPATH/src/github.com/hashicorp/consul
	git mv command/agent/command.go command/agent.go
	git mv command/agent/command_test.go command/agent_test.go
	git mv command/agent/flag_slice_value{,_test}.go command/
	git mv command/agent .
	git mv command/base/command.go command/base.go
	git mv command/base/config_util{,_test}.go command/
	git mv commands.go command/
	git mv consul agent
	rmdir command/base/

	gsed -i -e 's|package agent|package command|' command/agent{,_test}.go
	gsed -i -e 's|package agent|package command|' command/flag_slice_value{,_test}.go
	gsed -i -e 's|package base|package command|' command/base.go command/config_util{,_test}.go
	gsed -i -e 's|package main|package command|' command/commands.go

	gsed -i -e 's|base.Command|BaseCommand|' command/commands.go
	gsed -i -e 's|agent.Command|AgentCommand|' command/commands.go
	gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/commands.go
	gsed -i -e 's|base\.||' command/commands.go
	gsed -i -e 's|command\.||' command/commands.go

	gsed -i -e 's|command|c|' main.go
	gsed -i -e 's|range Commands|range command.Commands|' main.go
	gsed -i -e 's|Commands: Commands|Commands: command.Commands|' main.go

	gsed -i -e 's|base\.BoolValue|BoolValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.DurationValue|DurationValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.StringValue|StringValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.UintValue|UintValue|' command/operator_autopilot_set.go

	gsed -i -e 's|\bCommand\b|BaseCommand|' command/base.go
	gsed -i -e 's|BaseCommand Options|Command Options|' command/base.go
	gsed -i -e 's|base.Command|BaseCommand|' command/*.go
	gsed -i -e 's|c\.Command|c.BaseCommand|g' command/*.go
	gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/*_test.go
	gsed -i -e 's|base\.||' command/*_test.go

	gsed -i -e 's|\bCommand\b|AgentCommand|' command/agent{,_test}.go
	gsed -i -e 's|cmd.AgentCommand|cmd.BaseCommand|' command/agent.go

	gsed -i -e 's|cli.AgentCommand = new(Command)|cli.Command = new(AgentCommand)|' command/agent_test.go
	gsed -i -e 's|exec.AgentCommand|exec.Command|' command/agent_test.go
	gsed -i -e 's|exec.BaseCommand|exec.Command|' command/agent_test.go
	gsed -i -e 's|NewTestAgent|agent.NewTestAgent|' command/agent_test.go
	gsed -i -e 's|= TestConfig|= agent.TestConfig|' command/agent_test.go
	gsed -i -e 's|: RetryJoin|: agent.RetryJoin|' command/agent_test.go

	gsed -i -e 's|\.\./\.\./|../|' command/config_util_test.go

	gsed -i -e 's|\bverifyUniqueListeners|VerifyUniqueListeners|' agent/config{,_test}.go command/agent.go
	gsed -i -e 's|\bserfLANKeyring\b|SerfLANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
	gsed -i -e 's|\bserfWANKeyring\b|SerfWANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
	gsed -i -e 's|\bNewAgent\b|agent.New|g' command/agent{,_test}.go
	gsed -i -e 's|\bNewAgent|New|' agent/{acl_test,agent,testagent}.go

	gsed -i -e 's|\bAgent\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bBool\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bDefaultConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bDevConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bMergeConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bReadConfigPaths\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bParseMetaPair\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bSerfLANKeyring\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bSerfWANKeyring\b|agent.&|g' command/agent{,_test}.go

	gsed -i -e 's|circonus\.agent|circonus|g' command/agent{,_test}.go
	gsed -i -e 's|logger\.agent|logger|g' command/agent{,_test}.go
	gsed -i -e 's|metrics\.agent|metrics|g' command/agent{,_test}.go
	gsed -i -e 's|// agent.Agent|// agent|' command/agent{,_test}.go
	gsed -i -e 's|a\.agent\.Config|a.Config|' command/agent{,_test}.go

	gsed -i -e 's|agent\.AppendSliceValue|AppendSliceValue|' command/{configtest,validate}.go

	gsed -i -e 's|consul/consul|agent/consul|' GNUmakefile

	gsed -i -e 's|\.\./test|../../test|' agent/consul/server_test.go

	# fix imports
	f=$(grep -rl 'github.com/hashicorp/consul/command/agent' * | grep '\.go')
	gsed -i -e 's|github.com/hashicorp/consul/command/agent|github.com/hashicorp/consul/agent|' $f
	goimports -w $f

	f=$(grep -rl 'github.com/hashicorp/consul/consul' * | grep '\.go')
	gsed -i -e 's|github.com/hashicorp/consul/consul|github.com/hashicorp/consul/agent/consul|' $f
	goimports -w $f

	goimports -w command/*.go main.go
)
2017-06-10 18:52:45 +02:00
Frank Schroeder c1e6a77c75
test: start secondary agents also via TestAgent 2017-05-31 10:56:19 +02:00
Frank Schroeder 56cf015117
test: add test for starting/stopping lots of agents 2017-05-31 00:29:28 +02:00
Frank Schroeder 141efd9588
test: rename *Config vars to 'cfg'
'c' is ambigous since Command also uses this
and we want to use 'config' as a package name.
2017-05-31 00:29:24 +02:00
Frank Schroeder 308f9929b3
test: run agent tests in parallel
This brings down the test run from 108 sec to 15 sec.

There is an occasional port conflict because of the nature
the next port is chosen. So far it seems rare enough to live
with it.
2017-05-31 00:29:23 +02:00
Frank Schroeder 23a6ff383c
agent: refactor tests for TestAgent
Refactored tests that use

 * makeAgentXXX
 * makeDNSServerXXX
 * makeHTTPServerXXX
2017-05-31 00:29:23 +02:00
Frank Schroeder 3865f14a25
agent: simplify agent creation
This patch creates an agent with just a config struct
and allows for other fields to be set as required.
2017-05-31 00:29:23 +02:00
Frank Schroeder 8ad66f4bea
agent: support custom check id and name
This patch adds support for a custom check id and name when
registering a service.

This is achieved by adding a CheckID and a Name field to the
CheckType structure which is used to register checks with a
service and when returning health check definitions.

CheckDefinition is a superset of CheckType which duplicates
some of the fields of CheckType. This patch decouples these
two structures by removing the embedding of CheckType in
CheckDefinition.

Fixes #3047
2017-05-17 20:17:08 +02:00
Frank Schroeder e2c37b47ee agent: Replace client/server with delegate interface
This patch adds a new internal interface clientServer
which defines the common methods of consul.Client and
consul.Server. This allows to replace the following
code

    if a.server != nil {
        a.server.do()
    } else {
        a.client.do()
    }

with

    a.delegate.do()

In case a specific type is required a type check can
be performed:

    if srv, ok := a.delegate.(*consul.Server); ok {
        srv.doSrv()
    }
2017-05-15 18:35:38 +02:00
Frank Schroeder 65b5c51ec7 test: add helper for ioutil.TempDir/TempFile
This creates a simplified helper for temporary directories and files.
All path names are prefixed with the name of the current test.
All files and directories are stored either in /tmp/consul-test
or /tmp if the former could not be created.

Using the system temp dir breaks some tests on macOS where the unix
socket path becomes too long.
2017-05-12 22:12:47 +02:00
Frank Schroeder 98f6e3b7c5 test: suppress osx firewall warning
macOS displays a firewall warning dialog when an unsigned
application is trying to bind to a non-loopback address.

This patch updates some test configurations to ensure binding
to a loopback address where possible to suppress these warnings.
2017-05-12 22:12:47 +02:00
Frank Schroeder 3522a41bad test: include test name in temp file/dir
This helps identifying hanging tests by looking
at the process list.
2017-05-12 22:12:47 +02:00
James Phillips 0daacfe5e5
test: Turns down server health interval for faster convergence.
This fixes the autopilot tests.
2017-05-05 17:07:02 +02:00
Frank Schroeder fb83790cb9 Do not modify config after creation II
Move code for finding the advertise address via a
template into consulConfig() so that the config
object is not modified after creation.
2017-05-04 01:41:47 +02:00
Frank Schroeder 48fa2962eb Revert "test: Run command/agent tests in parallel"
This reverts commit 17be40a733.
2017-04-27 14:39:04 -07:00
Frank Schroeder 17be40a733 test: Run command/agent tests in parallel 2017-04-27 10:34:30 -07:00
Frank Schroeder 8bb7792799 golint: Drop the unused value from range
for i, _ := range foo -> for i := range foo
2017-04-25 09:26:13 -07:00
Frank Schroeder eddb1af603 Remove duplicate constants
This patch removes duplicate internal copies of constants in the structs
package which are also defined in the api package. The api.KVOp type
with all its values for the TXN endpoint and the api.HealthXXX constants
are now used throughout the codebase.

This resulted in some circular dependencies in the testutil package
which have been resolved by copying code and constants and moving the
WaitForLeader function into a separate testrpc package.
2017-04-20 09:54:49 -07:00
James Phillips fa04c24978
Adds a new -disable-host-node-id option to help when testing with containers.
Fixes #2877.
2017-04-12 22:07:18 -07:00
James Phillips 59a599a14b
Cleans up a stray mark and fixes unit tests.
Ended up removing the leader_test.go server address change test as part
of this. The join was failing becase we were using a new node name with
the new logic here, but realized this was hitting some of the memberlist
conflict logic and not working as we expected. We need some additional
work to fully support address changes, so removed the test for now.
2017-03-27 01:28:54 -07:00
Kyle Havlovitz 390f41d8d5
Add advanced autopilot features 2017-03-22 15:25:16 -07:00
James Phillips 1091c7314e
Removes remoteConsuls in favor of the new router.
This has the next wave of RTT integration with the router and also
factors some common RTT-related helpers out to lib. While we were
in here we also got rid of the coordinate disable config so we don't
need to deal with the complexity in the router (there was never a
user-visible way to disable coordinates).
2017-03-16 16:42:19 -07:00
James Phillips 26dddb428c
Forces user-supplied node IDs to lower case for consistency. 2017-03-13 19:51:56 -07:00
Kyle Havlovitz 9221aed856
Remove the RPC client interface and update docs 2017-02-28 13:41:09 -08:00
Kyle Havlovitz bbfd25b530
Fix test import 2017-01-23 21:34:01 -05:00
Kyle Havlovitz a55968f009
Merge branch 'master' into f-prepared-query-nodemeta 2017-01-23 20:17:48 -05:00
Kyle Havlovitz 3f3d7f9891
Add tests for node meta in prepared queries and update docs 2017-01-23 19:17:30 -05:00
James Phillips e719ff8ef1
Adds a test to make sure we get the same ID on the second run. 2017-01-18 09:52:34 -08:00
James Phillips bd605e330c
Adds basic support for node IDs. 2017-01-17 22:47:59 -08:00