85 Commits (0d7f97a080b7b7b90920fab6606f8c47e9a05142)

Author	SHA1	Message	Date
Ronald	17f4689379	backport ent changes to oss (#17614) ... * backport ent changes to oss * Update .changelog/_5669.txt Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com> --------- Co-authored-by: Michael Zalimeni <michael.zalimeni@hashicorp.com>	1 year ago
Ronald	4b2137c2fa	Add copyright headers for acl, api and bench folders (#16706) ... * copyright headers for agent folder * Ignore test data files * fix proto files and remove headers in agent/uiserver folder * ignore deep-copy files * copyright headers for agent folder * fix merge conflicts * copyright headers for agent folder * Ignore test data files * fix proto files * ignore agent/uiserver folder for now * copyright headers for agent folder * Add copyright headers for acl, api and bench folders	2 years ago
Eric Haberkorn	57e2493415	allow setting locality on services and nodes (#16581)	2 years ago
skpratt	db2bd404bf	Synthesize anonymous token pre-bootstrap when needed (#16200) ... * add bootstrapping detail for acl errors * error detail improvements * update acl bootstrapping test coverage * update namespace errors * update test coverage * consolidate error message code and update changelog * synthesize anonymous token * Update token language to distinguish Accessor and Secret ID usage (#16044) * remove legacy tokens * remove lingering legacy token references from docs * update language and naming for token secrets and accessor IDs * updates all tokenID references to clarify accessorID * remove token type references and lookup tokens by accessorID index * remove unnecessary constants * replace additional tokenID param names * Add warning info for deprecated -id parameter Co-authored-by: Paul Glass <pglass@hashicorp.com> * Update field comment Co-authored-by: Paul Glass <pglass@hashicorp.com> --------- Co-authored-by: Paul Glass <pglass@hashicorp.com> * revert naming change * add testing * revert naming change --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
skpratt	6f0b226b0d	ACL error improvements: incomplete bootstrapping and non-existent token (#16105) ... * add bootstrapping detail for acl errors * error detail improvements * update acl bootstrapping test coverage * update namespace errors * update test coverage * add changelog * update message for unbootstrapped error * consolidate error message code and update changelog * logout message change	2 years ago
skpratt	a010902978	Remove legacy acl policies (#15922) ... * remove legacy tokens * remove legacy acl policies * flatten test policies to *_prefix * address oss feedback re: phrasing and tests	2 years ago
skpratt	ad43846755	Remove legacy acl tokens (#15947) ... * remove legacy tokens * Update test comment Co-authored-by: Paul Glass <pglass@hashicorp.com> * fix imports * update docs for additional CLI changes * add test case for anonymous token * set deprecated api fields to json ignore and fix patch errors * update changelog to breaking-change * fix import * update api docs to remove legacy reference * fix docs nav data --------- Co-authored-by: Paul Glass <pglass@hashicorp.com>	2 years ago
Kyle Schochenmaier	bf0f61a878	removes ioutil usage everywhere which was deprecated in go1.16 (#15297) ... * update go version to 1.18 for api and sdk, go mod tidy * removes ioutil usage everywhere which was deprecated in go1.16 in favour of io and os packages. Also introduces a lint rule which forbids use of ioutil going forward. Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	2 years ago
freddygv	fac3ddc857	Use internal server certificate for peering TLS ... A previous commit introduced an internally-managed server certificate to use for peering-related purposes. Now the peering token has been updated to match that behavior: - The server name matches the structure of the server cert - The CA PEMs correspond to the Connect CA Note that if Conect is disabled, and by extension the Connect CA, we fall back to the previous behavior of returning the manually configured certs and local server SNI. Several tests were updated to use the gRPC TLS port since they enable Connect by default. This means that the peering token will embed the Connect CA, and the dialer will expect a TLS listener.	2 years ago
cskh	995671ff6f	fix(api): missing peer name in query option (#14835)	2 years ago
funkiestj	6167400b28	api: add the ability to specify a path prefix (#12914) ... Specifically meant for when consul is behind a reverse proxy / API gateway Co-authored-by: Evan Culver <eculver@hashicorp.com>	3 years ago
FFMMM	3b3f001580	[sync oss] api: add peering api module (#12911)	3 years ago
DanStough	95250e7915	Update go version to 1.18.1	3 years ago
R.B. Boyer	b60d89e7ef	bulk rewrite using this script ... set -euo pipefail unset CDPATH cd "$(dirname "$0")" for f in $(git grep '\brequire := require\.New(' | cut -d':' -f1 | sort -u); do echo "=== require: $f ===" sed -i '/require := require.New(t)/d' $f # require.XXX(blah) but not require.XXX(tblah) or require.XXX(rblah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\([^tr]\)/require.\1(t,\2/g' $f # require.XXX(tblah) but not require.XXX(t, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/require.\1(t,\2/g' $f # require.XXX(rblah) but not require.XXX(r, blah) sed -i 's/\brequire\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/require.\1(t,\2/g' $f gofmt -s -w $f done for f in $(git grep '\bassert := assert\.New(' | cut -d':' -f1 | sort -u); do echo "=== assert: $f ===" sed -i '/assert := assert.New(t)/d' $f # assert.XXX(blah) but not assert.XXX(tblah) or assert.XXX(rblah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\([^tr]\)/assert.\1(t,\2/g' $f # assert.XXX(tblah) but not assert.XXX(t, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(t[^,]\)/assert.\1(t,\2/g' $f # assert.XXX(rblah) but not assert.XXX(r, blah) sed -i 's/\bassert\.\([a-zA-Z0-9_]*\)(\(r[^,]\)/assert.\1(t,\2/g' $f gofmt -s -w $f done	3 years ago
Dan Upton	ca3aca92c4	[OSS] Remove remaining references to master (#11827)	3 years ago
Dan Upton	1e47e3c82b	Groundwork for exposing when queries are filtered by ACLs (#11569)	3 years ago
Dhia Ayachi	2801785710	regenerate expired certs (#11462) ... * regenerate expired certs * add documentation to generate tests certificates	3 years ago
R.B. Boyer	fc9b1a277d	sync changes to oss files made in enterprise (#10670)	3 years ago
Evan Culver	8fce7083c1	tests: skip cases that depend on test server when -short flag set (#10576)	3 years ago
Rémi Lapeyre	213524a657	Always set the Content-Type header when a body is present (#10204) ... * Always set the Content-Type header when a body is present Closes https://github.com/hashicorp/consul/issues/10011 * Add Changelog entry * Add more Content-Type exceptions * Fix tests	4 years ago
Conor Mongey	7a368bd2b6	Move header methods from config to client	4 years ago
Conor Mongey	f647569b84	Prefer http.Header over map[string]string to allow for multi-valued headers	4 years ago
Conor Mongey	cdc8cd7b0e	Allow setting arbitrary headers in API client	4 years ago
R.B. Boyer	61eac21f1a	agent: return the default ACL policy to callers as a header (#9101) ... Header is: X-Consul-Default-ACL-Policy=<allow|deny> This is of particular utility when fetching matching intentions, as the fallthrough for a request that doesn't match any intentions is to enforce using the default acl policy.	4 years ago
Daniel Nephin	d68edcecf4	testing: Remove all the defer os.Removeall ... Now that testutil uses t.Cleanup to remove the directory the caller no longer has to manage the removal	4 years ago
Daniel Nephin	51efba2c7d	testutil: NewLogBuffer - buffer logs until a test fails ... Replaces #7559 Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case. Attaching log output from background goroutines also cause data races. If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race. The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long. This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful. All of the logs are printed from the test goroutine, so they should be associated with the correct test. Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly. Related: https://github.com/golang/go/issues/38458 (may be fixed in go1.15) https://github.com/golang/go/issues/38382#issuecomment-612940030	4 years ago
Daniel Nephin	068b43df90	Enable gofmt simplify ... Code changes done automatically with 'gofmt -s -w'	5 years ago
R.B. Boyer	ffb9c7d6f7	acl: remove the deprecated `acl_enforce_version_8` option (#7991) ... Fixes #7292	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130) ... * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Michel Vocks	6681be918a	api: add option to set TLS options in-memory for API client (#7093) ... This PR adds the option to set in-memory certificates to the API client instead of requiring the certificate to be stored on disk in a file. This allows us to define API client TLS options per Consul secret backend in Vault. Related issue hashicorp/vault#4800	5 years ago
Freddy	2274f64cab	Add Namespace as an api query/write option (#6551)	5 years ago
Sarah Adams	8ff1f481fe	add flag to allow /operator/keyring requests to only hit local servers (#6279) ... Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic). HTTP API: GET /operator/keyring?local-only=true CLI: consul keyring -list --local-only Sending the local-only flag with any non-GET/list request will result in an error.	5 years ago
Freddy	5873c56a03	Flaky test overhaul (#6100)	5 years ago
Freddy	0b9111714b	Update TestServer creation in sdk/testutil (#6084) ... * Retry the creation of the test server three times. * Reduce the retry timeout for the API wait to 2 seconds, opting to fail faster and start over. * Remove wait for leader from server creation. This wait can be added on a test by test basis now that the function is being exported. * Remove wait for anti-entropy sync. This is built into the existing WaitForSerfCheck func, so that can be used if the anti-entropy wait is needed	5 years ago
R.B. Boyer	e47d7eeddb	acl: adding support for kubernetes auth provider login (#5600) ... * auth providers * binding rules * auth provider for kubernetes * login/logout	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579) ... Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791) ... This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Paul Banks	88388d760d	Support Agent Caching for Service Discovery Results (#4541) ... * Add cache types for catalog/services and health/services and basic test that caching works * Support non-blocking cache types with Cache-Control semantics. * Update API docs to include caching info for every endpoint. * Comment updates per PR feedback. * Add note on caching to the 10,000 foot view on the architecture page to make the new data path more clear. * Document prepared query staleness quirk and force all background requests to AllowStale so we can spread service discovery load across servers.	6 years ago
Pierre Souchay	22500f242e	Fix unstable tests in agent, api, and command/watch	6 years ago
Matt Keeler	7dfd2ab316	Add some tests for GenerateEnv	6 years ago
James Bardin	fea32fb7d7	retry locks on network errors (#3553) ... * retry locks on network errors When communicating with a local agent and watching a lock, a dropped connection between the agent and server will show up as a server error and immediately be retried. However if the client is connected to a remote server, a dropped connection immediately aborts the lock. * Updates comment about it being unsafe for writes.	7 years ago
Frank Schröder	21118cafeb	Recursive sanitize (#3505) ... * vendor: add github.com/sergi/go-diff/diffmatchpatch for diff'ing test output * config: refactor Sanitize to recursively clean runtime config and format complex fields * Removes an extra int cast. * Adds a top-level check test case for sanitization.	7 years ago
Frank Schröder	12216583a1	New config parser, HCL support, multiple bind addrs (#3480) ... * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt	7 years ago
Frank Schroeder	8edeabce9a	api: refactor: prefix all API tests with API_	7 years ago
Frank Schroeder	65b5c51ec7	test: add helper for ioutil.TempDir/TempFile ... This creates a simplified helper for temporary directories and files. All path names are prefixed with the name of the current test. All files and directories are stored either in /tmp/consul-test or /tmp if the former could not be created. Using the system temp dir breaks some tests on macOS where the unix socket path becomes too long.	8 years ago
Frank Schroeder	3522a41bad	test: include test name in temp file/dir ... This helps identifying hanging tests by looking at the process list.	8 years ago
Kyle Havlovitz	cd56a5ebdd	Add separate option for verifying incoming HTTPS traffic (#2974) ... * Add separate option for verifying incoming HTTPS traffic	8 years ago
Kyle Havlovitz	c6d0b4cf90	Make the API client's httpClient more pluggable (#2926)	8 years ago