1659 Commits (2274f64cab78cb878f93a724fd741b2543f79142)

Author	SHA1	Message	Date
R.B. Boyer	9566df524e	agent: cache notifications work after error if the underlying RPC returns index=1 (#6547) ... Fixes #6521 Ensure that initial failures to fetch an agent cache entry using the notify API where the underlying RPC returns a synthetic index of 1 correctly recovers when those RPCs resume working. The bug in the Cache.notifyBlockingQuery used to incorrectly "fix" the index for the next query from 0 to 1 for all queries, when it should have not done so for queries that errored. Also fixed some things that made debugging difficult: - config entry read/list endpoints send back QueryMeta headers - xds event loops don't swallow the cache notification errors	5 years ago
Matt Keeler	76cf54068b	Expand the QueryOptions and QueryMeta interfaces (#6545) ... In a previous PR I made it so that we had interfaces that would work enough to allow blockingQueries to work. However to complete this we need all fields to be settable and gettable. Notes: • If Go ever gets contracts/generics then we could get rid of all the Getters/Setters • protoc / protoc-gen-gogo are going to generate all the getters for us. • I copied all the getters/setters from the protobuf funcs into agent/structs/protobuf_compat.go • Also added JSON marshaling funcs that use jsonpb for protobuf types.	5 years ago
Freddy	fdd10dd8b8	Expose HTTP-based paths through Connect proxy (#6446) ... Fixes: #5396 This PR adds a proxy configuration stanza called expose. These flags register listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only listening on the loopback interface, while still accepting traffic from non Connect-enabled services. Under expose there is a boolean checks flag that would automatically expose all registered HTTP and gRPC check paths. This stanza also accepts a paths list to expose individual paths. The primary use case for this functionality would be to expose paths for third parties like Prometheus or the kubelet. Listeners for requests to exposed paths are be configured dynamically at run time. Any time a proxy, or check can be registered, a listener can also be created. In this initial implementation requests to these paths are not authenticated/encrypted.	5 years ago
R.B. Boyer	5882e21b2b	agent: tolerate more failure scenarios during service registration with central config enabled (#6472) ... Also: * Finished threading replaceExistingChecks setting (from GH-4905) through service manager. * Respected the original configSource value that was used to register a service or a check when restoring persisted data. * Run several existing tests with and without central config enabled (not exhaustive yet). * Switch to ioutil.ReadFile for all types of agent persistence.	5 years ago
Matt Keeler	100ebd63f9	Allow for enterprise only leader routines (#6533) ... Eventually I am thinking we may need a way to register these at different priority levels but for now sticking this here is fine	5 years ago
R.B. Boyer	af01d397a5	connect: don't colon-hex-encode the AuthorityKeyId and SubjectKeyId fields in connect certs (#6492) ... The fields in the certs are meant to hold the original binary representation of this data, not some ascii-encoded version. The only time we should be colon-hex-encoding fields is for display purposes or marshaling through non-TLS mediums (like RPC).	5 years ago
R.B. Boyer	796de297c8	connect: intermediate CA certs generated with the vault provider lack URI SANs (#6491) ... This only affects vault versions >=1.1.1 because the prior code accidentally relied upon a bug that was fixed in https://github.com/hashicorp/vault/pull/6505 The existing tests should have caught this, but they were using a vendored copy of vault version 0.10.3. This fixes the tests by running an actual copy of vault instead of an in-process copy. This has the added benefit of changing the dependency on vault to just vault/api. Also update VaultProvider to use similar SetIntermediate validation code as the ConsulProvider implementation.	5 years ago
Matt Keeler	51dcd126b7	Add support for implementing new requests with protobufs instea… (#6502) ... * Add build system support for protobuf generation This is done generically so that we don’t have to keep updating the makefile to add another proto generation. Note: anything not in the vendor directory and with a .proto extension will be run through protoc if the corresponding namespace.pb.go file is not up to date. If you want to rebuild just a single proto file you can do so with: make proto-rebuild PROTOFILES=<list of proto files to rebuild> Providing the PROTOFILES var will override the default behavior of finding all the .proto files. * Start adding types to the agent/proto package These will be needed for some other work and are by no means comprehensive. * Add ability to resolve/fixup the agentpb.ACLLinks structure in the state store. * Use protobuf marshalling of raft requests instead of msgpack for protoc generated types. This does not change any encoding of existing types. * Removed structs package automatically encoding with protobuf marshalling Instead the caller of raftApply that wants to opt-in to protobuf encoding will have to call `raftApplyProtobuf` * Run update-vendor to fixup modules.txt Nothing changed as far as dependencies go but the ordering of modules in that file depends on the time they are first seen and its not alphabetical. * Rename some things and implement the structs.RPCInfo interface bits agentpb.QueryOptions and agentpb.WriteRequest implement 3 of the 4 RPCInfo funcs and the new TargetDatacenter message type implements the fourth. * Use the right encoding function. * Renamed agent/proto package to agent/agentpb to prevent package name conflicts * Update modules.txt to fix ordering * Change blockingQuery to take in interfaces for the query options and meta * Add %T to error output. * Add/Update some comments	5 years ago
R.B. Boyer	f9496dc627	sdk: add freelist tracking and ephemeral port range skipping to freeport ... This should cut down on test flakiness. Problems handled: - If you had enough parallel test cases running, the former circular approach to handling the port block could hand out the same port to multiple cases before they each had a chance to bind them, leading to one of the two tests to fail. - The freeport library would allocate out of the ephemeral port range. This has been corrected for Linux (which should cover CI). - The library now waits until a formerly-in-use port is verified to be free before putting it back into circulation.	5 years ago
R.B. Boyer	7ccaa13514	fix typo of 'unknown' in log messages	5 years ago
R.B. Boyer	20eb0d3e94	cache: remove data race in agent cache ... In normal operations there is a read/write race related to request QueryOptions fields. An example race: WARNING: DATA RACE Read at 0x00c000836950 by goroutine 30: github.com/hashicorp/consul/agent/structs.(*ServiceConfigRequest).CacheInfo() /go/src/github.com/hashicorp/consul/agent/structs/config_entry.go:506 +0x109 github.com/hashicorp/consul/agent/cache.(*Cache).getWithIndex() /go/src/github.com/hashicorp/consul/agent/cache/cache.go:262 +0x5c github.com/hashicorp/consul/agent/cache.(*Cache).notifyBlockingQuery() /go/src/github.com/hashicorp/consul/agent/cache/watch.go:89 +0xd7 Previous write at 0x00c000836950 by goroutine 147: github.com/hashicorp/consul/agent/cache-types.(*ResolvedServiceConfig).Fetch() /go/src/github.com/hashicorp/consul/agent/cache-types/resolved_service_config.go:31 +0x219 github.com/hashicorp/consul/agent/cache.(*Cache).fetch.func1() /go/src/github.com/hashicorp/consul/agent/cache/cache.go:495 +0x112 This patch does a lightweight copy of the request struct so that the embedded QueryOptions fields that are mutated during Fetch() are scoped to just that one RPC.	5 years ago
hashicorp-ci	4f499a8044	update bindata_assetfs.go	5 years ago
Hans Hasselberg	4a20efda9b	agent: handleEnterpriseLeave (#6453)	5 years ago
R.B. Boyer	a86e63f81e	test: actually wait for the TestAgent to be fully shutdown (#6441)	5 years ago
Sarah Adams	001137e5e5	test: ensure all TestAgent constructions use a constructor (#6443) ... ensure all TestAgent constructions use a constructor to get start retries + test logs going to the right place Fixes #6435	5 years ago
Sarah Adams	74461406e0	remove funky panic/recover in agent tests (#6442)	5 years ago
Sarah Adams	4ed5515fca	refactor & add better retry logic to NewTestAgent (#6363) ... Fixes #6361	5 years ago
Pierre Souchay	be50400c62	Distinguish between DC not existing and not being available (#6399)	5 years ago
Aestek	7c7b7f24fd	Add option to register services and their checks idempotently (#4905)	5 years ago
Sarah Adams	9f4b329b6d	txn: don't try to decode request bodies > raft.SuggestedMaxDataSize (#6422) ... txn: don't try to decode request bodies > raft.SuggestedMaxDataSize	5 years ago
Matt Keeler	42d608587f	Store primaries root in secondary after intermediate signature (#6333) ... * Store primaries root in secondary after intermediate signature This ensures that the intermediate exists within the CA root stored in raft and not just in the CA provider state. This has the very nice benefit of actually outputting the intermediate cert within the ca roots HTTP/RPC endpoints. This change means that if signing the intermediate fails it will not set the root within raft. So far I have not come up with a reason why that is bad. The secondary CA roots watch will pull the root again and go through all the motions. So as soon as getting an intermediate CA works the root will get set. * Make TestAgentAntiEntropy_Check_DeferSync less flaky I am not sure this is the full fix but it seems to help for me.	5 years ago
R.B. Boyer	7deaba63e1	test: ensure the node name is a valid dns name (#6424) ... The space in the node name was making every test emit a useless warning.	5 years ago
R.B. Boyer	3be88df207	test: explicitly run the pprof tests for 1s instead of the 30s default (#6421)	5 years ago
R.B. Boyer	d979d8c239	test: add additional http status code assertions in coordinate HTTP API tests (#6410) ... When this test flakes sometimes this happens: --- FAIL: TestCoordinate_Node (1.69s) panic: interface conversion: interface {} is nil, not structs.Coordinates [recovered] FAIL github.com/hashicorp/consul/agent 19.999s Exit code: 1 panic: interface conversion: interface {} is nil, not structs.Coordinates [recovered] panic: interface conversion: interface {} is nil, not structs.Coordinates There is definitely a bug lurking, but the code seems to imply this can only return nil on 404. The tests previously were not checking the status code. The underlying cause of the flake is unknown, but this should turn the failure into a more normal test failure.	5 years ago
Pierre Souchay	58f04815d5	Display IPs of machines when node names conflict to ease troubleshooting ... When there is an node name conflicts, such messages are displayed within Consul: `consul.fsm: EnsureRegistration failed: failed inserting node: Error while renaming Node ID: "e1d456bc-f72d-98e5-ebb3-26ae80d785cf": Node name node001 is reserved by node 05f10209-1b9c-b90c-e3e2-059e64556d4a with name node001` While it is easy to find the node that has reserved the name, it is hard to find the node trying to aquire the name since it is not registered, because it is not part of `consul members` output This PR will display the IP of the offender and solve far more easily those issues.	5 years ago
Alvin Huang	c516fabfac	revert commits on master (#6413)	5 years ago
tradel	5a22b77340	update tests to match new method signatures	5 years ago
tradel	1ff46f3f0a	confi\gure providers with DC and domain	5 years ago
tradel	5ba28a6a7b	create a common name for autoTLS agent certs	5 years ago
tradel	9b1ac4e7ef	add subject names to issued certs	5 years ago
tradel	7f36a5b676	construct a common name for each CSR	5 years ago
tradel	672e181399	add serviceID to leaf cert request	5 years ago
tradel	a4312d2e6e	add domain and nodeName to agent cert request	5 years ago
tradel	82ae7caf3e	Added DC and domain args to Configure method	5 years ago
R.B. Boyer	b962fe38cd	test: send testagent logs through testing.Logf (#6411)	5 years ago
R.B. Boyer	91da908d2f	test: fix TestAgent.Start() to not segfault if the DNSServer cannot ListenAndServe (#6409) ... The embedded `Server` field on a `DNSServer` is only set inside of the `ListenAndServe` method. If that method fails for reasons like the address being in use and is not bindable, then the `Server` field will not be set and the overall `Agent.Start()` will fail. This will trigger the inner loop of `TestAgent.Start()` to invoke `ShutdownEndpoints` which will attempt to pretty print the DNS servers using fields on that inner `Server` field. Because it was never set, this causes a nil pointer dereference and crashes the test.	5 years ago
Alvin Huang	0be1531d80	add nil pointer check for pointer to ACLToken struct (#6407)	5 years ago
Hans Hasselberg	d051342902	make sure auto_encrypt has private key type and bits (#6392)	5 years ago
Hans Hasselberg	faa54ab989	auto_encrypt: verify_incoming_rpc is good enough for auto_encrypt.allow_tls (#6376) ... Previously `verify_incoming` was required when turning on `auto_encrypt.allow_tls`, but that doesn't work together with HTTPS UI in some scenarios. Adding `verify_incoming_rpc` to the allowed configurations.	5 years ago
R.B. Boyer	7bc941575c	test: don't leak agent goroutines in TestAgent_sidecarServiceFromNodeService (#6396) ... A goroutine dump using runtime.Stack() before/after shows a drop from 121 => 4.	5 years ago
Hans Hasselberg	f3def8c0d0	make sure auto_encrypt has private key type and bits	5 years ago
hashicorp-ci	3757cb6a96	update bindata_assetfs.go	5 years ago
R.B. Boyer	cc9a6f7993	Merge pull request #6388 from hashicorp/release/1-6 ... merging release/1-6 into master	5 years ago
Matt Keeler	cbd1857186	Secondary CA `establishLeadership` fix (#6383) ... This prevents ACL issues (or other issues) during intermediate CA cert signing from failing leader establishment.	5 years ago
Hans Hasselberg	3e46352ccb	auto_encrypt: use server-port (#6287) ... AutoEncrypt needs the server-port because it wants to talk via RPC. Information from gossip might not be available at that point and thats why the server-port is being used.	5 years ago
R.B. Boyer	dfcdc41ef8	connect: allow 'envoy_cluster_json' escape hatch to continue to function (#6378)	5 years ago
R.B. Boyer	7a6faccf2f	docs: document how envoy escape hatches work with the discovery chain (#6350) ... - Bootstrap escape hatches are OK. - Public listener/cluster escape hatches are OK. - Upstream listener/cluster escape hatches are not supported. If an unsupported escape hatch is configured and the discovery chain is activated log a warning and act like it was not configured. Fixes #6160	5 years ago
Matt Keeler	0c3cf47266	Ensure that config entry writes are forwarded to the primary DC (#6339)	5 years ago
R.B. Boyer	fd1c62ee8b	connect: ensure time.Duration fields retain their human readable forms in the API (#6348) ... This applies for both config entries and the compiled discovery chain. Also omit some other config entries fields when empty.	5 years ago
R.B. Boyer	561b2fe606	connect: generate the full SNI names for discovery targets in the compiler rather than in the xds package (#6340)	5 years ago