consul

Commit Graph

Author	SHA1	Message	Date
alex	876f3bb971	peering: expose IsLeader, hung up on dialer if follower (#13164 ) Signed-off-by: acpana <8968914+acpana@users.noreply.github.com> Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>	3 years ago
Matt Keeler	26f4ea3f01	Migrate from `protoc` to `buf` (#12841 ) * Install `buf` instead of `protoc` * Created `buf.yaml` and `buf.gen.yaml` files in the two proto directories to control how `buf` generates/lints proto code. * Invoke `buf` instead of `protoc` * Added a `proto-format` make target. * Committed the reformatted proto files. * Added a `proto-lint` make target. * Integrated proto linting with CI * Fixed tons of proto linter warnings. * Got rid of deprecated builtin protoc-gen-go grpc plugin usage. Moved to direct usage of protoc-gen-go-grpc. * Unified all proto directories / go packages around using pb prefixes but ensuring all proto packages do not have the prefix.	3 years ago
cskh	c986940fda	Upgrade golangci-lint for go v1.18 (#13176 )	3 years ago
R.B. Boyer	21bb0eef4a	test: fix flaky test TestEventBufferFuzz (#13175 )	3 years ago
Matt Keeler	d0fdf22f83	Fix tests broken in #13173 (#13178 ) I changed the error type returned in a situation but didn’t update the tests to expect that error.	3 years ago
Matt Keeler	3c1e17cbd5	Fix flaky tests in the agent/grpc/public/services/serverdiscovery package (#13173 ) Occasionally we had seen the TestWatchServers_ACLToken_PermissionDenied be flagged as flaky in circleci. This change should fix that. Why it fixes it is complicated. The test was failing with a panic when a mocked ACL Resolver was being called more times than expected. I struggled for a while to determine how that could be. This test should call authorize once and only once and the error returned should cause the stream to be terminated and the error returned to the gRPC client. Another oddity was no amount of running this test locally seemed to be able to reproduce the issue. I ran the test hundreds of thousands of time and it always passed. It turns out that there is nothing wrong with the test. It just so happens that the panic from unexpected invocation of a mocked call happened during the test but was caused by a previous test (specifically the TestWatchServers_StreamLifecycle test) The stream from the previous test remained open after all the test Cleanup functions were run and it just so happened that when the EventPublisher eventually picked up that the context was cancelled during cleanup, it force closes all subscriptions which causes some loops to be re-entered and the streams to be reauthorized. Its that looping in response to forced subscription closures that causes the mock to eventually panic. All the components, publisher, server, client all operate based on contexts. We cancel all those contexts but there is no syncrhonous way to know when they are stopped. We could have implemented a syncrhonous stop but in the context of an actual running Consul, context cancellation + async stopping is perfectly fine. What we (Dan and I) eventually thought was that the behavior of grpc streams such as this when a server was shutting down wasn’t super helpful. What we would want is for a client to be able to distinguish between subscription closed because something may have changed requiring re-authentication and subscription closed because the server is shutting down. That way we can send back appropriate error messages to detail that the server is shutting down and not confuse users with potentially needing to resubscribe. So thats what this PR does. We have introduced a shutting down state to our event subscriptions and the various streaming gRPC services that rely on the event publisher will all just behave correctly and actually stop the stream (not attempt transparent reauthorization) if this particular error is the one we get from the stream. Additionally the error that gets transmitted back through gRPC when this does occur indicates to the consumer that the server is going away. That is more helpful so that a client can then attempt to reconnect to another server.	3 years ago
R.B. Boyer	bbcb1fa805	agent: allow for service discovery queries involving peer name to use streaming (#13168 )	3 years ago
Dan Upton	d7f8a8e4ef	proxycfg: remove dependency on `cache.UpdateEvent` (#13144 ) OSS portion of enterprise PR 1857. This removes (most) references to the `cache.UpdateEvent` type in the `proxycfg` package. As we're going to be direct usage of the agent cache with interfaces that can be satisfied by alternative server-local datasources, it doesn't make sense to depend on this type everywhere anymore (particularly on the `state.ch` channel). We also plan to extract `proxycfg` out of Consul into a shared library in the future, which would require removing this dependency. Aside from a fairly rote find-and-replace, the main change is that the `cache.Cache` and `health.Client` types now accept a callback function parameter, rather than a `chan<- cache.UpdateEvents`. This allows us to do the type conversion without running another goroutine.	3 years ago
R.B. Boyer	2e72f44fda	peering: accept replication stream of discovery chain information at the importing side (#13151 )	3 years ago
R.B. Boyer	c27e186334	test: TestServer_RPC_MetricsIntercept should use a concurrency-safe metrics store (#13157 )	3 years ago
cskh	364d4f5efe	Retry on bad dogstatsd connection (#13091 ) - Introduce a new telemetry configurable parameter retry_failed_connection. User can set the value to true to let consul agent continue its start process on failed connection to datadog server. When set to false, agent will stop on failed start. The default behavior is true. Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Evan Culver <eculver@users.noreply.github.com>	3 years ago
R.B. Boyer	3e4a522882	peering: replicate discovery chains information to importing peers Treat each exported service as a "discovery chain" and replicate one synthetic CheckServiceNode for each chain and remote mesh gateway. The health will be a flattened generated check of the checks for that mesh gateway node.	3 years ago
R.B. Boyer	5a03536040	prefactor some functions out of the monolithic file	3 years ago
R.B. Boyer	1e31dc891a	test: fix incorrect use of t instead of r in retry test (#13146 )	3 years ago
Dan Upton	a76f63a695	config: prevent top-level `verify_incoming` enabling mTLS on gRPC port (#13118 ) Fixes #13088 This is a backwards-compatibility bug introduced in 1.12.	3 years ago
Freddy	b38be4c0ed	Patches to peering initiation for POC demo (#13076 ) Co-authored-by: R.B. Boyer <rb@hashicorp.com>	3 years ago
Dhia Ayachi	a0455774c0	When a host header is defined override `req.Host` in the metrics ui (#13071 ) * When a host header is defined override the req.Host in the metrics ui endpoint. * add changelog	3 years ago
Freddy	e874b860c0	Actually block when syncing subscriptions (#13066 ) By changing to use WatchCtx we will actually block for changes to the peering list. WatchCh creates a goroutine to collect errors from WatchCtx and returns immediately. The existing behavior wouldn't result in a tight loop because of the rate limiting in the surrounding function, but it would still lead to more work than is necessary.	3 years ago
Evan Culver	0fa5e7be5a	peering: add TrustBundleListByService endpoint (#13048 )	3 years ago
Freddy	4e215dc411	[OSS] Add upsert handling for receiving CheckServiceNode (#13061 )	3 years ago
Matt Keeler	b788691fa6	Watch the singular service resolver instead of the list + filtering to 1 (#13012 ) * Watch the singular service resolver instead of the list + filtering to 1 * Rename the ConfigEntries cache type to ConfigEntryList	3 years ago
R.B. Boyer	93b164aac3	structs: add convenience methods to sort slices of ServiceName values (#13038 )	3 years ago
R.B. Boyer	cc15a11f9c	test: ensure this package uses freeport for port allocation (#13036 )	3 years ago
R.B. Boyer	901fd4dd68	remove remaining shim runStep functions (#13015 ) Wraps up the refactor from #13013	3 years ago
R.B. Boyer	0d6d16ddfb	add general runstep test helper instead of copying it all over the place (#13013 )	3 years ago
Jared Kirschner	05a648f530	docs: clarify consistency mode operation Changes include: - Add diagrams of the operation of different consistency modes - Note that only stale reads benefit from horizontal scaling - Increase scannability with headings - Document consistency mode defaults and how to override for DNS and HTTP API interfaces - Document X-Consul-Effective-Consistency response header	3 years ago
FFMMM	b8ce8e36fb	add err msg on PeeringRead not found (#12986 ) Signed-off-by: FFMMM <FFMMM@users.noreply.github.com> Co-authored-by: Freddy <freddygv@users.noreply.github.com>	3 years ago
FFMMM	37a1e33834	expose meta tags for peering (#12964 )	3 years ago
Mark Anderson	4364e440db	Add oss test Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	346b68a441	Fix up enterprise version tag. Changes to how the version string was handled created small regression with the release of consul 1.12.0 enterprise. Many tools use the Config:Version field reported by the agent/self resource to determine whether Consul is an enterprise or OSS instance, expect something like 1.12.0+ent for enterprise and simply 1.12.0 for OSS. This was accidentally broken during the runup to 1.12.x This work fixes the value returned by both the self endpoint in ["Config"]["Version"] and the metrics consul.version field. Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Evan Culver	9c8606e138	peering: add store.PeeringsForService implementation (#12957 )	3 years ago
Eric	21c3134575	Support making requests to lambda from connect proxies.	3 years ago
FFMMM	745bd15b15	api: add PeeeringList, polish (#12934 )	3 years ago
Riddhi Shah	0c855fab98	Validate port on mesh service registration (#12881 ) Add validation to ensure connect native services have a port or socketpath specified on catalog registration. This was the only missing piece to ensure all mesh services are validated for a port (or socketpath) specification on catalog registration.	3 years ago
Mark Anderson	c6ff4ba7d8	Support vault namespaces in connect CA (#12904 ) * Support vault namespaces in connect CA Follow on to some missed items from #12655 From an internal ticket "Support standard "Vault namespace in the path" semantics for Connect Vault CA Provider" Vault allows the namespace to be specified as a prefix in the path of a PKI definition, but our usage of the Vault API includes calls that don't support a namespaced key. In particular the sys.* family of calls simply appends the key, instead of prefixing the namespace in front of the path. Unfortunately it is difficult to reliably parse a path with a namespace; only vault knows what namespaces are present, and the '/' separator can be inside a key name, as well as separating path elements. This is in use in the wild; for example 'dc1/intermediate-key' is a relatively common naming schema. Instead we add two new fields: RootPKINamespace and IntermediatePKINamespace, which are the absolute namespace paths 'prefixed' in front of the respective PKI Paths. Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Chris S. Kim	abc472f2a3	Default discovery chain when upstream targets a DestinationPeer (#12942 )	3 years ago
Dan Upton	a668c36930	acl: gRPC login and logout endpoints (#12935 ) Introduces two new public gRPC endpoints (`Login` and `Logout`) and includes refactoring of the equivalent net/rpc endpoints to enable the majority of logic to be reused (i.e. by extracting the `Binder` and `TokenWriter` types). This contains the OSS portions of the following enterprise commits: - 75fcdbfcfa6af21d7128cb2544829ead0b1df603 - bce14b714151af74a7f0110843d640204082630a - cc508b70fbf58eda144d9af3d71bd0f483985893	3 years ago
Mark Anderson	97f19a6ec1	Fix tests for APPEND_FORWARD change Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	863bc16530	Change to use APPEND_FORWARD for terminating gateway Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	6430af1c0e	Update mesh config tests Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	05dc5a26b7	Docs and changelog edits Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	fee6c7a7b6	Fixup missed config entry Signed-off-by: Mark Anderson <manderson@hashicorp.com>	3 years ago
Mark Anderson	28b4b3a85d	Add x-forwarded-client-cert headers Description Add x-fowarded-client-cert information on trusted incoming connections. Envoy provides support forwarding and annotating the x-forwarded-client-cert header via the forward_client_cert_details set_current_client_cert_details filter fields. It would be helpful for consul to support this directly in its config. The escape hatches are a bit cumbersome for this purpose. This has been implemented on incoming connections to envoy. Outgoing (from the local service through the sidecar) will not have a certificate, and so are left alone. A service on an incoming connection will now get headers something like this: ``` X-Forwarded-Client-Cert:[By=spiffe://efad7282-d9b2-3298-f6d8-38b37fb58df3.consul/ns/default/dc/dc1/svc/counting;Hash=61ad5cbdfcb50f5a3ec0ca60923d61613c149a9d4495010a64175c05a0268ab2;Cert="-----BEGIN%20CERTIFICATE-----%0AMIICHDCCAcOgAwIBAgIBCDAKBggqhkjOPQQDAjAxMS8wLQYDVQQDEyZwcmktMTli%0AYXdyb2YuY29uc3VsLmNhLmVmYWQ3MjgyLmNvbnN1bDAeFw0yMjA0MjkwMzE0NTBa%0AFw0yMjA1MDIwMzE0NTBaMAAwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARVIZ7Y%0AZEXfbOGBfxGa7Vuok1MIng%2FuzLQK2xLVlSTIPDbO5hstTGP%2B%2FGx182PYFP3jYqk5%0Aq6rYWe1wiPNMA30Io4H8MIH5MA4GA1UdDwEB%2FwQEAwIDuDAdBgNVHSUEFjAUBggr%0ABgEFBQcDAgYIKwYBBQUHAwEwDAYDVR0TAQH%2FBAIwADApBgNVHQ4EIgQgrp4q50oX%0AHHghMbxz5Bk8OJFWMdfgH0Upr350WlhyxvkwKwYDVR0jBCQwIoAgUe6uERAIj%2FLM%0AyuFzDc3Wbp9TGAKBJYAwyhF14ToOQCMwYgYDVR0RAQH%2FBFgwVoZUc3BpZmZlOi8v%0AZWZhZDcyODItZDliMi0zMjk4LWY2ZDgtMzhiMzdmYjU4ZGYzLmNvbnN1bC9ucy9k%0AZWZhdWx0L2RjL2RjMS9zdmMvZGFzaGJvYXJkMAoGCCqGSM49BAMCA0cAMEQCIDwb%0AFlchufggNTijnQ5SUcvTZrWlZyq%2FrdVC20nbbmWLAiAVshNNv1xBqJI1NmY2HI9n%0AgRMfb8aEPVSuxEHhqy57eQ%3D%3D%0A-----END%20CERTIFICATE-----%0A";Chain="-----BEGIN%20CERTIFICATE-----%0AMIICHDCCAcOgAwIBAgIBCDAKBggqhkjOPQQDAjAxMS8wLQYDVQQDEyZwcmktMTli%0AYXdyb2YuY29uc3VsLmNhLmVmYWQ3MjgyLmNvbnN1bDAeFw0yMjA0MjkwMzE0NTBa%0AFw0yMjA1MDIwMzE0NTBaMAAwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARVIZ7Y%0AZEXfbOGBfxGa7Vuok1MIng%2FuzLQK2xLVlSTIPDbO5hstTGP%2B%2FGx182PYFP3jYqk5%0Aq6rYWe1wiPNMA30Io4H8MIH5MA4GA1UdDwEB%2FwQEAwIDuDAdBgNVHSUEFjAUBggr%0ABgEFBQcDAgYIKwYBBQUHAwEwDAYDVR0TAQH%2FBAIwADApBgNVHQ4EIgQgrp4q50oX%0AHHghMbxz5Bk8OJFWMdfgH0Upr350WlhyxvkwKwYDVR0jBCQwIoAgUe6uERAIj%2FLM%0AyuFzDc3Wbp9TGAKBJYAwyhF14ToOQCMwYgYDVR0RAQH%2FBFgwVoZUc3BpZmZlOi8v%0AZWZhZDcyODItZDliMi0zMjk4LWY2ZDgtMzhiMzdmYjU4ZGYzLmNvbnN1bC9ucy9k%0AZWZhdWx0L2RjL2RjMS9zdmMvZGFzaGJvYXJkMAoGCCqGSM49BAMCA0cAMEQCIDwb%0AFlchufggNTijnQ5SUcvTZrWlZyq%2FrdVC20nbbmWLAiAVshNNv1xBqJI1NmY2HI9n%0AgRMfb8aEPVSuxEHhqy57eQ%3D%3D%0A-----END%20CERTIFICATE-----%0A";Subject="";URI=spiffe://efad7282-d9b2-3298-f6d8-38b37fb58df3.consul/ns/default/dc/dc1/svc/dashboard] ``` Closes #12852	3 years ago
Kyle Havlovitz	76d62a14f5	Return ACLRemoteError from cache and test it correctly	3 years ago
FFMMM	3b3f001580	[sync oss] api: add peering api module (#12911 )	3 years ago
Chris S. Kim	9791bad136	peering: Make Upstream peer-aware (#12900 ) Adds DestinationPeer field to Upstream. Adds Peer field to UpstreamID and its string conversion functions.	3 years ago
Chris S. Kim	0d66301ea7	Cleanup peering files that used error types that were removed (#12892 )	3 years ago
Mathew Estafanous	474385d153	Unify various status errors into one HTTP error type. (#12594 ) Replaces specific error types for HTTP Status codes with a generic HTTPError type. Co-authored-by: Chris S. Kim <ckim@hashicorp.com>	3 years ago
Kyle Havlovitz	0d8b187ea1	Store and return rpc error in acl cache entries	3 years ago
R.B. Boyer	11213ae180	health: ensure /v1/health/service/:service endpoint returns the most recent results when a filter is used with streaming (#12640 ) The primary bug here is in the streaming subsystem that makes the overall v1/health/service/:service request behave incorrectly when servicing a blocking request with a filter provided. There is a secondary non-streaming bug being fixed here that is much less obvious related to when to update the `reply` variable in a `blockingQuery` evaluation. It is unlikely that it is triggerable in practical environments and I could not actually get the bug to manifest, but I fixed it anyway while investigating the original issue. Simple reproduction (streaming): 1. Register a service with a tag. curl -sL --request PUT 'http://localhost:8500/v1/agent/service/register' \ --header 'Content-Type: application/json' \ --data-raw '{ "ID": "ID1", "Name": "test", "Tags":[ "a" ], "EnableTagOverride": true }' 2. Do an initial filter query that matches on the tag. curl -sLi --get 'http://localhost:8500/v1/health/service/test' --data-urlencode 'filter=a in Service.Tags' 3. Note you get one result. Use the `X-Consul-Index` header to establish a blocking query in another terminal, this should not return yet. curl -sLi --get 'http://localhost:8500/v1/health/service/test?index=$INDEX' --data-urlencode 'filter=a in Service.Tags' 4. Re-register that service with a different tag. curl -sL --request PUT 'http://localhost:8500/v1/agent/service/register' \ --header 'Content-Type: application/json' \ --data-raw '{ "ID": "ID1", "Name": "test", "Tags":[ "b" ], "EnableTagOverride": true }' 5. Your blocking query from (3) should return with a header `X-Consul-Query-Backend: streaming` and empty results if it works correctly `[]`. Attempts to reproduce with non-streaming failed (where you add `&near=_agent` to the read queries and ensure `X-Consul-Query-Backend: blocking-query` shows up in the results).	3 years ago

1 2 3 4 5 ...

4284 Commits (ec1eb227ba75a80f17ba11a9a4b5d854f81d0c01)