github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
20,267 Commits
1,700 Branches
460 Tags
692 MiB
Commit Graph

161 Commits (926db9c8a61c631a2a060dbdfbdf894249ba860c)

Author SHA1 Message Date
alex fd7a403e11
monitor leadership in peering service (#13257) 
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
 2022-05-26 17:55:16 -07:00
Chris S. Kim 6d3bea7129
Add support for streaming CA roots to peers (#13260) 
Sender watches for changes to CA roots and sends
them through the replication stream. Receiver saves
CA roots to tablePeeringTrustBundle
 2022-05-26 15:24:09 -04:00
R.B. Boyer 1a8834e1c8
peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218) 
The importing peer will need to know what SNI and SPIFFE name
corresponds to each exported service. Additionally it will need to know
at a high level the protocol in use (L4/L7) to generate the appropriate
connection pool and local metrics.

For replicated connect synthetic entities we edit the `Connect{}` part
of a `NodeService` to have a new section:

    {
      "PeerMeta": {
        "SNI": [
          "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul"
        ],
        "SpiffeID": [
          "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web"
        ],
        "Protocol": "tcp"
      }
    }

This data is then replicated and saved as-is at the importing side. Both
SNI and SpiffeID are slices for now until I can be sure we don't need
them for how mesh gateways will ultimately work.
 2022-05-25 12:37:44 -05:00
R.B. Boyer be631ebdce
peering: disable requirement for mesh gateways initially (#13213) 2022-05-25 10:13:23 -05:00
alex 876f3bb971
peering: expose IsLeader, hung up on dialer if follower (#13164) 
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
 2022-05-23 11:30:58 -07:00
R.B. Boyer 2e72f44fda
peering: accept replication stream of discovery chain information at the importing side (#13151) 2022-05-19 16:37:52 -05:00
R.B. Boyer 3e4a522882 peering: replicate discovery chains information to importing peers 
Treat each exported service as a "discovery chain" and replicate one
synthetic CheckServiceNode for each chain and remote mesh gateway.

The health will be a flattened generated check of the checks for that
mesh gateway node.
 2022-05-19 14:21:44 -05:00
R.B. Boyer 5a03536040 prefactor some functions out of the monolithic file 2022-05-19 14:21:29 -05:00
Freddy b38be4c0ed
Patches to peering initiation for POC demo (#13076) 
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
 2022-05-13 13:01:00 -06:00
Freddy e874b860c0
Actually block when syncing subscriptions (#13066) 
By changing to use WatchCtx we will actually block for changes to the peering list. WatchCh creates a goroutine to collect errors from WatchCtx and returns immediately.

The existing behavior wouldn't result in a tight loop because of the rate limiting in the surrounding function, but it would still lead to more work than is necessary.
 2022-05-12 17:36:14 -06:00
Evan Culver 0fa5e7be5a
peering: add TrustBundleListByService endpoint (#13048) 2022-05-12 15:58:22 -07:00
Freddy 4e215dc411
[OSS] Add upsert handling for receiving CheckServiceNode (#13061) 2022-05-12 15:04:44 -06:00
R.B. Boyer cc15a11f9c
test: ensure this package uses freeport for port allocation (#13036) 2022-05-11 14:20:50 -05:00
R.B. Boyer 901fd4dd68
remove remaining shim runStep functions (#13015) 
Wraps up the refactor from #13013
 2022-05-10 16:24:45 -05:00
R.B. Boyer 0d6d16ddfb
add general runstep test helper instead of copying it all over the place (#13013) 2022-05-10 15:25:51 -05:00
FFMMM 37a1e33834
expose meta tags for peering (#12964) 2022-05-09 13:47:37 -07:00
R.B. Boyer f507f62f3c
peering: initial sync (#12842) 
- Add endpoints related to peering: read, list, generate token, initiate peering
- Update node/service/check table indexing to account for peers
- Foundational changes for pushing service updates to a peer
- Plumb peer name through Health.ServiceNodes path

see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679,
     ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663,
     ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634,
     ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Evan Culver <eculver@hashicorp.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
 2022-04-21 17:34:40 -05:00
FFMMM a46bbe892d
add more labels to RequestRecorder (#12727) 
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-04-12 10:50:25 -07:00
FFMMM 5245251bbf
[rpc/middleware][consul] plumb intercept off, add server level happy test (#12692) 2022-04-06 14:33:05 -07:00
FFMMM 7ed356b338
lower log to trace (#12708) 2022-04-06 11:37:08 -07:00
FFMMM 1adfd7b94c
polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
FFMMM c39854de78
fix bad oss sync, use gauges not counters (#12611) 2022-03-24 14:41:30 -07:00
FFMMM a7e5ee005a
factor out recording func, add unit tests (#12585) 
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-22 09:31:54 -07:00
Dan Upton 7298967070
Restructure gRPC server setup (#12586) 
OSS sync of enterprise changes at 0b44395e
 2022-03-22 12:40:24 +00:00
FFMMM e5ebc47a94
pre register new rpc metric, rename metric (#12582) 2022-03-21 17:26:32 -07:00
FFMMM db27ea3484
[sync oss] add net/rpc interceptor implementation (#12573) 
* sync ent changes from 866dcb0667

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* update oss go.mod

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-17 16:02:26 -07:00
Dan Upton fdfe079674
streaming: split event buffer by key (#12080) 2022-01-28 12:27:00 +00:00
Giulio Micheloni af7b7b5693
Merge branch 'main' into serve-panic-recovery 2021-11-06 16:12:06 +01:00
Daniel Nephin 8ba760a2fc acl: remove id and revision from Policy constructors 
The fields were removed in a previous commit.

Also remove an unused constructor for PolicyMerger
 2021-11-05 15:45:08 -04:00
Daniel Nephin aea4cc5a6d acl: remove legacy arg to store.ACLTokenSet 
And remove the tests for legacy=true
 2021-10-25 17:25:14 -04:00
Giulio Micheloni 0c78ddacde Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
R.B. Boyer 706fc8bcd0
grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099) 
Fixes #11086
 2021-09-22 13:14:26 -05:00
Giulio Micheloni 655da1fc42
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 4b0eaa4bff grpc, xds: recovery middleware to return and log error in case of panic 
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
 2021-08-22 19:06:26 +01:00
R.B. Boyer 097e1645e3
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
R.B. Boyer 310e775a8a
state: partition nodes and coordinates in the state store (#10859) 
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
 2021-08-17 13:29:39 -05:00
Daniel Nephin f497d5ab30 acl: remove many instances of authz == nil 2021-07-30 13:58:35 -04:00
R.B. Boyer fc9b1a277d
sync changes to oss files made in enterprise (#10670) 2021-07-22 13:58:08 -05:00
R.B. Boyer 188e8dc51f
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
Daniel Nephin 71b0f0a7a6 structs: remove EnterpriseMeta.GetNamespace 
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
 2021-03-09 15:17:26 -05:00
Daniel Nephin 1d2d15b1e1 agent: add a test for streaming in the service health endpoint 
Co-authored-by: Paul Banks <banks@banksco.de>
 2021-02-25 14:08:10 -05:00
Daniel Nephin d1772ae305 structs: rename EnterpriseMeta constructor 
To match the Go convention.
 2021-02-16 14:45:43 -05:00
Daniel Nephin b9e60c0775 testing: skip slow tests with -short 
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
 2020-12-07 13:42:55 -05:00
Daniel Nephin fb70c8bac2 stream: document that Payload must be immutable 
If they are sent to EventPublisher.Publish.

Also document that PayloadEvents is expected to come from a subscription and that it is
not immutable.
 2020-11-06 13:00:33 -05:00
Daniel Nephin 868cfe1eac stream: Add HasReadPermission to Payload 
Required now that filter is a method on PayloadEvents instead of Event
 2020-11-05 19:17:18 -05:00
Daniel Nephin a33c50ef0d
Merge pull request #9073 from hashicorp/dnephin/backport-streaming-namespaces 
streaming: backport namespace changes
 2020-11-05 14:19:10 -05:00
Daniel Nephin c82f6ef2d8
Merge pull request #9061 from hashicorp/dnephin/event-fields 
stream: support filtering by namespace
 2020-11-05 14:18:35 -05:00
Daniel Nephin b532e092dc structs: add a namespace test for CheckServiceNode.CanRead 2020-10-30 15:07:04 -04:00
Daniel Nephin c42fe5ae43 subscribe: set the request namespace 2020-10-30 14:34:04 -04:00
Daniel Nephin a5dd2001cf stream: remove Event.Key 
Makes Payload a type with FilterByKey so that Payloads can implement
filtering by key. With this approach we don't need to expose a Namespace
field on Event, and we don't need to invest micro formats or require a
bunch of code to be aware of exactly how the key field is encoded.
 2020-10-28 16:48:04 -04:00
Daniel Nephin 68342a0cb5 proto: remove Event.Key field 
The field is never used, and the value is available from the payload.
 2020-10-28 16:33:00 -04:00
Daniel Nephin 9a1e845be8 proto: remove Event.Namespace field 
All events are part of a single Topic, so we don't need this field.
 2020-10-28 16:33:00 -04:00
Daniel Nephin 3dfb7c224b stream: Use a no-op event publisher if streaming is disabled 2020-10-28 13:54:19 -04:00
Daniel Nephin fb57d9b26a stream: close the subscription on Unsubscribe 2020-10-22 13:39:27 -04:00
Daniel Nephin c5d57c9f07 subscribe: add test cases for newEventFromStreamEvent 2020-10-08 18:48:17 -04:00
Daniel Nephin f185124320 subscribe: Add steps to rpc/subscribe tests 
To make them easier to follow
 2020-10-08 15:38:01 -04:00
Daniel Nephin b103568e98
Merge pull request #8818 from hashicorp/streaming/add-subscribe-service-batch-events 
stream: handle batch events as a special case of Event
 2020-10-07 21:25:32 -04:00
Daniel Nephin 21c21191f4 structs: add CheckServiceNode.CanRead 
And use it from the subscribe endpoint.
 2020-10-07 18:15:13 -04:00
Daniel Nephin b27068b72a stream: Return a single event from a subscription.Next 
Handle batch events as a single event
 2020-10-06 13:18:20 -04:00
Daniel Nephin f5d11562f2 subscribe: update to use NewSnapshotToFollow event 2020-10-06 12:49:35 -04:00
Daniel Nephin e3290f5971 Move agent/subscribe -> agent/rpc/subscribe 2020-10-06 12:49:35 -04:00