github
/
consul
mirror of https://github.com/hashicorp/consul
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
20,198 Commits
1,699 Branches
460 Tags
692 MiB
Commit Graph

160 Commits (0094dbf312001ae3c7650964aa5fa0eb9976f8eb)

Author SHA1 Message Date
Chris S. Kim 6d3bea7129
Add support for streaming CA roots to peers (#13260) 
Sender watches for changes to CA roots and sends
them through the replication stream. Receiver saves
CA roots to tablePeeringTrustBundle
 2022-05-26 15:24:09 -04:00
R.B. Boyer 1a8834e1c8
peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218) 
The importing peer will need to know what SNI and SPIFFE name
corresponds to each exported service. Additionally it will need to know
at a high level the protocol in use (L4/L7) to generate the appropriate
connection pool and local metrics.

For replicated connect synthetic entities we edit the `Connect{}` part
of a `NodeService` to have a new section:

    {
      "PeerMeta": {
        "SNI": [
          "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul"
        ],
        "SpiffeID": [
          "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web"
        ],
        "Protocol": "tcp"
      }
    }

This data is then replicated and saved as-is at the importing side. Both
SNI and SpiffeID are slices for now until I can be sure we don't need
them for how mesh gateways will ultimately work.
 2022-05-25 12:37:44 -05:00
R.B. Boyer be631ebdce
peering: disable requirement for mesh gateways initially (#13213) 2022-05-25 10:13:23 -05:00
alex 876f3bb971
peering: expose IsLeader, hung up on dialer if follower (#13164) 
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
 2022-05-23 11:30:58 -07:00
R.B. Boyer 2e72f44fda
peering: accept replication stream of discovery chain information at the importing side (#13151) 2022-05-19 16:37:52 -05:00
R.B. Boyer 3e4a522882 peering: replicate discovery chains information to importing peers 
Treat each exported service as a "discovery chain" and replicate one
synthetic CheckServiceNode for each chain and remote mesh gateway.

The health will be a flattened generated check of the checks for that
mesh gateway node.
 2022-05-19 14:21:44 -05:00
R.B. Boyer 5a03536040 prefactor some functions out of the monolithic file 2022-05-19 14:21:29 -05:00
Freddy b38be4c0ed
Patches to peering initiation for POC demo (#13076) 
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
 2022-05-13 13:01:00 -06:00
Freddy e874b860c0
Actually block when syncing subscriptions (#13066) 
By changing to use WatchCtx we will actually block for changes to the peering list. WatchCh creates a goroutine to collect errors from WatchCtx and returns immediately.

The existing behavior wouldn't result in a tight loop because of the rate limiting in the surrounding function, but it would still lead to more work than is necessary.
 2022-05-12 17:36:14 -06:00
Evan Culver 0fa5e7be5a
peering: add TrustBundleListByService endpoint (#13048) 2022-05-12 15:58:22 -07:00
Freddy 4e215dc411
[OSS] Add upsert handling for receiving CheckServiceNode (#13061) 2022-05-12 15:04:44 -06:00
R.B. Boyer cc15a11f9c
test: ensure this package uses freeport for port allocation (#13036) 2022-05-11 14:20:50 -05:00
R.B. Boyer 901fd4dd68
remove remaining shim runStep functions (#13015) 
Wraps up the refactor from #13013
 2022-05-10 16:24:45 -05:00
R.B. Boyer 0d6d16ddfb
add general runstep test helper instead of copying it all over the place (#13013) 2022-05-10 15:25:51 -05:00
FFMMM 37a1e33834
expose meta tags for peering (#12964) 2022-05-09 13:47:37 -07:00
R.B. Boyer f507f62f3c
peering: initial sync (#12842) 
- Add endpoints related to peering: read, list, generate token, initiate peering
- Update node/service/check table indexing to account for peers
- Foundational changes for pushing service updates to a peer
- Plumb peer name through Health.ServiceNodes path

see: ENT-1765, ENT-1280, ENT-1283, ENT-1283, ENT-1756, ENT-1739, ENT-1750, ENT-1679,
     ENT-1709, ENT-1704, ENT-1690, ENT-1689, ENT-1702, ENT-1701, ENT-1683, ENT-1663,
     ENT-1650, ENT-1678, ENT-1628, ENT-1658, ENT-1640, ENT-1637, ENT-1597, ENT-1634,
     ENT-1613, ENT-1616, ENT-1617, ENT-1591, ENT-1588, ENT-1596, ENT-1572, ENT-1555

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: freddygv <freddy@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Evan Culver <eculver@hashicorp.com>
Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>
 2022-04-21 17:34:40 -05:00
FFMMM a46bbe892d
add more labels to RequestRecorder (#12727) 
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-04-12 10:50:25 -07:00
FFMMM 5245251bbf
[rpc/middleware][consul] plumb intercept off, add server level happy test (#12692) 2022-04-06 14:33:05 -07:00
FFMMM 7ed356b338
lower log to trace (#12708) 2022-04-06 11:37:08 -07:00
FFMMM 1adfd7b94c
polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
FFMMM c39854de78
fix bad oss sync, use gauges not counters (#12611) 2022-03-24 14:41:30 -07:00
FFMMM a7e5ee005a
factor out recording func, add unit tests (#12585) 
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-22 09:31:54 -07:00
Dan Upton 7298967070
Restructure gRPC server setup (#12586) 
OSS sync of enterprise changes at 0b44395e
 2022-03-22 12:40:24 +00:00
FFMMM e5ebc47a94
pre register new rpc metric, rename metric (#12582) 2022-03-21 17:26:32 -07:00
FFMMM db27ea3484
[sync oss] add net/rpc interceptor implementation (#12573) 
* sync ent changes from 866dcb0667

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* update oss go.mod

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
 2022-03-17 16:02:26 -07:00
Dan Upton fdfe079674
streaming: split event buffer by key (#12080) 2022-01-28 12:27:00 +00:00
Giulio Micheloni af7b7b5693
Merge branch 'main' into serve-panic-recovery 2021-11-06 16:12:06 +01:00
Daniel Nephin 8ba760a2fc acl: remove id and revision from Policy constructors 
The fields were removed in a previous commit.

Also remove an unused constructor for PolicyMerger
 2021-11-05 15:45:08 -04:00
Daniel Nephin aea4cc5a6d acl: remove legacy arg to store.ACLTokenSet 
And remove the tests for legacy=true
 2021-10-25 17:25:14 -04:00
Giulio Micheloni 0c78ddacde Merge branch 'main' of https://github.com/hashicorp/consul into hashicorp-main 2021-10-16 16:59:32 +01:00
R.B. Boyer 706fc8bcd0
grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099) 
Fixes #11086
 2021-09-22 13:14:26 -05:00
Giulio Micheloni 655da1fc42
Merge branch 'main' into serve-panic-recovery 2021-08-22 20:31:11 +02:00
Giulio Micheloni 4b0eaa4bff grpc, xds: recovery middleware to return and log error in case of panic 
1) xds and grpc servers:
   1.1) to use recovery middleware with callback that prints stack trace to log
   1.2) callback turn the panic into a core.Internal error
2) added unit test for grpc server
 2021-08-22 19:06:26 +01:00
R.B. Boyer 097e1645e3
agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
R.B. Boyer 310e775a8a
state: partition nodes and coordinates in the state store (#10859) 
Additionally:

- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
 2021-08-17 13:29:39 -05:00
Daniel Nephin f497d5ab30 acl: remove many instances of authz == nil 2021-07-30 13:58:35 -04:00
R.B. Boyer fc9b1a277d
sync changes to oss files made in enterprise (#10670) 2021-07-22 13:58:08 -05:00
R.B. Boyer 188e8dc51f
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
Daniel Nephin 71b0f0a7a6 structs: remove EnterpriseMeta.GetNamespace 
I added this recently without realizing that the method already existed and was named
NamespaceOrEmpty. Replace all calls to GetNamespace with NamespaceOrEmpty or NamespaceOrDefault
as appropriate.
 2021-03-09 15:17:26 -05:00
Daniel Nephin 1d2d15b1e1 agent: add a test for streaming in the service health endpoint 
Co-authored-by: Paul Banks <banks@banksco.de>
 2021-02-25 14:08:10 -05:00
Daniel Nephin d1772ae305 structs: rename EnterpriseMeta constructor 
To match the Go convention.
 2021-02-16 14:45:43 -05:00
Daniel Nephin b9e60c0775 testing: skip slow tests with -short 
Add a skip condition to all tests slower than 100ms.

This change was made using `gotestsum tool slowest` with data from the
last 3 CI runs of master.
See https://github.com/gotestyourself/gotestsum#finding-and-skipping-slow-tests

With this change:

```
$ time go test -count=1 -short ./agent
ok      github.com/hashicorp/consul/agent       0.743s

real    0m4.791s

$ time go test -count=1 -short ./agent/consul
ok      github.com/hashicorp/consul/agent/consul        4.229s

real    0m8.769s
```
 2020-12-07 13:42:55 -05:00
Daniel Nephin fb70c8bac2 stream: document that Payload must be immutable 
If they are sent to EventPublisher.Publish.

Also document that PayloadEvents is expected to come from a subscription and that it is
not immutable.
 2020-11-06 13:00:33 -05:00
Daniel Nephin 868cfe1eac stream: Add HasReadPermission to Payload 
Required now that filter is a method on PayloadEvents instead of Event
 2020-11-05 19:17:18 -05:00
Daniel Nephin a33c50ef0d
Merge pull request #9073 from hashicorp/dnephin/backport-streaming-namespaces 
streaming: backport namespace changes
 2020-11-05 14:19:10 -05:00
Daniel Nephin c82f6ef2d8
Merge pull request #9061 from hashicorp/dnephin/event-fields 
stream: support filtering by namespace
 2020-11-05 14:18:35 -05:00
Daniel Nephin b532e092dc structs: add a namespace test for CheckServiceNode.CanRead 2020-10-30 15:07:04 -04:00
Daniel Nephin c42fe5ae43 subscribe: set the request namespace 2020-10-30 14:34:04 -04:00
Daniel Nephin a5dd2001cf stream: remove Event.Key 
Makes Payload a type with FilterByKey so that Payloads can implement
filtering by key. With this approach we don't need to expose a Namespace
field on Event, and we don't need to invest micro formats or require a
bunch of code to be aware of exactly how the key field is encoded.
 2020-10-28 16:48:04 -04:00
Daniel Nephin 68342a0cb5 proto: remove Event.Key field 
The field is never used, and the value is available from the payload.
 2020-10-28 16:33:00 -04:00
Daniel Nephin 9a1e845be8 proto: remove Event.Namespace field 
All events are part of a single Topic, so we don't need this field.
 2020-10-28 16:33:00 -04:00
Daniel Nephin 3dfb7c224b stream: Use a no-op event publisher if streaming is disabled 2020-10-28 13:54:19 -04:00
Daniel Nephin fb57d9b26a stream: close the subscription on Unsubscribe 2020-10-22 13:39:27 -04:00
Daniel Nephin c5d57c9f07 subscribe: add test cases for newEventFromStreamEvent 2020-10-08 18:48:17 -04:00
Daniel Nephin f185124320 subscribe: Add steps to rpc/subscribe tests 
To make them easier to follow
 2020-10-08 15:38:01 -04:00
Daniel Nephin b103568e98
Merge pull request #8818 from hashicorp/streaming/add-subscribe-service-batch-events 
stream: handle batch events as a special case of Event
 2020-10-07 21:25:32 -04:00
Daniel Nephin 21c21191f4 structs: add CheckServiceNode.CanRead 
And use it from the subscribe endpoint.
 2020-10-07 18:15:13 -04:00
Daniel Nephin b27068b72a stream: Return a single event from a subscription.Next 
Handle batch events as a single event
 2020-10-06 13:18:20 -04:00
Daniel Nephin f5d11562f2 subscribe: update to use NewSnapshotToFollow event 2020-10-06 12:49:35 -04:00
Daniel Nephin e3290f5971 Move agent/subscribe -> agent/rpc/subscribe 2020-10-06 12:49:35 -04:00