19592 Commits (326d5c02c608e5658c7ee1fc61a4013a3f54a4be)

 

All Branches   

Search

Author	SHA1	Message	Date
John Murret	794277371f	Integration test for server rate limiting (#15960) ... * rate limit test * Have tests for the 3 modes * added assertions for logs and metrics * add comments to test sections * add check for rate limit exceeded text in log assertion section. * fix linting error * updating test to use KV get and put. move log assertion tolast. * Adding logging for blocking messages in enforcing mode. refactoring tests. * modified test description * formatting * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> * expand log checking so that it ensures both logs are they when they are supposed to be and not there when they are not expected to be. * add retry on test * Warn once when rate limit exceed regardless of enforcing vs permissive. * Update test/integration/consul-container/test/ratelimit/ratelimit_test.go Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dan Upton <daniel@floppy.co> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>	2 years ago
Thomas Eckert	13da1a5285	Native API Gateway Config Entries (#15897) ... * Stub Config Entries for Consul Native API Gateway (#15644) * Add empty InlineCertificate struct and protobuf * apigateway stubs * Stub HTTPRoute in api pkg * Stub HTTPRoute in structs pkg * Simplify api.APIGatewayConfigEntry to be consistent w/ other entries * Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry * Add TCPRoute to MakeConfigEntry, return unique Kind * Stub BoundAPIGatewayConfigEntry in agent * Add RaftIndex to APIGatewayConfigEntry stub * Add new config entry kinds to validation allow-list * Add RaftIndex to other added config entry stubs * Update usage metrics assertions to include new cfg entries * Add Meta and acl.EnterpriseMeta to all new ConfigEntry types * Remove unnecessary Services field from added config entry types * Implement GetMeta(), GetEnterpriseMeta() for added config entry types * Add meta field to proto, name consistently w/ existing config entries * Format config_entry.proto * Add initial implementation of CanRead + CanWrite for new config entry types * Add unit tests for decoding of new config entry types * Add unit tests for parsing of new config entry types * Add unit tests for API Gateway config entry ACLs * Return typed PermissionDeniedError on BoundAPIGateway CanWrite * Add unit tests for added config entry ACLs * Add BoundAPIGateway type to AllConfigEntryKinds * Return proper kind from BoundAPIGateway * Add docstrings for new config entry types * Add missing config entry kinds to proto def * Update usagemetrics_oss_test.go * Use utility func for returning PermissionDeniedError * EventPublisher subscriptions for Consul Native API Gateway (#15757) * Create new event topics in subscribe proto * Add tests for PBSubscribe func * Make configs singular, add all configs to PBToStreamSubscribeRequest * Add snapshot methods * Add config_entry_events tests * Add config entry kind to topic for new configs * Add unit tests for snapshot methods * Start adding integration test * Test using the new controller code * Update agent/consul/state/config_entry_events.go * Check value of error * Add controller stubs for API Gateway (#15837) * update initial stub implementation * move files, clean up mutex references * Remove embed, use idiomatic names for constructors * Remove stray file introduced in merge * Add APIGateway validation (#15847) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * APIGateway InlineCertificate validation (#15856) * Add APIGateway validation * Add additional validations * Add protobuf definitions * Tabs to spaces * Add API structs * Move struct fields around a bit * Add validation for InlineCertificate * Fix ACL test * APIGateway BoundAPIGateway validation (#15858) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Move struct fields around a bit * Add validation for BoundAPIGateway * APIGateway TCPRoute validation (#15855) * Add APIGateway validation * Add additional validations * Add cert ref validation * Add protobuf definitions * Fix up field types * Add API structs * Add TCPRoute normalization and validation * Add forgotten Status * Add some more field docs in api package * Fix test * Format imports * Rename snapshot test variable names * Add plumbing for Native API GW Subscriptions (#16003) Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com> Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com> Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>	2 years ago
Anita Akaeze	4e154144a6	NET-2038: Add envoy assertion function of listener verification (#15969)	2 years ago
Chris Thain	2f4c8e50f2	Support Vault agent auth config for AWS/GCP CA provider auth (#15970)	2 years ago
Derek Menteer	2facf50923	Fix configuration merging for implicit tproxy upstreams. (#16000) ... Fix configuration merging for implicit tproxy upstreams. Change the merging logic so that the wildcard upstream has correct proxy-defaults and service-defaults values combined into it. It did not previously merge all fields, and the wildcard upstream did not exist unless service-defaults existed (it ignored proxy-defaults, essentially). Change the way we fetch upstream configuration in the xDS layer so that it falls back to the wildcard when no matching upstream is found. This is what allows implicit peer upstreams to have the correct "merged" config. Change proxycfg to always watch local mesh gateway endpoints whenever a peer upstream is found. This simplifies the logic so that we do not have to inspect the "merged" configuration on peer upstreams to extract the mesh gateway mode.	2 years ago
Dan Upton	7a55de375c	xds: don't attempt to load-balance sessions for local proxies (#15789) ... Previously, we'd begin a session with the xDS concurrency limiter regardless of whether the proxy was registered in the catalog or in the server's local agent state. This caused problems for users who run `consul connect envoy` directly against a server rather than a client agent, as the server's locally registered proxies wouldn't be included in the limiter's capacity. Now, the `ConfigSource` is responsible for beginning the session and we only do so for services in the catalog. Fixes: https://github.com/hashicorp/consul/issues/15753	2 years ago
Ashlee M Boyer	02869ce9a8	[docs] Adjusting links for rewrite project (#15999)	2 years ago
Kendall Strautman	7822bbb41a	chore: updates generated author docs (#15980)	2 years ago
Dan Stough	e8dde59bd0	chore(ci): fix compat ent compat tests for sidecars and gateways (#15997)	2 years ago
trujillo-adam	dfcc11ec5f	fixes pre-devdot links in service defaults ref docs (#15989)	2 years ago
Chris S. Kim	e4a268e33e	Warn if ACL is enabled but no token is provided to Envoy (#15967)	2 years ago
Dhia Ayachi	87ff8c1c95	avoid logging RPC errors when it's specific rate limiter errors (#15968) ... * avoid logging RPC errors when it's specific rate limiter errors * simplify if statements	2 years ago
Derek Menteer	19a46d6ca4	Enforce lowercase peer names. (#15697) ... Enforce lowercase peer names. Prior to this change peer names could be mixed case. This can cause issues, as peer names are used as DNS labels in various locations. It also caused issues with envoy configuration.	2 years ago
Ranjandas	db69cd6f65	Update TG Docs with SAN match option when using SNI (#15971) ... When using SNI in Terminating Gateway, Consul configures envoy to have strict SAN matching. This requires all external services to have SANs in their certificates and not having it will throw CERTIFICATE_VERIFY_FAILED error.	2 years ago
Frank DiRocco	59a3a0749c	Update go-discover to support ECS discovery (#13782) ... Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>	2 years ago
Michael Wilkerson	d94fc2d8ad	removed Consul requirements from documentation as it may be confusing (#15958) ... * removed Consul requirements from documentation as it may be confusing	2 years ago
R.B. Boyer	d59efd390c	test: general cleanup and fixes for the container integration test suite (#15959) ... - remove dep on consul main module - use 'consul tls' subcommands instead of tlsutil - use direct json config construction instead of agent/config structs - merge libcluster and libagent packages together - more widely use BuildContext - get the OSS/ENT runner stuff working properly - reduce some flakiness - fix some correctness related to http/https API	2 years ago
Dan Stough	6d2880e894	feat: add access logs to dataplane bootstrap rpc (#15951)	2 years ago
Chris Hut	cc45c3c044	ui/fix linting error and failing ui test (#15953) ... * fix linting error * Fix the failing version test as well	2 years ago
Dan Stough	5d3643f4f0	docs(access logs): new docs for access logging (#15948) ... Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com> Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>	2 years ago
Matt Keeler	5afd4657ec	Protobuf Modernization (#15949) ... * Protobuf Modernization Remove direct usage of golang/protobuf in favor of google.golang.org/protobuf Marshallers (protobuf and json) needed some changes to account for different APIs. Moved to using the google.golang.org/protobuf/types/known/* for the well known types including replacing some custom Struct manipulation with whats available in the structpb well known type package. This also updates our devtools script to install protoc-gen-go from the right location so that files it generates conform to the correct interfaces. * Fix go-mod-tidy make target to work on all modules	2 years ago
Paul Glass	f5231b9157	Add new config_file_service_registration token (#15828)	2 years ago
Dan Stough	88b9420a1a	[OSS] feat: add access log config to consul envoy connect (#15946)	2 years ago
Chris S. Kim	a7b34d50fc	Output user-friendly name for anonymous token (#15884)	2 years ago
Dan Upton	644cd864a5	Rate limit improvements and fixes (#15917) ... - Fixes a panic when Operation.SourceAddr is nil (internal net/rpc calls) - Adds proper HTTP response codes (429 and 503) for rate limit errors - Makes the error messages clearer - Enables automatic retries for rate-limit errors in the net/rpc stack	2 years ago
Semir Patel	40c0bb24ae	emit metrics for global rate limiting (#15891)	2 years ago
Jared Kirschner	8bfdc0c02f	docs: cluster peering upgrade clarification (#15935)	2 years ago
Thomas Eckert	c708e4d5dd	Update the README for the Consul API (#15936)	2 years ago
Derek Menteer	7b4f45e2d5	Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. (#15913) ... Fix issue where TLS configuration was ignored for unix sockets in consul connect envoy. Disable xds check on bootstrap mode and change check to warn only.	2 years ago
Dhia Ayachi	233eacf0a4	inject logger and create logdrop sink (#15822) ... * inject logger and create logdrop sink * init sink with an empty struct instead of nil * wrap a logger instead of a sink and add a discard logger to avoid double logging * fix compile errors * fix linter errors * Fix bug where log arguments aren't properly formatted * Move log sink construction outside of handler * Add prometheus definition and docs for log drop counter Co-authored-by: Daniel Upton <daniel@floppy.co>	2 years ago
Eric Haberkorn	8d923c1789	Add the Lua Envoy extension (#15906)	2 years ago
Jared Kirschner	3d7fcf7795	docs: fix markdown heading levels (#15912)	2 years ago
Matt Keeler	8b4eb0c8c1	Fix usage of protoc-gen-go-binary (#15920) ... * Fix usage of protoc-gen-go-binary * Pin protoc-gen-go-binary version	2 years ago
Tu Nguyen	7b8ccca498	Update example link for Consul K8s NGINX ingress controller (#15916)	2 years ago
lbausch	4795daa809	docs: Fix link (#14890) ... Co-authored-by: David Yu <dyu@hashicorp.com>	2 years ago
claire labry	590ae5fa5c	Onboard consul to use new version/VERSION file and reproducible actions (#15631) ... * Onboard consul to use new .release/VERSION file and reproducible actions-go-build * Onboard consul to use new .release/VERSION file and reproducible actions * Onboard consul to use new .release/VERSION file and reproducible actions * fix to consul * Onboard consul to use new .release/VERSION file and reproducible actions * Onboard consul to use new .release/VERSION file and reproducible actions * Onboard consul to use new .release/VERSION file and reproducible actions * test out ent changes * just or testing * Added setup go for build ui * try removing VERSION file out of .release dir * add checkout action for build ui and update checkout version * try no -dev marker * try removing extra ldflags * test version * add back in setup-go step? * Update utils.js read from static VERSION file * remove actions-setup go * add 1.15.0-dev * Using prepare workflow for pre-stable channel workflow * Test prepare workflow * Remove set-product-version branch from release pipeline * Use METADATA in environment * Correct env vars * Remove current branch from build trigger list Co-authored-by: emilymianeil <emilymianeil@gmail.com> Co-authored-by: Sarah <sthompson@hashicorp.com> Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com> Co-authored-by: emily neil <63985869+emilymianeil@users.noreply.github.com>	2 years ago
Paul Glass	666c2b2e2b	Fix TLS_BadVerify test assertions on macOS (#15903)	2 years ago
Dan Upton	b78de5a7a2	grpc/acl: fix bug where ACL token was required even if disabled (#15904) ... Fixes a bug introduced by #15346 where we'd always require an ACL token even if ACLs were disabled because we were erroneously treating `nil` identity as anonymous.	2 years ago
Dan Upton	ee2d47da83	Fix missing comment in generated enterprise rate-limit file (#15895) ... Fixes bug in #15564 where gofmt would strip out the generated code warning comment because it was on the same line as the build tag.	2 years ago
Dan Upton	d53ce39c32	grpc: switch servers and retry on error (#15892) ... This is the OSS portion of enterprise PR 3822. Adds a custom gRPC balancer that replicates the router's server cycling behavior. Also enables automatic retries for RESOURCE_EXHAUSTED errors, which we now get for free.	2 years ago
Nick Irvine	6fb628c07d	fix: return error when config file with unknown extension is passed (#15107)	2 years ago
Tu Nguyen	01a4510d6c	Update api gateway version to latest (#15197) ... * Update api gateway version to latest * Update website/content/docs/api-gateway/install.mdx * update to latest apigw version 0.5.1 * update consul and helm version	2 years ago
cskh	1113314641	Refactoring the peering integ test to accommodate coming changes of o… (#15885) ... * Refactoring the peering integ test to accommodate coming changes of other upgrade scenarios. - Add a utils package under test that contains methods to set up various test scenarios. - Deduplication: have a single CreatingPeeringClusterAndSetup replace CreatingAcceptingClusterAndSetup and CreateDialingClusterAndSetup. - Separate peering cluster creation and server registration. * Apply suggestions from code review Co-authored-by: Dan Stough <dan.stough@hashicorp.com>	2 years ago
Florian Apolloner	077b0a48a3	Allow Operator Generated bootstrap token (#14437) ... Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520	2 years ago
Semir Patel	a6482341a5	Wire up the rate limiter to net/rpc calls (#15879)	2 years ago
Tu Nguyen	4bbfad8dca	Update links to new get started tutorials (#15154)	2 years ago
Dan Upton	d4c435856b	grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) ... Adds automation for generating the map of `gRPC Method Name → Rate Limit Type` used by the middleware introduced in #15550, and will ensure we don't forget to add new endpoints. Engineers must annotate their RPCs in the proto file like so: ``` rpc Foo(FooRequest) returns (FooResponse) { option (consul.internal.ratelimit.spec) = { operation_type: READ, }; } ``` When they run `make proto` a protoc plugin `protoc-gen-consul-rate-limit` will be installed that writes rate-limit specs as a JSON array to a file called `.ratelimit.tmp` (one per protobuf package/directory). After running Buf, `make proto` will execute a post-process script that will ingest all of the `.ratelimit.tmp` files and generate a Go file containing the mappings in the `agent/grpc-middleware` package. In the enterprise repository, it will write an additional file with the enterprise-only endpoints. If an engineer forgets to add the annotation to a new RPC, the plugin will return an error like so: ``` RPC Foo is missing rate-limit specification, fix it with: import "proto-public/annotations/ratelimit/ratelimit.proto"; service Bar { rpc Foo(...) returns (...) { option (hashicorp.consul.internal.ratelimit.spec) = { operation_type: OPERATION_READ | OPERATION_WRITE | OPERATION_EXEMPT, }; } } ``` In the future, this annotation can be extended to support rate-limit category (e.g. KV vs Catalog) and to determine the retry policy.	2 years ago
Dan Upton	7c7503c849	grpc/acl: relax permissions required for "core" endpoints (#15346) ... Previously, these endpoints required `service:write` permission on _any_ service as a sort of proxy for "is the caller allowed to participate in the mesh?". Now, they're called as part of the process of establishing a server connection by any consumer of the consul-server-connection-manager library, which will include non-mesh workloads (e.g. Consul KV as a storage backend for Vault) as well as ancillary components such as consul-k8s' acl-init process, which likely won't have `service:write` permission. So this commit relaxes those requirements to accept *any* valid ACL token on the following gRPC endpoints: - `hashicorp.consul.dataplane.DataplaneService/GetSupportedDataplaneFeatures` - `hashicorp.consul.serverdiscovery.ServerDiscoveryService/WatchServers` - `hashicorp.consul.connectca.ConnectCAService/WatchRoots`	2 years ago
Hans Hasselberg	275a0b8e7f	fix cli string for id flag (#15695)	2 years ago
Derek Menteer	1f7e7abeac	Fix issue with incorrect proxycfg watch on upstream peer-targets. (#15865) ... This fixes an issue where the incorrect partition was given to the upstream target watch, which meant that failover logic would not work correctly.	2 years ago