* add docs for consul-k8s config read command
This PR adds documentation for the functionality introduced in
https://github.com/hashicorp/consul-k8s/pull/2078.
* add output
---------
Co-authored-by: David Yu <dyu@hashicorp.com>
Fix ACL check on health endpoint
Prior to this change, the service health API would not explicitly return an
error whenever a token with invalid permissions was given, and it would instead
return empty results. With this change, a "Permission denied" error is returned
whenever data is queried. This is done to better support the agent cache, which
performs a fetch backoff sleep whenever ACL errors are encountered. Affected
endpoints are: `/v1/health/connect/` and `/v1/health/ingress/`.
* agent: configure server lastseen timestamp
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use correct config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* use default age in test golden data
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add changelog
Signed-off-by: Dan Bond <danbond@protonmail.com>
* fix runtime test
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add server_metadata
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* correctly check if metadata file does not exist
Signed-off-by: Dan Bond <danbond@protonmail.com>
* follow instructions for adding new config
Signed-off-by: Dan Bond <danbond@protonmail.com>
* add comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* update comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* Update agent/agent.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* agent/config: add validation for duration with min
Signed-off-by: Dan Bond <danbond@protonmail.com>
* docs: add new server_rejoin_age_max config definition
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: add unit test for checking server last seen
Signed-off-by: Dan Bond <danbond@protonmail.com>
* agent: log continually for 60s before erroring
Signed-off-by: Dan Bond <danbond@protonmail.com>
* pr comments
Signed-off-by: Dan Bond <danbond@protonmail.com>
* remove unneeded todo
* agent: fix error message
Signed-off-by: Dan Bond <danbond@protonmail.com>
---------
Signed-off-by: Dan Bond <danbond@protonmail.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
* snapshot: some improvments to the snapshot process
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Remove outdated usage of "Consul Connect" instead of Consul service mesh.
The connect subsystem in Consul provides Consul's service mesh capabilities.
However, the term "Consul Connect" should not be used as an alternative to
the name "Consul service mesh".
* Add MaxEjectionPercent to config entry
* Add BaseEjectionTime to config entry
* Add MaxEjectionPercent and BaseEjectionTime to protobufs
* Add MaxEjectionPercent and BaseEjectionTime to api
* Fix integration test breakage
* Verify MaxEjectionPercent and BaseEjectionTime in integration test upstream confings
* Website docs for MaxEjectionPercent and BaseEjection time
* Add `make docs` to browse docs at http://localhost:3000
* Changelog entry
* so that is the difference between consul-docker and dev-docker
* blah
* update proto funcs
* update proto
---------
Co-authored-by: Maliz <maliheh.monshizadeh@hashicorp.com>
* added method for converting SamenessGroupConfigEntry
- added new method `ToQueryFailoverTargets` for converting a SamenessGroupConfigEntry's members to a list of QueryFailoverTargets
- renamed `ToFailoverTargets` ToServiceResolverFailoverTargets to distinguish it from `ToQueryFailoverTargets`
* Added SamenessGroup to PreparedQuery
- exposed Service.Partition to API when defining a prepared query
- added a method for determining if a QueryFailoverOptions is empty
- This will be useful for validation
- added unit tests
* added method for retrieving a SamenessGroup to state store
* added logic for using PQ with SamenessGroup
- added branching path for SamenessGroup handling in execute. It will be handled separate from the normal PQ case
- added a new interface so that the `GetSamenessGroupFailoverTargets` can be properly tested
- separated the execute logic into a `targetSelector` function so that it can be used for both failover and sameness group PQs
- split OSS only methods into new PQ OSS files
- added validation that `samenessGroup` is an enterprise only feature
* added documentation for PQ SamenessGroup
Prior to this change, peer services would be targeted by service-default
overrides as long as the new `peer` field was not found in the config entry.
This commit removes that deprecated backwards-compatibility behavior. Now
it is necessary to specify the `peer` field in order for upstream overrides
to apply to a peer upstream.
* Fix API GW broken link
* Update website/content/docs/api-gateway/upgrades.mdx
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
This is part of an effort to raise awareness that you need to monitor
your mesh CA if coming from an external source as you'll need to manage
the rotation.
* converted intentions conf entry to ref CT format
* set up intentions nav
* add page for intentions usage
* final intentions usage page
* final intentions overview page
* fixed old relative links
* updated diagram for overview
* updated links to intentions content
* fixed typo in updated links
* rename intentions overview page file to index
* rollback link updates to intentions overview
* fixed nav
* Updated custom HTML in API and CLI pages to MD
* applied suggestions from review to index page
* moved conf examples from usage to conf ref
* missed custom HTML section
* applied additional feedback
* Apply suggestions from code review
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* updated headings in usage page
* renamed files and udpated nav
* updated links to new file names
* added redirects and final tweaks
* typo
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* Fix broken links in Consul docs
* more broken link fixes
* more 404 fixes
* 404 fixes
* broken link fix
---------
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* First cluster grpc service should be NodePort
This is based on the issue opened here https://github.com/hashicorp/consul-k8s/issues/1903
If you follow the documentation https://developer.hashicorp.com/consul/docs/k8s/deployment-configurations/single-dc-multi-k8s exactly as it is, the first cluster will only create the consul UI service on NodePort but not the rest of the services (including for grpc). By default, from the helm chart, they are created as headless services by setting clusterIP None. This will cause an issue for the second cluster to discover consul server on the first cluster over gRPC as it cannot simply cannot through gRPC default port 8502 and it ends up in an error as shown in the issue https://github.com/hashicorp/consul-k8s/issues/1903
As a solution, the grpc service should be exposed using NodePort (or LoadBalancer). I added those changes required in both cluster1-values.yaml and cluster2-values.yaml, and also a description for those changes for the normal users to understand. Kindly review and I hope this PR will be accepted.
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/deployment-configurations/single-dc-multi-k8s.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Ashvitha Sridharan <ashvitha.sridharan@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Add a new envoy flag: "envoy_hcp_metrics_bind_socket_dir", a directory
where a unix socket will be created with the name
`<namespace>_<proxy_id>.sock` to forward Envoy metrics.
If set, this will configure:
- In bootstrap configuration a local stats_sink and static cluster.
These will forward metrics to a loopback listener sent over xDS.
- A dynamic listener listening at the socket path that the previously
defined static cluster is sending metrics to.
- A dynamic cluster that will forward traffic received at this listener
to the hcp-metrics-collector service.
Reasons for having a static cluster pointing at a dynamic listener:
- We want to secure the metrics stream using TLS, but the stats sink can
only be defined in bootstrap config. With dynamic listeners/clusters
we can use the proxy's leaf certificate issued by the Connect CA,
which isn't available at bootstrap time.
- We want to intelligently route to the HCP collector. Configuring its
addreess at bootstrap time limits our flexibility routing-wise. More
on this below.
Reasons for defining the collector as an upstream in `proxycfg`:
- The HCP collector will be deployed as a mesh service.
- Certificate management is taken care of, as mentioned above.
- Service discovery and routing logic is automatically taken care of,
meaning that no code changes are required in the xds package.
- Custom routing rules can be added for the collector using discovery
chain config entries. Initially the collector is expected to be
deployed to each admin partition, but in the future could be deployed
centrally in the default partition. These config entries could even be
managed by HCP itself.
* fixes for unsupported partitions field in CRD metadata block
* Apply suggestions from code review
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
---------
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
* Update the consul-k8s cli docs for the new `proxy log` subcommand
* Updated consul-k8s docs from PR feedback
* Added proxy log command to release notes
Updated Params field to re-frame as supporting arguments specific to the
supported vault-agent auth-auth methods with links to each methods
"#configuration" section.
Included a call out limits on parameters supported.
* Leverage ServiceResolver ConnectTimeout for route timeouts to make TerminatingGateway upstream timeouts configurable
* Regenerate golden files
* Add RequestTimeout field
* Add changelog entry
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* converted main services page to services overview page
* set up services usage dirs
* added Define Services usage page
* converted health checks everything page to Define Health Checks usage page
* added Register Services and Nodes usage page
* converted Query with DNS to Discover Services and Nodes Overview page
* added Configure DNS Behavior usage page
* added Enable Static DNS Lookups usage page
* added the Enable Dynamic Queries DNS Queries usage page
* added the Configuration dir and overview page - may not need the overview, tho
* fixed the nav from previous commit
* added the Services Configuration Reference page
* added Health Checks Configuration Reference page
* updated service defaults configuraiton entry to new configuration ref format
* fixed some bad links found by checker
* more bad links found by checker
* another bad link found by checker
* fixed cross-links between new topics
* updated links to the new services pages
* fixed bad links in scale file
* tweaks to titles and phrasing
* fixed typo in checks.mdx
* started updating the conf ref to latest template
* update SD conf ref to match latest CT standard
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* remove previous version of the checks page
* fixed cross-links
* Apply suggestions from code review
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
---------
Co-authored-by: Eddie Rowe <74205376+eddie-rowe@users.noreply.github.com>
* Update ingress-gateways.mdx
Added an example of running the HELM install for the ingress gateways using values.yaml
* Apply suggestions from code review
* Update ingress-gateways.mdx
Adds closing back ticks on example command. The suggesting UI strips them out.
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Add WAL documentation. Also fix some minor metrics registration details
* Add tests to verify metrics are registered correctly
* refactor and move wal docs
* Updates to the WAL overview page
* updates to enable WAL usage topic
* updates to the monitoring WAL backend topic
* updates for revert WAL topic
* a few tweaks to overview and udpated metadescriptions
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* make revert docs consistent with enable
* Apply suggestions from code review
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
* address feedback
* address final feedback
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
---------
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: trujillo-adam <ajosetru@gmail.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Create empty files
* Copy over content for overview
* Copy over content for usage
* Copy over content for api-gateway config
* Copy over content for http-route config
* Copy over content for tcp-route config
* Copy over content for inline-certificate config
* Add docs to the sidebar
* Clean up overview. Start cleaning up usage
* Add BETA badge to API Gateways portion of nav
* Fix header
* Fix up usage
* Fix up API Gateway config
* Update paths to be consistent w/ other gateway docs
* Fix up http-route
* Fix up inline-certificate
* rename path
* Fix up tcp-route
* Add CodeTabs
* Add headers to config pages
* Fix configuration model for http route and inline certificate
* Add version callout to API gateway overview page
* Fix values for inline certificate
* Fix values for api gateway configuration
* Fix values for TCP Route config
* Fix values for HTTP Route config
* Adds link from k8s gateway to vm gateway page
* Remove versioning warning
* Serve overview page at ../api-gateway, consistent w/ mesh-gateway
* Remove weight field from tcp-route docs
* Linking to usage instead of overview from k8s api-gateway to vm api-gateway
* Fix issues in usage page
* Fix links in usage
* Capitalize Kubernetes
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* remove optional callout
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
* Update website/content/docs/connect/gateways/api-gateway/configuration/api-gateway.mdx
* Fix formatting of Hostnames
* Update website/content/docs/api-gateway/index.mdx
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
* Add cross-linking of config entries
* Fix rendering error on new operator usage docs
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/connect/gateways/api-gateway/configuration/http-route.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
* Add BETA badges to config entry links
* http route updates
* Add Enterprise keys
* Use map instead of list for meta field, use consistent formatting
* Convert spaces to tabs
* Add all Enterprise info to TCP Route
* Use pascal case for JSON api-gateway example
* Add enterprise to HCL api-gw cfg
* Use pascal case for missed JSON config fields
* Add enterprise to JSON api-gw cfg
* Add enterprise to api-gw values
* adds enterprise to http route
* Update website/content/docs/connect/gateways/api-gateway/index.mdx
Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
* Add enterprise to api-gw spec
* Add missing namespace, partition + meta to specification
* fixes for http route
* Fix ordering of API Gatetway cfg spec items
* whitespace
* Add linking of values to tcp
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Fix comma in wrong place
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Move Certificates down
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Tabs to spaces in httproute
* Use configuration entry instead of config entry
* Fix indentations on api-gateway and tcp-route
* Add whitespace between code block and prose
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* adds <> to http route
---------
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Melisa Griffin <melisa.griffin@hashicorp.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Tu Nguyen <im2nguyen@gmail.com>
Co-authored-by: Melisa Griffin <missylbytes@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Remove Consul Client installation option
With Consul-K8S 1.0 and introduction of Consul-Dataplane, K8S has
the option to run without running Consul Client agents.
* remove note referring to the same documentation
* Added instructions on the use of httpsPort when servers are not running TLS enabled
* Modified titile and description
* Update existing docs from Consul API Gateway -> API Gateway for Kubernetes
* Update page header to reflect page title change
* Update nav title to match new page title
* for tab testing
* updates
* Update
* adding sandbox to test conf ref types
* testing tweaks to the conf ref template
* reintroduce tabbed specification
* applied feedback from MKO session
* applied feedback on format from luke and jared
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* fixed some minor HCL formatting in complete conf
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* fixed bad link
* resolving conflicts
---------
Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>
Co-authored-by: Dan Upton <daniel@floppy.co>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* remove legacy tokens
* remove lingering legacy token references from docs
* update language and naming for token secrets and accessor IDs
* updates all tokenID references to clarify accessorID
* remove token type references and lookup tokens by accessorID index
* remove unnecessary constants
* replace additional tokenID param names
* Add warning info for deprecated -id parameter
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* Update field comment
Co-authored-by: Paul Glass <pglass@hashicorp.com>
---------
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* doc: clarify the default time unit in service-resolver.ConnectTimeout
* Update website/content/docs/connect/config-entries/service-resolver.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
The generate_lease=true configuration is unnecessary and generates a note about performance implications in Vault logs. Remove this configuration so that the default value of generate_lease=false is used instead.
* Document how numRetries can't be set to 0
Resolves https://github.com/hashicorp/consul/issues/11816 and https://github.com/hashicorp/consul/issues/8516.
* Update website/content/docs/connect/config-entries/service-router.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
---------
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update connect-internals.mdx
Removed most references for 'Connect' given the terminology has long been deprecated in official use.
* Apply suggestions from code review
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* Update connect-internals.mdx
Updates based on Blakes recommendations
* Update connect-internals.mdx
---------
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Kendall Strautman <36613477+kendallstrautman@users.noreply.github.com>
* remove legacy tokens
* Update test comment
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* fix imports
* update docs for additional CLI changes
* add test case for anonymous token
* set deprecated api fields to json ignore and fix patch errors
* update changelog to breaking-change
* fix import
* update api docs to remove legacy reference
* fix docs nav data
---------
Co-authored-by: Paul Glass <pglass@hashicorp.com>
* Initial page and nav data
* Formatting
* Fixes
* Page description
* DNS lookup fixes
* admin partition link
* Control Plane Resiliency rephrase
* Dataplanes/xDS callout
* word choice correction
* Consul as Vault backend clarifications
* Link to blog post on testing
* Update website/content/docs/architecture/scale.mdx
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Update website/content/docs/architecture/scale.mdx
* Apply suggestions from code review
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* Update website/content/docs/architecture/scale.mdx
* Update website/content/docs/architecture/scale.mdx
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Add support for envoy readiness flags
- add flags 'envoy-ready-bind-port` and `envoy-ready-bind-addr` on consul connect envoy to create a ready listener on that address.
* Stub Config Entries for Consul Native API Gateway (#15644)
* Add empty InlineCertificate struct and protobuf
* apigateway stubs
* Stub HTTPRoute in api pkg
* Stub HTTPRoute in structs pkg
* Simplify api.APIGatewayConfigEntry to be consistent w/ other entries
* Update makeConfigEntry switch, add docstring for HTTPRouteConfigEntry
* Add TCPRoute to MakeConfigEntry, return unique Kind
* Stub BoundAPIGatewayConfigEntry in agent
* Add RaftIndex to APIGatewayConfigEntry stub
* Add new config entry kinds to validation allow-list
* Add RaftIndex to other added config entry stubs
* Update usage metrics assertions to include new cfg entries
* Add Meta and acl.EnterpriseMeta to all new ConfigEntry types
* Remove unnecessary Services field from added config entry types
* Implement GetMeta(), GetEnterpriseMeta() for added config entry types
* Add meta field to proto, name consistently w/ existing config entries
* Format config_entry.proto
* Add initial implementation of CanRead + CanWrite for new config entry types
* Add unit tests for decoding of new config entry types
* Add unit tests for parsing of new config entry types
* Add unit tests for API Gateway config entry ACLs
* Return typed PermissionDeniedError on BoundAPIGateway CanWrite
* Add unit tests for added config entry ACLs
* Add BoundAPIGateway type to AllConfigEntryKinds
* Return proper kind from BoundAPIGateway
* Add docstrings for new config entry types
* Add missing config entry kinds to proto def
* Update usagemetrics_oss_test.go
* Use utility func for returning PermissionDeniedError
* EventPublisher subscriptions for Consul Native API Gateway (#15757)
* Create new event topics in subscribe proto
* Add tests for PBSubscribe func
* Make configs singular, add all configs to PBToStreamSubscribeRequest
* Add snapshot methods
* Add config_entry_events tests
* Add config entry kind to topic for new configs
* Add unit tests for snapshot methods
* Start adding integration test
* Test using the new controller code
* Update agent/consul/state/config_entry_events.go
* Check value of error
* Add controller stubs for API Gateway (#15837)
* update initial stub implementation
* move files, clean up mutex references
* Remove embed, use idiomatic names for constructors
* Remove stray file introduced in merge
* Add APIGateway validation (#15847)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* APIGateway InlineCertificate validation (#15856)
* Add APIGateway validation
* Add additional validations
* Add protobuf definitions
* Tabs to spaces
* Add API structs
* Move struct fields around a bit
* Add validation for InlineCertificate
* Fix ACL test
* APIGateway BoundAPIGateway validation (#15858)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Move struct fields around a bit
* Add validation for BoundAPIGateway
* APIGateway TCPRoute validation (#15855)
* Add APIGateway validation
* Add additional validations
* Add cert ref validation
* Add protobuf definitions
* Fix up field types
* Add API structs
* Add TCPRoute normalization and validation
* Add forgotten Status
* Add some more field docs in api package
* Fix test
* Format imports
* Rename snapshot test variable names
* Add plumbing for Native API GW Subscriptions (#16003)
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
Co-authored-by: Andrew Stucki <andrew.stucki@hashicorp.com>
When using SNI in Terminating Gateway, Consul configures envoy to
have strict SAN matching. This requires all external services to
have SANs in their certificates and not having it will throw
CERTIFICATE_VERIFY_FAILED error.
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* inject logger and create logdrop sink
* init sink with an empty struct instead of nil
* wrap a logger instead of a sink and add a discard logger to avoid double logging
* fix compile errors
* fix linter errors
* Fix bug where log arguments aren't properly formatted
* Move log sink construction outside of handler
* Add prometheus definition and docs for log drop counter
Co-authored-by: Daniel Upton <daniel@floppy.co>
* Update api gateway version to latest
* Update website/content/docs/api-gateway/install.mdx
* update to latest apigw version 0.5.1
* update consul and helm version
* add functions for returning the max and min Envoy major versions
- added an UnsupportedEnvoyVersions list
- removed an unused error from TestDetermineSupportedProxyFeaturesFromString
- modified minSupportedVersion to use the function for getting the Min Envoy major version. Using just the major version without the patch is equivalent to using `.0`
* added a function for executing the envoy --version command
- added a new exec.go file to not be locked to unix system
* added envoy version check when using consul connect envoy
* added changelog entry
* added docs change
* added a NS folder and refactored main page into the overview page
* added NS usage page to NS folder
* updated links to NS docs
* updated nav
* addressed feedback from review
* update docs to reflect vault and consul compatibility
* Update website/content/docs/connect/ca/vault.mdx
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* server: add placeholder glue for rate limit handler
This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.
This allows us to start working on the net/rpc and gRPC interceptors and
config logic.
* Add handler errors
* Set the global read and write limits
* fixing multilimiter moving packages
* Fix typo
* Simplify globalLimit usage
* add multilimiter and tests
* exporting LimitedEntity
* Apply suggestions from code review
Co-authored-by: John Murret <john.murret@hashicorp.com>
* add config update and rename config params
* add doc string and split config
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* use timer to avoid go routine leak and change the interface
* add comments to tests
* fix failing test
* add prefix with config edge, refactor tests
* Apply suggestions from code review
Co-authored-by: Dan Upton <daniel@floppy.co>
* refactor to apply configs for limiters under a prefix
* add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic
* make KeyType an exported type
* split the config and limiter trees to fix race conditions in config update
* rename variables
* fix race in test and remove dead code
* fix reconcile loop to not create a timer on each loop
* add extra benchmark tests and fix tests
* fix benchmark test to pass value to func
* server: add placeholder glue for rate limit handler
This commit adds a no-op implementation of the rate-limit handler and
adds it to the `consul.Server` struct and setup code.
This allows us to start working on the net/rpc and gRPC interceptors and
config logic.
* Set the global read and write limits
* fixing multilimiter moving packages
* add server configuration for global rate limiting.
* remove agent test
* remove added stuff from handler
* remove added stuff from multilimiter
* removing unnecessary TODOs
* Removing TODO comment from handler
* adding in defaulting to infinite
* add disabled status in there
* adding in documentation for disabled mode.
* make disabled the default.
* Add mock and agent test
* addig documentation and missing mock file.
* Fixing test TestLoad_IntegrationWithFlags
* updating docs based on PR feedback.
* Updating Request Limits mode to use int based on PR feedback.
* Adding RequestLimits struct so we have a nested struct in ReloadableConfig.
* fixing linting references
* Update agent/consul/rate/handler.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* Update agent/consul/config.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* removing the ignore of the request limits in JSON. addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf
* added conversion function to convert request limits object to handler config.
* Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well.
* Updating values for TestLoad_FullConfig() so that they were different and discernable.
* Updating TestRuntimeConfig_Sanitize
* Fixing TestLoad_IntegrationWithFlags test
* putting nil check in place
* fixing rebase
* removing change for missing error checks. will put in another PR
* Rebasing after default multilimiter config change
* resolving rebase issues
* updating reference for incomingRPCLimiter to use interface
* updating interface
* Updating interfaces
* Fixing mock reference
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
* feat(ingress-gateway): support outlier detection of upstream service for ingress gateway
* changelog
Co-authored-by: Eric Haberkorn <erichaberkorn@gmail.com>
* Fixing CLI instructions so that the installing a cluster that is embedded in the mac instructions is moved outside of the tabbed instructions.
Co-authored-by: boruszak <jeffrey.boruszak@hashicorp.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* Table update
* Old table re-added and alphabetized
* Update website/content/docs/enterprise/index.mdx
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
* added usage folder to organize use case docs for CAPIgw
* Add peer field to MeshService configuration page
* Add first pass at guide for routing to peered services
* Add exception to same-datacenter restriction for referenced Consul service
* Add example HTTPRoute referencing the MeshService as backendRef
* Add example ServiceResolver
* Add note about current ServiceResolver requirement
ServiceResolver may eventually be created implicitly by the API gateway controller, but that decision is pending.
* tweaks to the usage page for routing to peered services
* tweaks to the description in the configuration reference
* resolved TO-DOs from previous iteration
* Remove datacenter federation from limited support matrix
* added tolerations doc
* Remove note excluding k8s 1.24 since we now support it
* Reorder sections to maintain alphabetical sort
* Add example configuration for MeshService resource
* Adjust wording + indentation of other docs
* Use consistent "example-" prefix for resource names in example code
* reframed the tolerations documentation; STILL A WIP
* add helm chart documentation
* removed tolerations from gwcconfig configuration model reference
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* update version to 0.5.0
* Update install.mdx
* added release notes for v.0.5.x
Co-authored-by: Nathan Coleman <nathan.coleman@hashicorp.com>
Co-authored-by: Sarah Alsmiller <sarah.alsmiller@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com>
* Consul Architecture update
* Consul on Kubernetes architecture
* Install Consul on Kubernetes with Helm updates
* Vault as the Secrets Backend Data Integration
* Kubernetes Service Mesh Overview
* Terminating Gateways
* Fully updated
* Join external service to k8s
* Consul on Kubernetes
* Configure metrics for Consul on Kubernetes
* Service Sync for Consul on Kubernetes
* Custom Resource Definitions for Consul on k8s
* Upgrading Consul on Kubernetes Components
* Rolling Updates to TLS
* Dataplanes diagram
* Upgrade instructions
* k8s architecture page updates
* Update website/content/docs/k8s/connect/observability/metrics.mdx
Co-authored-by: Riddhi Shah <riddhi@hashicorp.com>
* Update website/content/docs/architecture/index.mdx
* Update website/content/docs/k8s/connect/terminating-gateways.mdx
* CRDs
* updating version numbers
* Updated example config
* Image clean up
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update website/content/docs/k8s/architecture.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Riddhi Shah <riddhi@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Update guidance for vault PKI CA provider
* clarify workarounds if already using vault 1.11+
* Update website/content/docs/connect/ca/vault.mdx
* Update website/content/docs/k8s/connect/connect-ca-provider.mdx
* Update website/content/docs/k8s/deployment-configurations/vault/data-integration/connect-ca.mdx
* Apply suggestions from code review
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* add suggestion from Matt
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* fix: clarifying error message when acquiring a lock in remote dc
* Update website/content/commands/lock.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* auto-config: relax node name validation for JWT authorization
This changes the JWT authorization logic to allow all non-whitespace,
non-quote characters when validating node names. Consul had previously
allowed these characters in node names, until this validation was added
to fix a security vulnerability with whitespace/quotes being passed to
the `bexpr` library. This unintentionally broke node names with
characters like `.` which aren't related to this vulnerability.
* Update website/content/docs/agent/config/cli-flags.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* add leadership transfer command
* add RPC call test (flaky)
* add missing import
* add changelog
* add command registration
* Apply suggestions from code review
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
* add the possibility of providing an id to raft leadership transfer. Add few tests.
* delete old file from cherry pick
* rename changelog filename to PR #
* rename changelog and fix import
* fix failing test
* check for OperatorWrite
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
* rename from leader-transfer to transfer-leader
* remove version check and add test for operator read
* move struct to operator.go
* first pass
* add code for leader transfer in the grpc backend and tests
* wire the http endpoint to the new grpc endpoint
* remove the RPC endpoint
* remove non needed struct
* fix naming
* add mog glue to API
* fix comment
* remove dead code
* fix linter error
* change package name for proto file
* remove error wrapping
* fix failing test
* add command registration
* add grpc service mock tests
* fix receiver to be pointer
* use defined values
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
* reuse MockAclAuthorizer
* add documentation
* remove usage of external.TokenFromContext
* fix failing tests
* fix proto generation
* Apply suggestions from code review
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
* Apply suggestions from code review
* add more context in doc for the reason
* Apply suggestions from docs code review
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
* regenerate proto
* fix linter errors
Co-authored-by: github-team-consul-core <github-team-consul-core@hashicorp.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Jared Kirschner <85913323+jkirschner-hashicorp@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Prevent serving TLS via ports.grpc
We remove the ability to run the ports.grpc in TLS mode to avoid
confusion and to simplify configuration. This breaking change
ensures that any user currently using ports.grpc in an encrypted
mode will receive an error message indicating that ports.grpc_tls
must be explicitly used.
The suggested action for these users is to simply swap their ports.grpc
to ports.grpc_tls in the configuration file. If both ports are defined,
or if the user has not configured TLS for grpc, then the error message
will not be printed.
Re-add ServerExternalAddresses parameter in GenerateToken endpoint
This reverts commit 5e156772f6
and adds extra functionality to support newer peering behaviors.
* Update Consul Dataplane CLI reference
* Add new page for Consul Dataplane telemetry
* Add `server_type` label to agent grpc metrics
* Callout Consul Dataplane in Envoy bootstrap configuration section
* Update consul-dataplane unsupported features
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Riddhi Shah <riddhi@hashicorp.com>
Fix an issue where rpc_hold_timeout was being used as the timeout for non-blocking queries. Users should be able to tune read timeouts without fiddling with rpc_hold_timeout. A new configuration `rpc_read_timeout` is created.
Refactor some implementation from the original PR 11500 to remove the misleading linkage between RPCInfo's timeout (used to retry in case of certain modes of failures) and the client RPC timeouts.
Care must be taken when replacing mesh gateways in the primary
datacenter, because if the old addresses become unreachable before the
secondary datacenters receive the new addresses then the primary
datacenter overall will become unreachable.
This commit adds docs related to this class of upgrades.
In 1.13.2 we added a new flag called use_auto_cert to address issues
previously documented in the upgrade guide. Originally there was no way
to disable TLS for gRPC when auto-encrypt was in use, because TLS was
enabled for gRPC due to the presence of auto-encrypt certs.
As of 1.13.2, using auto-encrypt certs as the signal to enable TLS for
gRPC is opt-in only. Meaning that if anyone who had upgraded to 1.13
relied on that side-effect, they now need to explicitly configure it.
Adds a user-configurable rate limiter to proxycfg snapshot delivery,
with a default limit of 250 updates per second.
This addresses a problem observed in our load testing of Consul
Dataplane where updating a "global" resource such as a wildcard
intention or the proxy-defaults config entry could starve the Raft or
Memberlist goroutines of CPU time, causing general cluster instability.
* Add metrics for consul.memberlist.node.instances, consul.memberlist.queue.broadcast, consul.memberlist.size.local, and consul.memberlist.size.remote
* Fixing last table on page that does not render properly
* fixing queue name
Luke Kysow