It is not clear that this page is to configure an external CA for Connect CA. Added line to clarify that this page is for configuring external CA's for the Connect CA. For the built-in CA, no config is needed.
* docs: Update uninstall to ensure CRDs are deleted
* Update website/content/docs/k8s/operations/uninstall.mdx
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* add more details around CRD deletion
* move around crd deletion to before unsintall
* slight wording
* move deletion of CRDs to first line
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* Clarify CTS monitoring of service and instances
Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Add overview example for multiple interfaces with go-sockaddr
* Include go-sockaddr examples in agent configuration
* Add changelog entry
* Make suggested changes
* Simplify hcl comment
* Update link and fix gRPC
* Switch index.mdx from Tabs to CodeTabs
* Reformat new links for screen readers
* Apply suggestions from code review
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
* Fix spacing in code block
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
This section was actually about authentication (not authorization).
We already have sections in our api and cli docs. This commit removes the section and replaces
it with a short paragraph in the Tokens section which links to the existing docs.
- Split examples into sections with headers
- Hide the clipboard on examples as the copied text isn't useful
- Format inline flags as code using backticks
- Split examples into sections with headers
- Hide the clipboard on examples as the copied text isn't useful
- Add an example of supplying data in a heredoc
- Move the flags section to the bottom to clearly separate it from CAS
which also mentions "flags" of a different kind
- Slight re-wording for clarity
The mesh gateway docs at /docs/connect/gateways/mesh-gateway were
moved in #11859 to a new location in order to accommodate the addition
of separate instructions for using gateways with admin partitions.
This commit redirects the old mesh gateway page to its new location at
/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters.
K8s Vault CA config docs:
* Re-add filename label on K8s Connect CA config.
* Remove call to `jq` when retrieving CA configuration.
* Clarify `connect.ca_config` and `connect.ca_provider` agent configs
are only used at cluster initialization.
Admin Partitions tutorial:
* Fix Helm client values filename.
* Use kubectl's template output to base64 decode Consul bootstrap token.
* Add documentation for the consul with vault integration that covers Server TLS, Connect CA and gossip encryption
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
* Documenting the new raft_boltdb configuration options
* Add documentation around new boltdb metrics.
* Correct documentation for the consul.raft.fsm.apply metric
* update connect ca leaf endpoint docs
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
* pr feedback
* Update website/content/api-docs/agent/connect.mdx
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* nia/docs: Add TLS options for the CTS API
* docs: Add workspace tags (#11564)
* nia/docs: Change CLI options to table format
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>
* nia/docs: Update TLS CLI defaults
Also clarifies some behavior for the CLI options.
Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: Kim Ngo <6362111+findkim@users.noreply.github.com>
Co-authored-by: Melissa Kam <3768460+mkam@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>
* Support vault auth methods for the Vault connect CA provider
* Rotate the token (re-authenticate to vault using auth method) when the token can no longer be renewed
changed 'segments' in this page to 'resource labels' to disambiguate from 'network segments
updated the code snippets to use CodeBlock component and to include JSON
* Support Vault Namespaces explicitly in CA config
If there is a Namespace entry included in the Vault CA configuration,
set it as the Vault Namespace on the Vault client
Currently the only way to support Vault namespaces in the Consul CA
config is by doing one of the following:
1) Set the VAULT_NAMESPACE environment variable which will be picked up
by the Vault API client
2) Prefix all Vault paths with the namespace
Neither of these are super pleasant. The first requires direct access
and modification to the Consul runtime environment. It's possible and
expected, not super pleasant.
The second requires more indepth knowledge of Vault and how it uses
Namespaces and could be confusing for anyone without that context. It
also infers that it is not supported
* Add changelog
* Remove fmt.Fprint calls
* Make comment clearer
* Add next consul version to website docs
* Add new test for default configuration
* go mod tidy
* Add skip if vault not present
* Tweak changelog text
* docs: consul-k8s uninstall with namespace
Uninstall with namespace
* change release name to consul in uninstall
* Update website/content/docs/k8s/operations/uninstall.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* add --create-namespace command to install for custom values file
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
The Consul UI topology view has an icon with the text
"Configure metrics dashboard" that links to this page. Add a notice at
the top of the page that links them directly to the relevant section.
The UpstreamConfig.Defaults field does not support setting Name or
Namespace because the purpose is to apply defaults to all upstreams.
I think this was just missed in the docs since those fields would
error if set under Defaults.
i.e. this is not supported:
```
UpstreamConfig {
Defaults {
Name = "foo"
Namespace = "bar"
# Defaults config here
}
}
```
* add root_cert_ttl option for consul connect, vault ca providers
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
* add changelog, pr feedback
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
* Update .changelog/11428.txt, more docs
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
* Update website/content/docs/agent/options.mdx
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
* docs: revised Helm install to create namespace and install on dedicated Consul namespace
* Update website/content/docs/k8s/installation/install.mdx
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
* Update install.mdx
* changing to Helm 3.2+ as a pre-req to make it easier to follow
* might as well bump to latest version
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
* Update share card image & switch to og-image
* Remove path from api-docs-nav-data. Working
* Add redirect back in
Co-authored-by: Pamela Bortnick <pbortnick@gmail.com>
* Start new page
* reset
* Consul on Kubernetes - Features List Section (#11078)
* Fix conflicts
* Start adding and placing content
* Add a lot of styling and background image
* Looking like design. Before adding more global styles
* Work on editing styles
* Move imports and switch to flex
* Add more styles and bottom left background image
* Fix conflicts
* Fix styles on mobile
* Change images path in public
* Updates from code review - Move data and update styles
* Convert to tsx and add types
* Change button props to get desired styling without using css
* Remove margin on mobile
* Consul On Kubernetes - Hero (#11089)
* Start hero section
* Increase max sizes for container
* Minor Edits
* Use g-grid-container
* Edit video styles and test
* Rename component
* Start moving global styles over
* Move hero styles to locl
* Remove composes g-hero
* Fix flex basis on media
* Add display flex to media
* Clean up some styles
* Consul On Kubernetes - Block List (#11114)
* Start block list
* Enter data and start styles
* Get all images in and update styles
* Move data and convert to ts
* Add comment in page
* Consul On Kubernetes - Side by Side Section (#11122)
* Start block list
* Enter data and start styles
* Get all images in and update styles
* Start side by side
* Add content and more styles
* Some more styles
* Add styles for text and titles
* Edit styles and clean
* Fix spacing between button and text in overview
* Delete public folder
* Fix images in block on page
* remove extra file
* Fix classnames import
* Use fragment
* Consul On Kubernetes - Docs List Component (#11150)
* Add docs list component
* Add docs list content
* Change type declaration
* Remove unecessary style
* Use fragment
* Change icons
* Consul On Kubernetes - Card List & Get Started Section (#11168)
* Start card list component
* Begin adding content
* Start wrapper styles
* Add more styles for card
* Fix style
* Edit styles
* Use next Link
* Add minor formatting
* Make entire card a link
* Add transition
* Change import
* Use svg instead of button and make target blank
* Move wrapper div to component and add classname prop, use color variable for border
* Change min of card in grid
* Less pxels for min
* Update copy
* Consul on Kubernetes Content (#11179)
* Add content so far
* Add hero content and switch video to embed - needs editing
* Add overview and docs links
* Use iframe in hero and style
* Remove = null on prop
* Add learn tutorials content so far
* Change learn tutorials content
* Change placeholder learn content
* Add requested copy updates
* Align numbers
* Consul on Kubernetes Content & Design Updates (#11217)
* Update docs icons
* Update learn cards
* Update challenges icon
* Video poster pending
* New image
* Add split up background images
* Looking pretty good
* Fix up background image
* Add more styles
* Add meta description and new feature images
* Revert img change
* Fix up images and replace poster
* Switch to grid
* Move images
* Clean up styles
* Change hero button text
* Update styles for hero video
* Update youtube url
* Use gray variable
* Consul on Kubernetes Metadata (#11219)
* Add meta data for sharing
* Test
* Test 2
* Add meta title
* Update share image
* CHange name
* Test
* Test again
* Use relative url
* Swap urls for hero ctas
* Update tutorial card to be uniform
* Change overview button text
* Resolve conflicts and fix dependencies
Add changelog to document what changed.
Add entry to telemetry section of the website to document what changed
Add docs to the usagemetric endpoint to help document the metrics in code
Replace it with an implementation that returns an error, and rename some symbols
to use a Deprecated suffix to make it clear.
Also remove the ACLRequest struct, which is no longer referenced.
* docs/nia: Add Consul KV condition
* docs/nia: Clarify boolean Consul KV condition options
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* nia/docs: Remove TFE-only restriction
Also updates Terraform Enterprise references to the more
general Terraform Cloud term.
* nia/docs: Update Terraform Cloud features
* nia/docs: Callouts for v0.4.0-beta
* docs/nia: Indicate version for removal of tag field
Clarifying when this tag will be removed so there is no confusion
when it is not present in the v0.4.0-beta release.
Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: Melissa Kam <3768460+mkam@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Use kubectl's base64decode template function (added in K8s 1.11) to
decode values in Secrets. Removes external call to `base64` utility on
the host system.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* agent: add failures_before_warning setting
The new setting allows users to specify the number of check failures
that have to happen before a service status us updated to be `warning`.
This allows for more visibility for detected issues without creating
alerts and pinging administrators. Unlike the previous behavior, which
caused the service status to not update until it reached the configured
`failures_before_critical` setting, now Consul updates the Web UI view
with the `warning` state and the output of the service check when
`failures_before_warning` is breached.
The default value of `FailuresBeforeWarning` is the same as the value of
`FailuresBeforeCritical`, which allows for retaining the previous default
behavior of not triggering a warning.
When `FailuresBeforeWarning` is set to a value higher than that of
`FailuresBeforeCritical it has no effect as `FailuresBeforeCritical`
takes precedence.
Resolves: https://github.com/hashicorp/consul/issues/10680
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: Kim Ngo <6362111+findkim@users.noreply.github.com>
Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: Michael Wilkerson <mwilkerson@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
It can only work if there is a running service instance in the local DC,
so this is a bit misleading, since failover and redirects are typically
used when there is not an instance in the local DC.
Add the list of common Connect CA configuration options to the
provider-specific CA docs.
Previously these options were only documented under the agent
configuration options. This change makes it so that all supported CA
provider configuration options are available from a single location.
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
The current suggests the option expects a string of either "enabled" or "disabled" but this results in an error `'acl.enable_key_list_policy' expected type 'bool', got unconvertible type 'string', value: 'enabled'`. Setting to a boolean value resolves this, also had a quick look at the code (d2b58cd0d6/agent/config/runtime.go (L109)) and it suggests this too
Add a section to the Connect Security page which highlights the risks
of exposing Envoy's administration interface outside of localhost.
Resolves#5692
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Kent 'picat' Gruber <kent@hashicorp.com>
Add section for tagged addresses on service definition documentation.
Resolves#6989
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Clarify the function of `-address` flag when instantiating an ingress
gateway.
Resolves#9849
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Missed the need to add support for unix domain socket config via
api/command line. This is a variant of the problems described in
it is easy to drop one.
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
This was necessary in older versions of Consul, but was obsoleted by
making Consul add the port number itself when constructing the Envoy
configuration.
* Use CodeTabs for examples in multiple formats.
* Ensure correct language on code fences.
* Use CodeBlockConfig for examples with filenames, or which need
highlighted content.
This commit adds example JSON configs for several config entry
resources were missing examples in this language.
The examples have been updated to use the new CodeTabs resource
instead of the Tab component.
The ServiceChecks parameter was incorrectly documented in e515c9d44 to
state that it accepted a list of string values, when actually the API
requires an array of ServiceCheck objects.
This commit updates the docs for the parameter to correctly reflect
the fields required by the API.
Resolves#10752
Add a note to the docs for the service defaults config entry which
informs users that the service protocol can be configured for all
services using the proxy defaults config entry.
Resolves#8279
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Document the namespace parameter can be specified on HTTP Check,
Connect CA leaf, and Discovery Chain API endpoints.
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
The base64 CLI utility has two different short flag arguments for decode
depending on the platform: -D and -d.
Previously, the docs used the -D flag exclusively with the base64 utility.
Luckily, the long form of the flag is the same across platforms: --decode.
All uses of the base64 -D flag have been replaced with --decode.
Update output for /v1/session/ endpoints to match output post Consul
1.7.0.
Documents new `NodeChecks` and `ServiceChecks` parameters which were
added in that release.
Resolves#7341, resolves#10095
This change adds a new `dns_config.recursor_strategy` option which
controls how Consul queries DNS resolvers listed in the `recursors`
config option. The supported options are `sequential` (default), and
`random`.
Closes#8807
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Priyanka Sengupta <psengupta@flatiron.com>
* add intermediate ca metric routine
* add Gauge config for intermediate cert
* Stop metrics routine when stopping leader
* add changelog entry
* updage changelog
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
* use variables instead of a map
* go imports sort
* Add metrics for primary and secondary ca
* start metrics routine in the right DC
* add telemetry documentation
* update docs
* extract expiry fetching in a func
* merge metrics for primary and secondary into signing ca metric
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Add high level documentation on how to enable ingress controllers in consul on k8s.
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>
The docs note an alpha pre-release version on K8s observability. Updating to only reference the GA 1.10.0 version of Consul for observability on k8s and metrics merging.
The main branch is being renamed from master->main. This commit should
update all references to the main branch to the new name.
Co-Authored-By: Mike Morris <mikemorris@users.noreply.github.com>
* docs: Add info on using cloud auto-join with Network Segments
Resolves hashicorp/go-discover#57
* Add note about joining network segments
Specifically call out that agents can be configured to join a network
segment by either specifying the Serf LAN port in the join address,
changing the agent's default Serf LAN port by configuring
`ports.serf_lan`, or specifying the port in the `-serf-lan-port`
command line flag.
Resolves#9972
* docs: Remove Helm 2 mention in Consul K8s install and uninstall
Helm 2 is no longer supported via Consul K8s. Helm 3 is now the supported version for Consul K8s.
The query metrics are actually reported for all read queries, not only
ones that use a MinIndex to block for updates.
Also clarify the raft.apply metric is only on the leader.
* docs: Removal of Consul vs ZooKeeper
Although Consul does have a KV, we are not positioning Consul as a first class KV store versus other alternatives such as etcd or Zookeeper. Will remove this since this has not been updated with further analysis since this content was created.
* Removing from Zookeeper analysis Navbar
* Removing Zookeeper analysis from redirects
* docs/nia: Add section on upgrading Terraform in CTS
* docs/nia: Add service filter configuration, deprecate tag
* docs/nia: Add version to deprecated note, use path to reference
* docs/nia: catalog-services condition
Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: Melissa Kam <3768460+mkam@users.noreply.github.com>
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.
This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
Previously if you were to follow these docs and register two external
services, you would set the Address field on the node. The second
registered service would change the address of the node for the first
service.
Now the docs explain the address key and how to register more than one
external service.
* updating hero with ecs info
* updates to hero
* Include back the Basic Hero styles
The basic hero is still used on the use case pages
* Revert the tsconfig changes
Nothing in the scope of this PR requires these changes!
* Remove the old Carousel CSS file
This is no longer needed as we're using the @hashicorp/react-hero
which comes with all the styling required for this carousel to work.
* Rename ConsulHero -> HomepageHero imports/exports
This will help prevent any confusion for future devs here -- this is a
convention we have that helps us from having to trace every import,
which helps us find the source of the component without actually having
to look at the import.
* Pin the deps
These were previously pinned to the exact version; including ^ will
allow minor & patch updates to sneak in, which normally shouldn't cause
an issue but we tend to be more conservative on dep upgrades.
* Revert unneeded changes to the document file
* Revert changes to app.js file
Not needed in the scope of this PR!
* Hard pin react-alert
* Remove unneeded css
Co-authored-by: Brandon Romano <brandon@hashicorp.com>
* debug: remove the CLI check for debug_enabled
The API allows collecting profiles even debug_enabled=false as long as
ACLs are enabled. Remove this check from the CLI so that users do not
need to set debug_enabled=true for no reason.
Also:
- fix the API client to return errors on non-200 status codes for debug
endpoints
- improve the failure messages when pprof data can not be collected
Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com>
* remove parallel test runs
parallel runs create a race condition that fail the debug tests
* snapshot the timestamp at the beginning of the capture
- timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures
- capture append to the file if it already exist
* Revert "snapshot the timestamp at the beginning of the capture"
This reverts commit c2d03346
* Refactor captureDynamic to extract capture logic for each item in a different func
* snapshot the timestamp at the beginning of the capture
- timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures
- capture append to the file if it already exist
* Revert "snapshot the timestamp at the beginning of the capture"
This reverts commit c2d03346
* Refactor captureDynamic to extract capture logic for each item in a different func
* extract wait group outside the go routine to avoid a race condition
* capture pprof in a separate go routine
* perform a single capture for pprof data for the whole duration
* add missing vendor dependency
* add a change log and fix documentation to reflect the change
* create function for timestamp dir creation and simplify error handling
* use error groups and ticker to simplify interval capture loop
* Logs, profile and traces are captured for the full duration. Metrics, Heap and Go routines are captured every interval
* refactor Logs capture routine and add log capture specific test
* improve error reporting when log test fail
* change test duration to 1s
* make time parsing in log line more robust
* refactor log time format in a const
* test on log line empty the earliest possible and return
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
* rename function to captureShortLived
* more specific changelog
Co-authored-by: Paul Banks <banks@banksco.de>
* update documentation to reflect current implementation
* add test for behavior when invalid param is passed to the command
* fix argument line in test
* a more detailed description of the new behaviour
Co-authored-by: Paul Banks <banks@banksco.de>
* print success right after the capture is done
* remove an unnecessary error check
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
* upgraded github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57 => v0.0.0-20210601050228-01bbb1931b22
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: Paul Banks <banks@banksco.de>
* Docs for Unix Domain Sockets
There are a number of cases where a user might wish to either 1)
expose a service through a Unix Domain Socket in the filesystem
('downstream') or 2) connect to an upstream service by a local unix
domain socket (upstream).
As of Consul (1.10-beta2) we've added new syntax and support to configure
the Envoy proxy to support this
To connect to a service via local Unix Domain Socket instead of a
port, add local_bind_socket_path and optionally local_bind_socket_mode
to the upstream config for a service:
upstreams = [
{
destination_name = "service-1"
local_bind_socket_path = "/tmp/socket_service_1"
local_bind_socket_mode = "0700"
...
}
...
]
This will cause Envoy to create a socket with the path and mode
provided, and connect that to service-1
The mode field is optional, and if omitted will use the default mode
for Envoy. This is not applicable for abstract sockets. See
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/address.proto#envoy-v3-api-msg-config-core-v3-pipe
for details
NOTE: These options conflict the local_bind_socket_port and
local_bind_socket_address options. We can bind to an port or we can
bind to a socket, but not both.
To expose a service listening on a Unix Domain socket to the service
mesh use either the 'socket_path' field in the service definition or the
'local_service_socket_path' field in the proxy definition. These
fields are analogous to the 'port' and 'service_port' fields in their
respective locations.
services {
name = "service-2"
socket_path = "/tmp/socket_service_2"
...
}
OR
proxy {
local_service_socket_path = "/tmp/socket_service_2"
...
}
There is no mode field since the service is expected to create the
socket it is listening on, not the Envoy proxy.
Again, the socket_path and local_service_socket_path fields conflict
with address/port and local_service_address/local_service_port
configuration entries.
Set up a simple service mesh with dummy services:
socat -d UNIX-LISTEN:/tmp/downstream.sock,fork UNIX-CONNECT:/tmp/upstream.sock
socat -v tcp-l:4444,fork exec:/bin/cat
services {
name = "sock_forwarder"
id = "sock_forwarder.1"
socket_path = "/tmp/downstream.sock"
connect {
sidecar_service {
proxy {
upstreams = [
{
destination_name = "echo-service"
local_bind_socket_path = "/tmp/upstream.sock"
config {
passive_health_check {
interval = "10s"
max_failures = 42
}
}
}
]
}
}
}
}
services {
name = "echo-service"
port = 4444
connect = { sidecar_service {} }
Kind = "ingress-gateway"
Name = "ingress-service"
Listeners = [
{
Port = 8080
Protocol = "tcp"
Services = [
{
Name = "sock_forwarder"
}
]
}
]
consul agent -dev -enable-script-checks -config-dir=./consul.d
consul connect envoy -sidecar-for sock_forwarder.1
consul connect envoy -sidecar-for echo-service -admin-bind localhost:19001
consul config write ingress-gateway.hcl
consul connect envoy -gateway=ingress -register -service ingress-service -address '{{ GetInterfaceIP "eth0" }}:8888' -admin-bind localhost:19002
netcat 127.0.0.1 4444
netcat 127.0.0.1 8080
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* fixup Unix capitalization
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* Update website/content/docs/connect/registration/service-registration.mdx
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* Provide examples in hcl and json
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* Apply suggestions from code review
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* One more fixup for docs
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
This PR adds cluster members to the metrics API. The number of members per
segment are reported as well as the total number of members.
Tested by running a multi-node cluster locally and ensuring the numbers were
correct. Also added unit test coverage to add the new expected gauges to
existing test cases.
Adds more clear indicators that the collections on the learn.hashicorp.com sites have specific instructions for single node deployments.
Co-Authored by: soonoo <qpseh2m7@gmail.com>
* Update glossary.mdx
1. Update header to the first section to "Consul Vocabulary" since these are the terms used in the context of Consul conversations.
2. Kept the header "Consul Glossary" since these are the terms useful for practitioners in the consul space.
3. Removed interlinking to terms on the same page.
Co-authored-by: Hans Hasselberg <me@hans.io>
Co-authored-by: Swarna Podila <swarnap@users.noreply.github.com>
Update register check documentation clarify that Id returns as CheckId in the response
Co-Authored-By: Shaker Islam <shaqq@users.noreply.github.com>
Co-authored-by: Shaker Islam <shaqq@users.noreply.github.com>
Plug in callouts content
Most styles for the callout blade
Re-export Consul Stack image
Update icons
Misc content tweaks
Optimize mobile for the consul stack icon
Better alignment
Make full CTA box clickable
Per Consul PM, kubeconfig is not required for manual join. I believe this should be clarified in the docs as the current wording refers to the auto join steps above which state kubeconfig is required.
* website: bump to docs-page prerelease with hidden page support
* website: replace hidden pages hot fix for /docs, /api-docs, and /commmands
* website: remove unused files for old hidden pages hotfix
* website: bump to stable docs-page, w next-mdx-remote bump
* website: bump to latest markdown-page
* WIP reloadable raft config
* Pre-define new raft gauges
* Update go-metrics to change gauge reset behaviour
* Update raft to pull in new metric and reloadable config
* Add snapshot persistance timing and installSnapshot to our 'protected' list as they can be infrequent but are important
* Update telemetry docs
* Update config and telemetry docs
* Add note to oldestLogAge on when it is visible
* Add changelog entry
* Update website/content/docs/agent/options.mdx
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
Initially we were loading every potential upstream address into Envoy
and then routing traffic to the logical upstream service. The downside
of this behavior is that traffic meant to go to a specific instance
would be load balanced across ALL instances.
Traffic to specific instance IPs should be forwarded to the original
destination and if it's a destination in the mesh then we should ensure
the appropriate certificates are used.
This PR makes transparent proxying a Kubernetes-only feature for now
since support for other environments requires generating virtual IPs,
and Consul does not do that at the moment.
The only thing that needed fixing up pertained to this section of the 1.18.x release notes:
> grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default.
For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default.
Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.
* Update website/content/docs/discovery/services.mdx with address field behavior.
Co-authored-by: Jono Sosulska <42216911+jsosulska@users.noreply.github.com>
Co-authored-by: Jono Sosulska <42216911+jsosulska@users.noreply.github.com>
* Use proxy outbound port from TransparentProxyConfig if provided
* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
and additional user IDs to be excluded from traffic redirection.
This affects both the traffic-redirect command and the iptables SDK package.
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.
Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
* CLI: Add support for reading internal raft snapshots to snapshot inspect
* Add snapshot inspect test for raw state files
* Add changelog entry
* Update .changelog/10089.txt
* website: add back unlinked pages to match previous state
* website: add unlinked content check
* website: add hidden nav-data to unlinked content check
* add http2 ping checks
* fix test issue
* add h2ping check to config resources
* add new test and docs for h2ping
* fix grammatical inconsistency in H2PING documentation
* resolve rebase conflicts, add test for h2ping tls verification failure
* api documentation for h2ping
* update test config data with H2PING
* add H2PING to protocol buffers and update changelog
* fix typo in changelog entry
* Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled.
* Add new iptables package for applying traffic redirection rules with iptables.
* website: migrate to new nav-data format
* website: clean up unused intro content
* website: remove deprecated sidebar_title from frontmatter
* website: add react-content to fix global style import issue
The Agent token policy when using rexec should have `write` on "_rexec"
key prefix. Updated the exec command documentation to explicitly state
this requirement.
* Fixes #2379-Improve interval explanation in the telemetry doc
* Fixes #4734-Update consul memory metrics
* Fixes #4836-Removed node.deregistration as that isn't in state.go
* Fixes#8986 partially-Trim redundant language
* Fixes #9087-Adds helpful details to telemetry on autopilot
* Fixes #9274-Addresses NaN output in autopilot
Some TLS servers require SNI, but the Golang HTTP client doesn't
include it in the ClientHello when connecting to an IP address. This
change adds a new TLSServerName field to health check definitions to
optionally set it. This fixes#9473.
Allows setting -prometheus-backend-port to configure the cluster
envoy_prometheus_bind_addr points to.
Allows setting -prometheus-scrape-path to configure which path
envoy_prometheus_bind_addr exposes metrics on.
-prometheus-backend-port is used by the consul-k8s metrics merging feature, to
configure envoy_prometheus_bind_addr to point to the merged metrics
endpoint that combines Envoy and service metrics so that one set of
annotations on a Pod can scrape metrics from the service and it's Envoy
sidecar.
-prometheus-scrape-path is used to allow configurability of the path
where prometheus metrics are exposed on envoy_prometheus_bind_addr.
The Server certificates used for Federation require the node name in the form of `<node>.server.<dc>.<domain>`. Not having this would through `bad tls certificate` error.
* Fixed cert create command
* Added note to create a wildcard cert (like the ones on Kubernetes)
* Fixed numbering
nia/docs 0.1.0-beta
Co-authored-by: Kent 'picat' Gruber <kent@hashicorp.com>
Co-authored-by: Lorna Song <lorna@hashicorp.com>
Co-authored-by: John Eikenberry <jae@zhar.net>
Note that this does NOT upgrade to xDS v3. That will come in a future PR.
Additionally:
- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.
Fixes#8425
Currently when data is imported via `consul kv import` it overwrites
keys under the root key. Since `consul kv export` can retrieve data for
the given prefix, i.e. part of the KV tree, importing it under root may
be not what users want.
To mirror prefix behavior from export this PR adds prefix feature to the
import command that adds prefix to all keys that are imported.
* Document how users can migrate to CRDs.
* Update documentation for federation with new `ProxyDefaults`
requirement.
* Ensure `controller.enabled: true` is set in our example configs.
* Remove `connect-service-protocol` annotation docs.
* Add docs for MIME sniffing on metrics endpoint.
This was added in 1.7.2 last year but I realised we don't document it so it's kinda surprising Prometheus "just works" now.
* Update website/content/api-docs/agent/index.mdx
This allows setting ForceWithoutCrossSigning when reconfiguring the CA
for any provider, in order to forcibly move to a new root in cases where
the old provider isn't reachable or able to cross-sign for whatever
reason.
This PR is based on the previous work by @snuggie12 in PR #6825. It adds the command consul intention list to list all available intentions. The list functionality for intentions seems a bit overdue as it's just very handy. The web UI cannot list intentions outside of the default namespace, and using the API is sometimes not the friendliest option. ;)
I cherry picked snuggie12's commits who did most of the heavy lifting (thanks again @snuggie12 for your great work!). The changes in the original commit mostly still worked on the current HEAD. On top of that I added support for namespaces and fixed the docs as they are managed differently today. Also the requested changes related to the "Connect" references in the original PRs have been addressed.
Fixes#5652
Co-authored-by: Matt Hoey <mhoey05@jcu.edu>
CTS running with default configuration will communicate over http
unless the Consul client is configured with TLS. Having the example
set the scheme to https is misleading and will result in an error:
"http: server gave HTTP response to HTTPS client"
Hi everyone,
I'm developing an open source CLI tool for easily editing Consul KV values from your favorite text editor. I've added to Client Libraries & SDKs page on website.
Repository: https://github.com/sadedil/kvit
Kind regards,
Mustafa
* Adds a new react component `ConfigEntryReference` that allows us to
document config entries for both HCL and Kubernetes CRDs.
* Rewrites config entry docs to use new component
* Add CRD examples alongside HCL examples
* Removes examples from Kube CRD doc because the examples are now in the
main CRD docs
* update example for tag_name
In the example for azure the "tag_name" field is wrong cause say "tag_key" and not "tag_name"
* change tag_key by tag_name in the description of azure
* reorganize for clarity and update for value syntax
* fix quotes around value
* Apply suggestions from code review
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
* Apply suggestions from code review
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
* create consul version metric with version label
* agent/agent.go: add pre-release Version as well as label
Co-Authored-By: Radha13 <kumari.radha3@gmail.com>
* verion and pre-release version labels.
* hyphen/- breaks prometheus
* Add Prometheus gauge defintion for version metric
* Add new metric to telemetry docs
Co-authored-by: Radha Kumari <kumari.radha3@gmail.com>
Co-authored-by: Aestek <thib.gilles@gmail.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
- Fix anchors for client.extraEnvironmentVars and
server.extraEnvironmentVars.
- Change extraEnvironmentVars data type to `map`.
- Fix external link to kubernetes.io under
connectInject.namespaceSelector.
A vulnerability was identified in Consul and Consul Enterprise (“Consul”) such that operators with `operator:read` ACL permissions are able to read the Consul Connect CA configuration when explicitly configured with the `/v1/connect/ca/configuration` endpoint, including the private key. This allows the user to effectively privilege escalate by enabling the ability to mint certificates for any Consul Connect services. This would potentially allow them to masquerade (receive/send traffic) as any service in the mesh.
--
This PR increases the permissions required to read the Connect CA's private key when it was configured via the `/connect/ca/configuration` endpoint. They are now `operator:write`.
Header is: X-Consul-Default-ACL-Policy=<allow|deny>
This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
It's better to avoid the ambiguous Vault statement that was not clarified and drop the loaded "roles" term in favor of "capabilities" since the ACL system is described as capability-based in previous ACL documentation.
Consul's Connect CA documentation mentions future releases will
support a pluggable CA system. This sentence has existed in the docs
for over two years, however there are currently no plans to develop
this feature on the near-term roadmap.
This commit removes this sentence to avoid giving the impression that
this feature will be available in an upcoming release.
* Update to CRD docs
* Update website/pages/docs/k8s/crds.mdx
* Modify proxy default and service default protocols
Carry over from previous PR that I forgot to submit a review/suggestion to, TCP and HTTP are not valid protocols for Proxy Defaults and Service Defaults
kubectl apply -f sdefault.yml
Error from server: error when creating "sdefault.yml": admission webhook "mutate-servicedefaults.consul.hashicorp.com" denied the request: servicedefaults.consul.hashicorp.com "your-service-name" is invalid: spec.expose.paths[0].protocol: Invalid value: "tcp": must be one of "http", "http2"
kubectl apply -f sdefault.yml
Error from server: error when creating "sdefault.yml": admission webhook "mutate-servicedefaults.consul.hashicorp.com" denied the request: servicedefaults.consul.hashicorp.com "your-service-name" is invalid: spec.expose.paths[0].protocol: Invalid value: "tcp": must be one of "http", "http2"
Co-authored-by: David Yu <dyu@hashicorp.com>
* Add NIA Integration Program page
* Update name to Consul-Terraform-Sync and add Tech Preview tags
* Update diagram to include sequence numbers
* Remove Tech Preview tags and Update Images
* Add TF module naming convention, update image and links
* Add a note, update PANW link, and working updates
* Update URLs to local path
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.
This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
* Update the Azure cloud auto join documentation with more explicit information on how to configure the infrastructure.
* Add a note regarding the length of time taken for Azure to sync the MSI permissions.
* Update references from tag_name to tag_key in the Azure examples
Co-authored-by: Jono Sosulska <42216911+jsosulska@users.noreply.github.com>
Load balancing policies are configured in service-resolvers. All Envoy load balancing algorithms are currently supported. Including: least_request, round_robin, random, maglev, and ring_hash.
At the moment hash-based load balancing configuration is not applied at mesh gateways since they cannot decrypt traffic to inspect HTTP attributes like headers.
* k8s > ambassador integration moved to k8s > service mesh > ambassador integration
* k8s > get started > overview moved to k8s > get started > install with
helm chart
* k8s > helm chart reference renamed to helm chart configuration
- moved and renamed files/folders based on new structure
- updated docs navigation based on new structure
- moved CLI to top nav (created commands.jsx and commands-navigation.js)
- updated and added redirects
- updating to be consistent with standalone categories
- changing "overview" link in top nav to lead to where intro was moved (docs/intro)
- adding redirects for intro content
- deleting old intro folders
- format all data/navigation files
- deleting old commands folder
- reverting changes to glossary page
- adjust intro navigation for removal of 'vs' paths
- add helm page redirect
- fix more redirects
- add a missing redirect
- fix broken anchor links and formatting mistakes
- deleted duplicate section, added redirect, changed link
- removed duplicate glossary page
* Update k8s sync docs
- remove docs that said for nodeport service we register each instance
on a node with its same node name. We instead register each instance
onto the k8s-sync node
- add docs describing which ports and ips are used
* update bindata_assetfs.go
* Release v1.8.2
* Putting source back into Dev Mode
* changelog: add entries for 1.7.6, 1.7.5 and 1.6.7
Co-authored-by: hashicorp-ci <hashicorp-ci@users.noreply.github.com>
Related changes:
- hard-fail the xDS connection attempt if the envoy version is known to be too old to be supported
- remove the RouterMatchSafeRegex proxy feature since all supported envoy versions have it
- stop using --max-obj-name-len (due to: envoyproxy/envoy#11740)
This implements a solution for #7863
It does:
Add a new config cache.entry_fetch_rate to limit the number of calls/s for a given cache entry, default value = rate.Inf
Add cache.entry_fetch_max_burst size of rate limit (default value = 2)
The new configuration now supports the following syntax for instance to allow 1 query every 3s:
command line HCL: -hcl 'cache = { entry_fetch_rate = 0.333}'
in JSON
{
"cache": {
"entry_fetch_rate": 0.333
}
}
The "Overview" link previously went to a summary of Consul features and a
comparison to other products. This commit restores that destination at the
request of Consul PMs.
This commit previously removed the link (it went to `/` instead of `/intro`).
ac612a9cdc (diff-f98b55875118725d1373dd2da36d9ee5)
* docs: add section for /health/ingress/:service API
* Add documentation around consul version for API
* docs: add note about gateway-services API release version
* Fix typos on commandline flags, updated config opts
- Added anchors to https://github.com/hashicorp/consul/pull/8223
- Fix Typos
Updated to include config file options as well as CLI.
Fixes#7764
Until now these two fields could only be set through on-disk agent configuration.
This change adds the fields to the agent API struct definition so that they can
be set using the agent HTTP API.
Highlights:
- add new endpoint to query for intentions by exact match
- using this endpoint from the CLI instead of the dump+filter approach
- enforcing that OSS can only read/write intentions with a SourceNS or
DestinationNS field of "default".
- preexisting OSS intentions with now-invalid namespace fields will
delete those intentions on initial election or for wildcard namespaces
an attempt will be made to downgrade them to "default" unless one
exists.
- also allow the '-namespace' CLI arg on all of the intention subcommands
- update lots of docs
This example shows a TLS enabled ingress config on a non-https port.
Currently, that means we require the port to be specified in one of the
host entries to route traffic.
* Updates docs with ingress Host header changes
Clarify that a Host header is required for L7 protocols, and specify
that the default is to use the Consul DNS ingress subdomain
* Add sentence about using '*' by itself for testing
* Add optional step for using L7 routing config
* Note that port numbers may need to be added in the Hosts field
* Formatting spaces between keys in Config entries
* Service Router spacing
* Missing Camel Case proxy-defaults
* Remove extra spaces service-splitter
* Remove extra spsaces service-resolver
* More spaces a la hclfmt
* Nice!
* Oh joy!
* More spaces on proxy-defaults
* Update website/pages/docs/agent/config-entries/proxy-defaults.mdx
Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>
A Node Identity is very similar to a service identity. Its main targeted use is to allow creating tokens for use by Consul agents that will grant the necessary permissions for all the typical agent operations (node registration, coordinate updates, anti-entropy).
Half of this commit is for golden file based tests of the acl token and role cli output. Another big updates was to refactor many of the tests in agent/consul/acl_endpoint_test.go to use the same style of tests and the same helpers. Besides being less boiler plate in the tests it also uses a common way of starting a test server with ACLs that should operate without any warnings regarding deprecated non-uuid master tokens etc.
Get the tertiary links to wrap below buttons
Adjust color/spacing of tertiary via override
Remove overrides, implement custom link
Extract arrow icon to file
Increase top margin for third link
Apply Brandon's fixes
Co-authored-by: Brandon Romano <BrandonRRomano@gmail.com>
This allows the operator to disable agent caching for the http endpoint.
It is on by default for backwards compatibility and if disabled will
ignore the url parameter `cached`.
Found using staticcheck.
binary.Write does not accept int types without a size. The error from binary.Write was ignored, so we never saw this error. Casting the data to uint64 produces a correct hash.
Also deprecate the Default{Addr,Port} fields, and prevent them from being encoded. These fields will always be empty and are not used.
Removing these would break backwards compatibility, so they are left in place for now.
Co-authored-by: Hans Hasselberg <me@hans.io>
The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option.
If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS.
Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.
The doc says: "When the Connect injector is installed, the Connect sidecar is automatically added to all pods." But, it depends on the configuration, so I think it's better to say: "When the Connect injector is installed, the Connect sidecar can automatically added to all pods."
Based on work done in https://github.com/hashicorp/memberlist/pull/196
this allows to restrict the IP ranges that can join a given Serf cluster
and be a member of the cluster.
Restrictions on IPs can be done separatly using 2 new differents flags
and config options to restrict IPs for LAN and WAN Serf.
This removes "Intro", which is actually no longer the recommended
get-started place (Learn takes that spot).
Additionally, there is now a separator between the high level business
value stuff, and the low level developer stuff.
This PR contains documentation additions for ingress and terminating gateways. New pages for the config-entries and overall feature description were added, as well as various additions to related pages.
Co-authored-by: Jono Sosulska <42216911+jsosulska@users.noreply.github.com>
Co-authored-by: freddygv <gh@freddygv.xyz>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* docs: Link to compatibility matrix for imageEnvoy
Added a link to the Envoy supported version in the documentation for `imageEnvoy` parameter.
* Update website/source/docs/platform/k8s/helm.html.md
* Enable filtering language support for the v1/connect/intentions listing API
* Update website for filtering of Intentions
* Update website/source/api/connect/intentions.html.md
* Change style to match "join" singular
- Replaced "(Consul) cluster" with "Consul Datacenter"
- Removed "ing" so the feature fits "Consul Auto-join", and that the tense is correct.
Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
* [docs] Built-in Proxies not meant for production
* Adding link to Envoy for Connect
* Update website/source/docs/connect/proxies/built-in.md
Co-Authored-By: Blake Covarrubias <blake@covarrubi.as>
* Revising note
* Update website/source/docs/connect/proxies/built-in.md
period
Co-Authored-By: Hans Hasselberg <me@hans.io>
Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Hans Hasselberg <me@hans.io>
* Add ACL CLI commands output format option.
Add command level formatter, that incapsulates command output printing
logiс that depends on the command `-format` option.
Move Print* functions from acl_helpers to prettyFormatter. Add jsonFormatter.
* Return error code in case of formatting failure.
* Add acl commands -format option to doc.
This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch:
There are several distinct chunks of code that are affected:
* new flags and config options for the server
* retry join WAN is slightly different
* retry join code is shared to discover primary mesh gateways from secondary datacenters
* because retry join logic runs in the *agent* and the results of that
operation for primary mesh gateways are needed in the *server* there are
some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur
at multiple layers of abstraction just to pass the data down to the right
layer.
* new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers
* the function signature for RPC dialing picked up a new required field (the
node name of the destination)
* several new RPCs for manipulating a FederationState object:
`FederationState:{Apply,Get,List,ListMeshGateways}`
* 3 read-only internal APIs for debugging use to invoke those RPCs from curl
* raft and fsm changes to persist these FederationStates
* replication for FederationStates as they are canonically stored in the
Primary and replicated to the Secondaries.
* a special derivative of anti-entropy that runs in secondaries to snapshot
their local mesh gateway `CheckServiceNodes` and sync them into their upstream
FederationState in the primary (this works in conjunction with the
replication to distribute addresses for all mesh gateways in all DCs to all
other DCs)
* a "gateway locator" convenience object to make use of this data to choose
the addresses of gateways to use for any given RPC or gossip operation to a
remote DC. This gets data from the "retry join" logic in the agent and also
directly calls into the FSM.
* RPC (`:8300`) on the server sniffs the first byte of a new connection to
determine if it's actually doing native TLS. If so it checks the ALPN header
for protocol determination (just like how the existing system uses the
type-byte marker).
* 2 new kinds of protocols are exclusively decoded via this native TLS
mechanism: one for ferrying "packet" operations (udp-like) from the gossip
layer and one for "stream" operations (tcp-like). The packet operations
re-use sockets (using length-prefixing) to cut down on TLS re-negotiation
overhead.
* the server instances specially wrap the `memberlist.NetTransport` when running
with gateway federation enabled (in a `wanfed.Transport`). The general gist is
that if it tries to dial a node in the SAME datacenter (deduced by looking
at the suffix of the node name) there is no change. If dialing a DIFFERENT
datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh
gateways to eventually end up in a server's :8300 port.
* a new flag when launching a mesh gateway via `consul connect envoy` to
indicate that the servers are to be exposed. This sets a special service
meta when registering the gateway into the catalog.
* `proxycfg/xds` notice this metadata blob to activate additional watches for
the FederationState objects as well as the location of all of the consul
servers in that datacenter.
* `xds:` if the extra metadata is in place additional clusters are defined in a
DC to bulk sink all traffic to another DC's gateways. For the current
datacenter we listen on a wildcard name (`server.<dc>.consul`) that load
balances all servers as well as one mini-cluster per node
(`<node>.server.<dc>.consul`)
* the `consul tls cert create` command got a new flag (`-node`) to help create
an additional SAN in certs that can be used with this flavor of federation.
The Session API in Consul 1.7.0 and 1.7.1 is incompatible with prior versions of Consul.
This PR adds a note to our version-specific upgrade guide to guard against users upgrading before the fix in 1.7.2 is released.
The go-discover library supports Linode. This adds support for
discovering other Consul agents running on Linode. Consul has supported
this since [66b8c20][1] was merged, so this commit just updates the
documentation to match current features.
[1]: 66b8c20990
* Remove trailing whitespace in DNS forwarding guide.
* Add example for enabling reverse lookup of IP addrseses to .consul domain on systemd-resolved platforms
The option `username` does not work. Need to use user_name with underscore
> Authentication failed: Exactly one of Username and UserID must be provided for password authentication
The option `user_name` works, however, it's need to use `region`, `domain_name` in additional.
* add 1.12.2
* add envoy 1.13.0
* Introduce -envoy-version to get 1.10.0 passing.
* update old version and fix consul-exec case
* add envoy_version and fix check
* Update Envoy CLI tests to account for the 1.13 compatibility changes.
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
* agent: measure blocking queries
* agent.rpc: update docs to mention we only record blocking queries
* agent.rpc: make go fmt happy
* agent.rpc: fix non-atomic read and decrement with bitwise xor of uint64 0
* agent.rpc: clarify review question
* agent.rpc: today I learned that one must declare all variables before interacting with goto labels
* Update agent/consul/server.go
agent.rpc: more precise comment on `Server.queriesBlocking`
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/agent/telemetry.html.md
agent.rpc: improve queries_blocking description
Co-Authored-By: Paul Banks <banks@banksco.de>
* agent.rpc: fix some bugs found in review
* add a note about the updated counter behavior to telemetry.md
* docs: add upgrade-specific note on consul.rpc.quer{y,ies_blocking} behavior
Co-authored-by: Paul Banks <banks@banksco.de>
* Expose Envoy /stats for statsd agents; Add testcases
* Remove merge conflict leftover
* Add support for prefix instead of path; Fix docstring to mirror these changes
* Add new config field to docs; Add testcases to check that /stats/prometheus is exposed as well
* Parametrize matchType (prefix or path) and value
* Update website/source/docs/connect/proxies/envoy.md
Co-Authored-By: Paul Banks <banks@banksco.de>
Co-authored-by: Paul Banks <banks@banksco.de>
Update the description for the Helm chart's connectInject.imageEnvoy
parameter to reflect the correct organization name for images published by
EnvoyProxy.io.
* Updates to the Txn API for namespaces
* Update agent/consul/txn_endpoint.go
Co-Authored-By: R.B. Boyer <rb@hashicorp.com>
Co-authored-by: R.B. Boyer <public@richardboyer.net>
This commit changes the health check example shown for the
success/failures_before_passing option to correctly show that the value
of `checks` is an array of objects, not an object.
Added text clarifying these check parameters are available in Consul
1.7.0 and later.
Expanded the health check to provide a more complete configuration
example.
Resolves#7114.
The backing RPC already existed but the endpoint will be useful for other service syncing processes such as consul-k8s as this endpoint can return all services registered with a node regardless of namespacing.
Removing automatic connection wording for applications for the time being. From @blake
> They can automatically establish TLS connections without being aware that TLS is happening. They are aware that they’re routed through the Connect proxy, the app has to configure itself to use the local upstream port.
Currently when using the built-in CA provider for Connect, root certificates are valid for 10 years, however secondary DCs get intermediates that are valid for only 1 year. There is no mechanism currently short of rotating the root in the primary that will cause the secondary DCs to renew their intermediates.
This PR adds a check that renews the cert if it is half way through its validity period.
In order to be able to test these changes, a new configuration option was added: IntermediateCertTTL which is set extremely low in the tests.
* Add CreateCSRWithSAN
* Use CreateCSRWithSAN in auto_encrypt and cache
* Copy DNSNames and IPAddresses to cert
* Verify auto_encrypt.sign returns cert with SAN
* provide configuration options for auto_encrypt dnssan and ipsan
* rename CreateCSRWithSAN to CreateCSR
* docs/connect add link to intentions and minor phrasing change
* docs/connect pluralize 'applications'
* Update website/source/docs/connect/connect-internals.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Renamed structs.IntentionWildcard to structs.WildcardSpecifier
* Refactor ACL Config
Get rid of remnants of enterprise only renaming.
Add a WildcardName field for specifying what string should be used to indicate a wildcard.
* Add wildcard support in the ACL package
For read operations they can call anyAllowed to determine if any read access to the given resource would be granted.
For write operations they can call allAllowed to ensure that write access is granted to everything.
* Make v1/agent/connect/authorize namespace aware
* Update intention ACL enforcement
This also changes how intention:read is granted. Before the Intention.List RPC would allow viewing an intention if the token had intention:read on the destination. However Intention.Match allowed viewing if access was allowed for either the source or dest side. Now Intention.List and Intention.Get fall in line with Intention.Matches previous behavior.
Due to this being done a few different places ACL enforcement for a singular intention is now done with the CanRead and CanWrite methods on the intention itself.
* Refactor Intention.Apply to make things easier to follow.
* added disclaimer about network segments due to Serf limitations
using work made at https://github.com/hashicorp/consul/pull/6558 by @thepomeranian
* Lowercasing functionality name
* Update website/source/docs/enterprise/network-segments/index.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
Co-authored-by: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Azure MSI documentation
Adding in note about support for Azure MSI authentication method for Cloud auto-join
* fixing text formatting
fixing text formatting
* missing word
missing word - variable
* Update website/source/docs/agent/cloud-auto-join.html.md
Language change to be specific about where the security risk mitigation is concerned
Co-Authored-By: Jack Pearkes <jackpearkes@gmail.com>
Co-authored-by: Jack Pearkes <jackpearkes@gmail.com>
The global.bootstrapACLs key in the Helm chart docs was inadvertently
moved to a top-level key in commit 12e6ef8, which is incorrect.
This commit reverts that error.
* Handle discard all logfiles properly
Fixes https://github.com/hashicorp/consul/issues/6892.
The [docs](https://www.consul.io/docs/agent/options.html#_log_rotate_max_files) are stating:
> -log-rotate-max-files - to specify the maximum number of older log
> file archives to keep. Defaults to 0 (no files are ever deleted). Set to
> -1 to disable rotation and discard all log files.
But the `-1` case was not implemented and led to a panic when being
used.
Co-Authored-By: Freddy <freddygv@users.noreply.github.com>
- website: embed yt videos on intro pages
- for /docs/connect
- for /intro
- css to handle iframe responding at smaller viewports
- Update consul connect video with introductory description. Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
- Update consul connect intro with introductory description. Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
Update -retry-join documentation to explicitly state the option can be
specified multiple times. Add corresponding config example showing
multiple join addresses.
* relax requirements for auto_encrypt on server
* better error message when auto_encrypt and verify_incoming on
* docs: explain verify_incoming on Consul clients.
Also update the Docs and fixup the HTTP API to return proper errors when someone attempts to use Namespaces with an OSS agent.
Add Namespace HTTP API docs
Make all API endpoints disallow unknown fields
The listener ports specified in the headings for the HTTP and HTTP2
examples do not match the ports in the corresponding service
registration configurations.
This commit changes the port specified in the heading for the HTTP
listener to match the port used in the service registration example.
In addition, the listener_port specified for the HTTP2 listener is
modified to match the port number specified in the heading.
- Remove duplicate install instructions from the Helm Chart page and
kept them in Running Consul
- Renamed Helm Chart to Helm Chart Reference because that's mostly what
it contains (along with some examples)
- Renamed Running Consul to Installing Consul
- Changed instructions to be for installing using Helm 3 and added
notes if using Helm 2
- Used release name "hashicorp" so subsequent instructions can be more
concise and pastable, e.g. "port forward to svc/hashicorp-consul-server" vs. "port
forward to svc/<your release name>-consul-server"
- Use config.yaml as the name for the override values file since it
differentiates from the default values.yaml file and its the name of the
file used in the helm docs
(https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing)
* Adds 'limits' field to the upstream configuration of a connect proxy
This allows a user to configure the envoy connect proxy with
'max_connections', 'max_queued_requests', and 'max_concurrent_requests'. These
values are defined in the local proxy on a per-service instance basis
and should thus NOT be thought of as a global-level or even service-level value.
* Update AWS SDK to use PCA features.
* Add AWS PCA provider
* Add plumbing for config, config validation tests, add test for inheriting existing CA resources created by user
* Unparallel the tests so we don't exhaust PCA limits
* Merge updates
* More aggressive polling; rate limit pass through on sign; Timeout on Sign and CA create
* Add AWS PCA docs
* Fix Vault doc typo too
* Doc typo
* Apply suggestions from code review
Co-Authored-By: R.B. Boyer <rb@hashicorp.com>
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Doc fixes; tests for erroring if State is modified via API
* More review cleanup
* Uncomment tests!
* Minor suggested clean ups
- Remove incorrect statement that `LockDelay` must be greater than 0
- Add sentence to the top of the page pointing to the internal document
describing the sessions mechanism for more context
* Support Connect CAs that can't cross sign
* revert spurios mod changes from make tools
* Add log warning when forcing CA rotation
* Fixup SupportsCrossSigning to report errors and work with Plugin interface (fixes tests)
* Fix failing snake_case test
* Remove misleading comment
* Revert "Remove misleading comment"
This reverts commit bc4db9cabed8ad5d0e39b30e1fe79196d248349c.
* Remove misleading comment
* Regen proto files messed up by rebase
* updating the landing page with jtbd
* changed the buttons to pink
* updating CSS based on John's help
* updating a use case
* updating the language and rearranging the guides
* adding icons
* fixed image width
* fixing buttons and updating traffic splitting language.
Fix spelling errors, API doc inconsistencies, and formatting issues.
* Fix several spelling errors.
* Prepend / to v1/event/list path in Watches.
* Rename script handlers to match Watch type.
* Remove /v1 path prefix on service health API endpoints.
Makes request path consistent with the rest of the HTTP API
documentation which does not include the /v1 prefix.
* Fix bracket formatting issue on Telemetry page.
The HTML codes used for brackets inside of the code block are not
interpolated, and are shown as literal strings.
Replace the numeric HTML codes with the intended character value to
fix display formatting.
Also placed variable reference on agent/options.html inside code block
for consistency with the presentation of other options on the page.
* Add missing word to Coordinate.Node docstring.
Resolves#6014
* Allow RSA CA certs for consul and vault providers to correctly sign EC leaf certs.
* Ensure key type ad bits are populated from CA cert and clean up tests
* Add integration test and fix error when initializing secondary CA with RSA key.
* Add more tests, fix review feedback
* Update docs with key type config and output
* Apply suggestions from code review
Co-Authored-By: R.B. Boyer <rb@hashicorp.com>
Updating all .io Community sites to direct practitioners to the Forum as the first medium for communicating with other users and HashiCorp employees. Deleted Gitter link and Google Group link, as these will be phased out over the next few months. Updated what appeared to be a typo on the page description. Chatted with Nic Jackson before submitting PR.
* Changed Guides to Learn in the top nav and added utm parameters to the guide index page
* Update website/source/docs/guides/index.html.md
* Update website/source/docs/guides/index.html.md
* Update website/source/layouts/layout.erb
A check may be set to become passing/critical only if a specified number of successive
checks return passing/critical in a row. Status will stay identical as before until
the threshold is reached.
This feature is available for HTTP, TCP, gRPC, Docker & Monitor checks.
Add text listing Consul's L7 features (via Envoy). Re-organize text to
flow similarly to Istio section.
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
Fixes#2742
Previously the docs didn't clarify that if a server restarts as a client then force-leave won't lead to removing the node from the raft config. This is because the node, which is alive after a restart, will refute messages about it having left . These messages about members leaving are in turn what trigger Consul's leader to remove a server from raft.
Fixes: #5396
This PR adds a proxy configuration stanza called expose. These flags register
listeners in Connect sidecar proxies to allow requests to specific HTTP paths from outside of the node. This allows services to protect themselves by only
listening on the loopback interface, while still accepting traffic from non
Connect-enabled services.
Under expose there is a boolean checks flag that would automatically expose all
registered HTTP and gRPC check paths.
This stanza also accepts a paths list to expose individual paths. The primary
use case for this functionality would be to expose paths for third parties like
Prometheus or the kubelet.
Listeners for requests to exposed paths are be configured dynamically at run
time. Any time a proxy, or check can be registered, a listener can also be
created.
In this initial implementation requests to these paths are not
authenticated/encrypted.
The fields in the certs are meant to hold the original binary
representation of this data, not some ascii-encoded version.
The only time we should be colon-hex-encoding fields is for display
purposes or marshaling through non-TLS mediums (like RPC).
- fix instructions for CoreDNS (it updated)
- fix instructions for new component names
- recommend installing with the name 'consul'
- add disclaimer that catalog sync is not always required
- clean up example values.yaml files
* website: Update middleman-hashicorp container and Gemfile.lock
Time marches on, and so do security vulnerabilities in Nokogiri. So it's time
for a new container.
As with last time, here's a reminder for the next person who needs to update
this:
- You shouldn't just update the dependency in Gemfile.lock, because your build
times will go to heck as you compile Nokogiri from source on every run. So you
need an updated container with all the dependencies.
- To update the container, you need to push a new tag to the middleman-hashicorp
repo. Teamcity does the rest, and will ship a new container to Docker Hub
(unless its credentials are out of date, in which case go ask team-eng-serv.)
- Once that's pushed:
- Update Makefile
- Update the Gemfile
- Delete Gemfile.lock
- `make website` until it comes up, then ctrl-C
- Commit the changes
* website: Specify a different json version in Gemfile.lock
The Consul website uses different containers for preview and deploy, and this
oddball JSON version was causing issues. This commit sacrifices a little bit
of preview startup speed for (hopefully) working deploys.
- Bootstrap escape hatches are OK.
- Public listener/cluster escape hatches are OK.
- Upstream listener/cluster escape hatches are not supported.
If an unsupported escape hatch is configured and the discovery chain is
activated log a warning and act like it was not configured.
Fixes#6160
* website: update the vs. envoy and proxies page
This is the second result on Google for "consul envoy" and
it seemed like it needed a bit of an upgrade to help clarify the
current state.
* Update website/source/intro/vs/proxies.html.md
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
* Update website/source/intro/vs/proxies.html.md
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
* Update website/source/intro/vs/proxies.html.md
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
* Update website/source/intro/vs/proxies.html.md
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
* Apply suggestions from code review
Co-Authored-By: Judith Malnick <judith.patudith@gmail.com>
Add parameter local-only to operator keyring list requests to force queries to only hit local servers (no WAN traffic).
HTTP API: GET /operator/keyring?local-only=true
CLI: consul keyring -list --local-only
Sending the local-only flag with any non-GET/list request will result in an error.
The main change is that we no longer filter service instances by health,
preferring instead to render all results down into EDS endpoints in
envoy and merely label the endpoints as HEALTHY or UNHEALTHY.
When OnlyPassing is set to true we will force consul checks in a
'warning' state to render as UNHEALTHY in envoy.
Fixes#6171
* Update go-bexpr to v0.1.1
This brings in:
• `in`/`not in` operators to do substring matching
• `matches` / `not matches` operators to perform regex string matching.
* Add the capability to auto-generate the filtering selector ops tables for our docs
This fixes pathological cases where the write throughput and snapshot size are both so large that more than 10k log entries are written in the time it takes to restore the snapshot from disk. In this case followers that restart can never catch up with leader replication again and enter a loop of constantly downloading a full snapshot and restoring it only to find that snapshot is already out of date and the leader has truncated its logs so a new snapshot is sent etc.
In general if you need to adjust this, you are probably abusing Consul for purposes outside its design envelope and should reconsider your usage to reduce data size and/or write volume.
* Bad link in encryption docs
* clarifying the guide link
* Update website/source/docs/agent/encryption.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/agent/encryption.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* add connect gateway guide
* Remove stray space
Co-Authored-By: Freddy <freddygv@users.noreply.github.com>
* Specify stanza and exact options
Co-Authored-By: Freddy <freddygv@users.noreply.github.com>
* incorporate comments from freddy
* integrate feedback from matt
* make snippets all json
* incorporate more comments from matt
* added links
* incorporate comments from neena on google doc draft
* make learn lnks relative
* clarify that gateways are new
* change socat to netcat
* add more description about replication token permissions
* Apply suggestions from code review
Co-Authored-By: Matt Keeler <mkeeler@users.noreply.github.com>
* add the prerequisite to enable centralized service config
* finish adding docs links
* website: link to 1.6.0 beta in downloads page
* website: reorganize intention replication/ca federation
* website: remove announcement bar
* Update website/source/docs/connect/connect-internals.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* website: update homepage and service mesh page
Aligning messaging to current product.
* website: fix link TODOs
* Add Mesh Gateway to mesh page, update use case wording
* connect: allow overriding envoy listener bind_address
* Update agent/xds/config.go
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
* connect: allow overriding envoy listener bind_port
* envoy: support unix sockets for grpc in bootstrap
Add AgentSocket BootstrapTplArgs which if set overrides the AgentAddress
and AgentPort to generate a bootstrap which points Envoy to a unix
socket file instead of an ip:port.
* Add a test for passing the consul addr as a unix socket
* Fix config formatting for envoy bootstrap tests
* Fix listeners test cases for bind addr/port
* Update website/source/docs/connect/proxies/envoy.md
* Add ui-content-path flag
* tests complete, regex validator on string, index.html updated
* cleaning up debugging stuff
* ui: Enable ember environment configuration to be set via the go binary at runtime (#5934)
* ui: Only inject {{.ContentPath}} if we are makeing a prod build...
...otherwise we just use the current rootURL
This gets injected into a <base /> node which solves the assets path
problem but not the ember problem
* ui: Pull out the <base href=""> value and inject it into ember env
See previous commit:
The <base href=""> value is 'sometimes' injected from go at index
serve time. We pass this value down to ember by overwriting the ember
config that is injected via a <meta> tag. This has to be done before
ember bootup.
Sometimes (during testing and development, basically not production)
this is injected with the already existing value, in which case this
essentially changes nothing.
The code here is slightly abstracted away from our specific usage to
make it easier for anyone else to use, and also make sure we can cope
with using this same method to pass variables down from the CLI through
to ember in the future.
* ui: We can't use <base /> move everything to javascript (#5941)
Unfortuantely we can't seem to be able to use <base> and rootURL
together as URL paths will get doubled up (`ui/ui/`).
This moves all the things that we need to interpolate with .ContentPath
to the `startup` javascript so we can conditionally print out
`{{.ContentPath}}` in lots of places (now we can't use base)
* fixed when we serve index.html
* ui: For writing a ContentPath, we also need to cope with testing... (#5945)
...and potentially more environments
Testing has more additional things in a separate index.html in `tests/`
This make the entire thing a little saner and uses just javascriopt
template literals instead of a pseudo handbrake synatx for our
templating of these files.
Intead of just templating the entire file this way, we still only
template `{{content-for 'head'}}` and `{{content-for 'body'}}`
in this way to ensure we support other plugins/addons
* build: Loosen up the regex for retrieving the CONSUL_VERSION (#5946)
* build: Loosen up the regex for retrieving the CONSUL_VERSION
1. Previously the `sed` replacement was searching for the CONSUL_VERSION
comment at the start of a line, it no longer does this to allow for
indentation.
2. Both `grep` and `sed` where looking for the omment at the end of the
line. We've removed this restriction here. We don't need to remove it
right now, but if we ever put the comment followed by something here the
searching would break.
3. Added `xargs` for trimming the resulting version string. We aren't
using this already in the rest of the scripts, but we are pretty sure
this is available on most systems.
* ui: Fix erroneous variable, and also force an ember cache clean on build
1. We referenced a variable incorrectly here, this fixes that.
2. We also made sure that every `make` target clears ember's `tmp` cache
to ensure that its not using any caches that have since been edited
everytime we call a `make` target.
* added docs, fixed encoding
* fixed go fmt
* Update agent/config/config.go
Co-Authored-By: R.B. Boyer <public@richardboyer.net>
* Completed Suggestions
* run gofmt on http.go
* fix testsanitize
* fix fullconfig/hcl by setting correct 'want'
* ran gofmt on agent/config/runtime_test.go
* Update website/source/docs/agent/options.html.md
Co-Authored-By: Hans Hasselberg <me@hans.io>
* Update website/source/docs/agent/options.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* remove contentpath from redirectFS struct
* Support for maximum size for Output of checks
This PR allows users to limit the size of output produced by checks at the agent
and check level.
When set at the agent level, it will limit the output for all checks monitored
by the agent.
When set at the check level, it can override the agent max for a specific check but
only if it is lower than the agent max.
Default value is 4k, and input must be at least 1.
* Moved the glossary to a new page and removed the advanced warnings from all internals docs.
* Update website/source/layouts/docs.erb
Co-Authored-By: Judith Malnick <judith@hashicorp.com>
* Updates based on PR feedback
* Update website/source/docs/internals/index.html.md
* Update website/source/docs/internals/index.html.md
* Update website/source/docs/internals/index.html.md
* Update website/source/docs/internals/index.html.md
* Update website/source/docs/internals/index.html.md
* Upating the term node to be more clear
* Update website/source/docs/internals/architecture.html.md
* Update website/source/docs/internals/architecture.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Addressing the failure detection comment
This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.
* clarify possibilities for centralized proxy configuration
* add line breaks to config entries file
* add info about centralized config to built in proxy doc
* mondify connect landing page to help with navigation
* move internals details to its own page
* link fixes and shortening text on main page
* put built-in proxy options on its own page
* add configuration details for connect
* clarify security title and add observability page
* reorganize menu
* remove observability from configuration section
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* rename connect section to include service mesh
* reorganize sections per suggestions from paul
* add configuration edits from paul
* add internals edits from paul
* add observability edits from paul
* reorganize pages and menu
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* menu corrections and edits
* incorporate some of pauls comments
* incorporate more of pauls comments
* Update website/source/docs/connect/configuration.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/index.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* incorporate kaitlin and pavanni feedback
* add redirect
* fix conflicts in index file
* Resolve conflicts in index file
* correct links for new organization
* Update website/source/docs/connect/proxies.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* Update website/source/docs/connect/registration.html.md
Co-Authored-By: Paul Banks <banks@banksco.de>
* add title to service registration page
* Upgrade xDS (go-control-plane) API to support Envoy 1.10.
This includes backwards compatibility shim to work around the ext_authz package rename in 1.10.
It also adds integration test support in CI for 1.10.0.
* Fix go vet complaints
* go mod vendor
* Update Envoy version info in docs
* Update website/source/docs/connect/proxies/envoy.md
The policy in the time-of-day Sentinel example incorrectly references
the top-level time.hour constant. This is actually the same as the
time.Hour Go value, so in other words, 3600000000000 (the int64 value
representing the time in nanoseconds).
This is corrected by just using time.now.hour instead.
* bump middleman-hashicorp to 0.3.40 and exclude guide rendering
* add notes to Makefile for volume mounts hack PR#5847
* make note of the PR number in the comment
Reference: https://github.com/hashicorp/consul/issues/4090
Examples covering a variety of potential use cases. Verified via `sockaddr eval` and `console agent -bind` on a test machine:
```console
# Baseline
$ sockaddr eval 'GetAllInterfaces'
[127.0.0.1/8 {1 65536 lo up|loopback} ::1 {1 65536 lo up|loopback} 10.0.0.10/8 {2 1500 eth0 b8:27:eb:7b:36:95 up|broadcast|multicast} fe80::12dc:5e4d:8ff8:2d96/64 {2 1500 eth0 b8:27:eb:7b:36:95 up|broadcast|multicast} 192.168.1.10/24 {3 1500 wlan0 b8:27:eb:2e:63:c0 up|broadcast|multicast} fe80::b6dc:5758:c306:b15b/64 {3 1500 wlan0 b8:27:eb:2e:63:c0 up|broadcast|multicast}]
# Using address within a specific CIDR
$ sockaddr eval 'GetPrivateInterfaces | include "network" "10.0.0.0/8" | attr "address"'
10.0.0.10
# Using a static network interface name
$ sockaddr eval 'GetInterfaceIP "eth0"'
10.0.0.10
# Using regular expression matching for network interface name that is forwardable and up
$ sockaddr eval 'GetAllInterfaces | include "name" "^eth" | include "flags" "forwardable|up" | attr "address"'
10.0.0.10
```
https://github.com/hashicorp/consul/issues/2121https://www.freedesktop.org/software/systemd/man/systemd.service.html
When set to notify, systemd will not attempt to start any dependent
services until after consul sends the notify signal. This is useful
in cases where there services that rely on the local consul agent
to be up and functional before they can start. The default is simple,
which will immediately mark the service as up and functioning even
if consul has not yet joined the cluster and has started listening
for connnections.
* Updating docs landing page since all the guides have moved
* fixing line wrap
* Giving the docs landing page content with CTAs
* Updated the language for the config and learn boxes
* fixing spacing
* WIP
* Document all the new Envoy L7 configs
* Apply suggestions from code review
Co-Authored-By: banks <banks@banksco.de>
* Rewrite dynamic config and add in TODO links
* Add some config entry docs
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Get rid of double negative
* Some incremental updates
* Update the config list docs to not point to service-default related things.
* A few more doc updates to get rid of some service-defaults specific linking info in the cli docs
* In progress update
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Reword bootstrap section
* Update example proxy-defaults config
* Finish up the examples section for managing config entries with the CLI
* Update website/source/docs/agent/config_entries.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* Use $ for shell command start
* Make it very clear that the normal way to manage things is via the API/CLI
* Update website/source/docs/agent/options.html.md
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* New guide listing common error messages and what to do about them
* Pasted in updates from StackPad edit
* Update website/source/docs/guides/common-errors.md
Co-Authored-By: tradel <todd@radel.us>
* Update website/source/docs/guides/common-errors.md
Co-Authored-By: tradel <todd@radel.us>
* Update website/source/docs/guides/common-errors.md
Co-Authored-By: tradel <todd@radel.us>
* add subheadings in config file errors, move doc out of guides
* modify guide index to point to learn
* adding to the redirects is a wip, updated the side navigation.
* add tracks to list
* Fixing redirects
* Update website/source/docs/guides/index.html.md
Co-Authored-By: judithpatudith <judith@hashicorp.com>
* Update website/source/docs/guides/index.html.md
Co-Authored-By: judithpatudith <judith@hashicorp.com>
* Update website/source/docs/guides/index.html.md
Co-Authored-By: judithpatudith <judith@hashicorp.com>
* Update website/source/docs/guides/index.html.md
Co-Authored-By: judithpatudith <judith@hashicorp.com>
* finish out track listings
* finishing redirects.
* change advanced language to production, add relationship of guides and docs
* Update website/source/redirects.txt
Co-Authored-By: judithpatudith <judith@hashicorp.com>
* Add api docs for the config entry endpoints
* Add enable_central_service_config field to agent docs
* Add docs for config entry CLI operations
* Fix wording and links in config entry docs
* Add links to the central service config option
* Update the central service config setting description.
Also update some snapshot agent docs
* Enforce correct permissions when registering a check
Previously we had attempted to enforce service:write for a check associated with a service instead of node:write on the agent but due to how we decoded the health check from the request it would never do it properly. This commit fixes that.
* Update website/source/docs/commands/snapshot/agent.html.markdown.erb
Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>
* starting broken link fixes
* Updating the other links for ACLs
* Updating the rest of the links
* fixing acl required links.
* update a bunch of other links
* updated a couple more broken links based on Alvins checker
* removed the extra s
Fixes: #4222
# Data Filtering
This PR will implement filtering for the following endpoints:
## Supported HTTP Endpoints
- `/agent/checks`
- `/agent/services`
- `/catalog/nodes`
- `/catalog/service/:service`
- `/catalog/connect/:service`
- `/catalog/node/:node`
- `/health/node/:node`
- `/health/checks/:service`
- `/health/service/:service`
- `/health/connect/:service`
- `/health/state/:state`
- `/internal/ui/nodes`
- `/internal/ui/services`
More can be added going forward and any endpoint which is used to list some data is a good candidate.
## Usage
When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of:
```
<selector> == <value>
<selector> != <value>
<value> in <selector>
<value> not in <selector>
<selector> contains <value>
<selector> not contains <value>
<selector> is empty
<selector> is not empty
not <other expression>
<expression 1> and <expression 2>
<expression 1> or <expression 2>
```
Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library
## Other changes
Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.
* [docs] security warning about Helm chart
Correct security warning about helm chart to caution about default Consul install.
* clarify that k8s or consul should be secure
* website: specify value of acquire/release params for kv
* website: clarify leader election usage in TTL docs
* website: document minimal value of lockdelay
I believe it uses the default when parsing 0 as it
views that as an empty parameter in this case.
- the space after `=` is significant in sc parameters (binPath= "<path> <args>, start= auto)
- given the text "the service automatically starts up during/after boot", added `start= auto` to
the example, otherwise the service will be set to Manual start mode.
* Added a note to GCE auto-join credentials option
Simply added a note to remind users that putting a json file in the config-dir will make consul parse it as a config file.
Hope to help someone else avoid wasting a day because of these errors:
==> Error parsing /etc/consul.d/credentials.json: 10 error(s) occurred: * invalid config key private_key
* Updated according to style guidelines
Co-Authored-By: delamart <erik@delamarter.ch>
* Docs: Remove default_policy From Code Example
It is not needed according to:
https://www.consul.io/docs/agent/acl-system.html#configuring-acls
* Docs: Cleanup Commands And Their Output On ACL Guide Page
Remove extra spaces and newlines
Ensure rules match input rules
* Docs: Remove Incomplete "Added In Version" Statement
Version added is specified on parent option
* Docs: Fix Broken Links
* Docs: Minor Sentence Tweaks
This PR introduces reloading tls configuration. Consul will now be able to reload the TLS configuration which previously required a restart. It is not yet possible to turn TLS ON or OFF with these changes. Only when TLS is already turned on, the configuration can be reloaded. Most importantly the certificates and CAs.
* Fix race condition in DNS when using cache
The healty node filtering was modifying the result from the cache, which
caused a crash when multiple queries were made to the same service
simultaneously.
We now copy the node slice before filtering to ensure we do not modify
the data stored in the cache.
* Fix wording in dns cache config doc
s/dns_max_age/cache_max_age/
This PR adds two features which will be useful for operators when ACLs are in use.
1. Tokens set in configuration files are now reloadable.
2. If `acl.enable_token_persistence` is set to `true` in the configuration, tokens set via the `v1/agent/token` endpoint are now persisted to disk and loaded when the agent starts (or during configuration reload)
Note that token persistence is opt-in so our users who do not want tokens on the local disk will see no change.
Some other secondary changes:
* Refactored a bunch of places where the replication token is retrieved from the token store. This token isn't just for replicating ACLs and now it is named accordingly.
* Allowed better paths in the `v1/agent/token/` API. Instead of paths like: `v1/agent/token/acl_replication_token` the path can now be just `v1/agent/token/replication`. The old paths remain to be valid.
* Added a couple new API functions to set tokens via the new paths. Deprecated the old ones and pointed to the new names. The names are also generally better and don't imply that what you are setting is for ACLs but rather are setting ACL tokens. There is a minor semantic difference there especially for the replication token as again, its no longer used only for ACL token/policy replication. The new functions will detect 404s and fallback to using the older token paths when talking to pre-1.4.3 agents.
* Docs updated to reflect the API additions and to show using the new endpoints.
* Updated the ACL CLI set-agent-tokens command to use the non-deprecated APIs.
Adds two new configuration parameters "dns_config.use_cache" and
"dns_config.cache_max_age" controlling how DNS requests use the agent
cache when querying servers.
* Add common blocking implementation details to docs
These come up over and over again with blocking query loops in our own code and third-party's. #5333 is possibly a case (unconfirmed) where "badly behaved" blocking clients cause issues, however since we've never explicitly documented these things it's not reasonable for third-party clients to have guessed that they are needed!
This hopefully gives us something to point to for the future.
It's a little wordy - happy to consider breaking some of the blocking stuff out of this page if we think it's appropriate but just wanted to quickly plaster over this gap in our docs for now.
* Update index.html.md
* Apply suggestions from code review
Co-Authored-By: banks <banks@banksco.de>
* Update index.html.md
* Update index.html.md
* Clearified monotonically
* Fixing formating
* Updating the Helm chart to include ACL parameter and examples.
* Updates based on feedback.
* Update website/source/docs/platform/k8s/helm.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Confirm RA against Consul 1.3
Change product_version frontmatter to ea_version and increase to 1.3
* Confirm DG against Consul 1.3
Change product_version frontmatter to ea_version and increase to 1.3
The guide currently uses node, service, and service for the UI Policy.
This will cause a practically useless UI. This patch uses the _prefix
variants instead which will have the intended behavior.
It appears that the `read` command for ACL policies was used to template the `read` command for ACL tokens, and an invalid option was not dropped from the docs.
Given a query like:
```
{
"Name": "tagged-connect-query",
"Service": {
"Service": "foo",
"Tags": ["tag"],
"Connect": true
}
}
```
And a Consul configuration like:
```
{
"services": [
"name": "foo",
"port": 8080,
"connect": { "sidecar_service": {} },
"tags": ["tag"]
]
}
```
If you executed the query it would always turn up with 0 results. This was because the sidecar service was being created without any tags. You could instead make your config look like:
```
{
"services": [
"name": "foo",
"port": 8080,
"connect": { "sidecar_service": {
"tags": ["tag"]
} },
"tags": ["tag"]
]
}
```
However that is a bit redundant for most cases. This PR ensures that the tags and service meta of the parent service get copied to the sidecar service. If there are any tags or service meta set in the sidecar service definition then this copying does not take place. After the changes, the query will now return the expected results.
A second change was made to prepared queries in this PR which is to allow filtering on ServiceMeta just like we allow for filtering on NodeMeta.
trujillo-adam