Commit Graph

2067 Commits (496e6833602acfe6407a6690845e1cf2572175c2)

Author SHA1 Message Date
Chris Piraino 496e683360
Merge pull request #8064 from hashicorp/ingress/health-query-param
Add API query parameter ?ingress to allow users to find ingress gateways associated to a service
2020-06-09 16:08:28 -05:00
Chris Piraino c1d329c5dd Remove TODO note about ingress API, it is done! 2020-06-09 14:58:30 -05:00
Chris Piraino ca41f80493 Set connect or ingress boolean after checking for query param 2020-06-09 14:45:21 -05:00
Daniel Nephin 08f1ed16b4
Merge pull request #7900 from hashicorp/dnephin/add-linter-staticcheck-2
intentions: fix a bug in Intention.SetHash
2020-06-09 15:40:20 -04:00
Daniel Nephin 62a1125c7b
Merge pull request #8037 from hashicorp/dnephin/add-linter-staticcheck-5
ci: Enabled SA2002 staticcheck check
2020-06-09 15:31:24 -04:00
Hans Hasselberg 242994a016
acl: do not resolve local tokens from remote dcs (#8068) 2020-06-09 21:13:09 +02:00
Kyle Havlovitz 0c8966220f
Merge pull request #8040 from hashicorp/ingress/expose-cli
Ingress expose CLI command
2020-06-09 12:11:23 -07:00
Chris Piraino 3c037d9b96 Add ?ingress query parameter on /v1/health/connect
Refactor boolean query parameter logic from ?passing value to re-use
with ingress
2020-06-09 11:44:31 -05:00
Daniel Nephin c66c533d73
Merge pull request #7964 from hashicorp/dnephin/remove-patch-slice-of-maps-forward-compat
config: Use HookWeakDecodeFromSlice in place of PatchSliceOfMaps
2020-06-08 19:53:04 -04:00
Daniel Nephin 75cbbe2702 config: add HookWeakDecodeFromSlice
Currently opaque config blocks (config entries, and CA provider config) are
modified by PatchSliceOfMaps, making it impossible for these opaque
config sections to contain slices of maps.

In order to fix this problem, any lazy-decoding of these blocks needs to support
weak decoding of []map[string]interface{} to a struct type before
PatchSliceOfMaps is replaces. This is necessary because these config
blobs are persisted, and during an upgrade an older version of Consul
could read one of the new configuration values, which would cause an error.

To support the upgrade path, this commit first introduces the new hooks
for weak decoding of []map[string]interface{} and uses them only in the
lazy-decode paths. That way, in a future release, new style
configuration will be supported by the older version of Consul.

This decode hook has a number of advantages:

1. It no longer panics. It allows mapstructure to report the error
2. It no longer requires the user to declare which fields are slices of
   structs. It can deduce that information from the 'to' value.
3. It will make it possible to preserve opaque configuration, allowing
   for structured opaque config.
2020-06-08 17:05:09 -04:00
Hans Hasselberg 98eea08d3b
Tokens converted from legacy ACLs get their Hash computed (#8047)
* Fixes #5606: Tokens converted from legacy ACLs get their Hash computed

This allows new style token replication to work for legacy tokens as well when they change.

* tests: fix timestamp comparison

Co-authored-by: Matt Keeler <mjkeeler7@gmail.com>
2020-06-08 21:44:06 +02:00
Chris Piraino 1a853fc954
Always require Host header values for http services (#7990)
Previously, we did not require the 'service-name.*' host header value
when on a single http service was exposed. However, this allows a user
to get into a situation where, if they add another service to the
listener, suddenly the previous service's traffic might not be routed
correctly. Thus, we always require the Host header, even if there is
only 1 service.

Also, we add the make the default domain matching more restrictive by
matching "service-name.ingress.*" by default. This lines up better with
the namespace case and more accurately matches the Consul DNS value we
expect people to use in this case.
2020-06-08 13:16:24 -05:00
Hans Hasselberg c7e6c9ebec
http: use default minsize for gzip handler. (#7354)
Fixes #6306
2020-06-08 10:10:08 +02:00
Hans Hasselberg 72f92ae7ca
agent: add option to disable agent cache for HTTP endpoints (#8023)
This allows the operator to disable agent caching for the http endpoint.
It is on by default for backwards compatibility and if disabled will
ignore the url parameter `cached`.
2020-06-08 10:08:12 +02:00
Kyle Havlovitz b874c8ef0c Add connect expose CLI command 2020-06-05 14:54:29 -07:00
Daniel Nephin caa692deea ci: Enabled SA2002 staticcheck check
And handle errors in the main test goroutine
2020-06-05 17:50:11 -04:00
Hans Hasselberg 5281cb74db
Setup intermediate_pki_path on secondary when using vault (#8001)
Make sure to mount vault backend for intermediate_pki_path on secondary
dc.
2020-06-05 21:36:22 +02:00
Daniel Nephin ce6cc094a1 intentions: fix a bug in Intention.SetHash
Found using staticcheck.

binary.Write does not accept int types without a size. The error from binary.Write was ignored, so we never saw this error. Casting the data to uint64 produces a correct hash.

Also deprecate the Default{Addr,Port} fields, and prevent them from being encoded. These fields will always be empty and are not used.
Removing these would break backwards compatibility, so they are left in place for now.

Co-authored-by: Hans Hasselberg <me@hans.io>
2020-06-05 14:51:43 -04:00
R.B. Boyer 9cfa4a3fc9
tests: ensure that the ServiceExists helper function normalizes entmeta (#8025)
This fixes a unit test failure over in enterprise due to https://github.com/hashicorp/consul/pull/7384
2020-06-05 10:41:39 +02:00
R.B. Boyer b88bd6660e
server: don't activate federation state replication or anti-entropy until all servers are running 1.8.0+ (#8014) 2020-06-04 16:05:27 -05:00
Hans Hasselberg dfcf45c6cf
tests: use constructor instead init (#8024) 2020-06-04 22:59:06 +02:00
Pierre Souchay 9813ae512b
checks: when a service does not exists in an alias, consider it failing (#7384)
In current implementation of Consul, check alias cannot determine
if a service exists or not. Because a service without any check
is semantically considered as passing, so when no healthchecks
are found for an agent, the check was considered as passing.

But this make little sense as the current implementation does not
make any difference between:
 * a non-existing service (passing)
 * a service without any check (passing as well)

In order to make it work, we have to ensure that when a check did
not find any healthcheck, the service does indeed exists. If it
does not, lets consider the check as failing.
2020-06-04 14:50:52 +02:00
Hans Hasselberg 0f343332da
Merge pull request #7966 from hashicorp/pool_improvements
Agent connection pool cleanup
2020-06-04 08:56:26 +02:00
Freddy 9ed325ba8b
Enable gateways to resolve hostnames to IPv4 addresses (#7999)
The DNS resolution will be handled by Envoy and defaults to LOGICAL_DNS. This discovery type can be overridden on a per-gateway basis with the envoy_dns_discovery_type Gateway Option.

If a service contains an instance with a hostname as an address we set the Envoy cluster to use DNS as the discovery type rather than EDS. Since both mesh gateways and terminating gateways route to clusters using SNI, whenever there is a mix of hostnames and IP addresses associated with a service we use the hostname + CDS rather than the IPs + EDS.

Note that we detect hostnames by attempting to parse the service instance's address as an IP. If it is not a valid IP we assume it is a hostname.
2020-06-03 15:28:45 -06:00
Matt Keeler 771c613dae
Fix legacy management tokens in unupgraded secondary dcs (#7908)
The ACL.GetPolicy RPC endpoint was supposed to return the “parent” policy and not always the default policy. In the case of legacy management tokens the parent policy was supposed to be “manage”. The result of us not sending this properly was that operations that required specifically a management token such as saving a snapshot would not work in secondary DCs until they were upgraded.
2020-06-03 11:22:22 -04:00
Matt Keeler 0e4c65d422
Fix segfault due to race condition for checking server versions (#7957)
The ACL monitoring routine uses c.routers to check for server version updates. Therefore it needs to be started after initializing the routers.
2020-06-03 10:36:32 -04:00
Daniel Nephin 99eb583ebc
Replace goe/verify.Values with testify/require.Equal (#7993)
* testing: replace most goe/verify.Values with require.Equal

One difference between these two comparisons is that go/verify considers
nil slices/maps to be equal to empty slices/maps, where as testify/require
does not, and does not appear to provide any way to enable that behaviour.

Because of this difference some expected values were changed from empty
slices to nil slices, and some calls to verify.Values were left.

* Remove github.com/pascaldekloe/goe/verify

Reduce the number of assertion packages we use from 2 to 1
2020-06-02 12:41:25 -04:00
Alvin Huang 80c34f0461
Merge pull request #7956 from hashicorp/update-master-to-1.8.0-beta2
Update master to 1.8.0 beta2
2020-06-01 16:52:19 -04:00
R.B. Boyer 833211c14c
acl: allow auth methods created in the primary datacenter to optionally create global tokens (#7899) 2020-06-01 11:44:47 -05:00
R.B. Boyer ffb9c7d6f7
acl: remove the deprecated `acl_enforce_version_8` option (#7991)
Fixes #7292
2020-05-29 16:16:03 -05:00
Jono Sosulska c554ba9e10
Replace whitelist/blacklist terminology with allowlist/denylist (#7971)
* Replace whitelist/blacklist terminology with allowlist/denylist
2020-05-29 14:19:16 -04:00
Hans Hasselberg 1fbc1d4777 pool: remove timeout parameter
Timeout was never used in a meaningful way by callers, which is why it
is now entirely internal to the pool.
2020-05-29 08:21:28 +02:00
Hans Hasselberg ad03f863ff pool: remove useTLS and ForceTLS
In the past TLS usage was enforced with these variables, but these days
this decision is made by TLSConfigurator and there is no reason to keep
using the variables.
2020-05-29 08:21:24 +02:00
Hans Hasselberg c45432014b pool: remove version
The version field has been used to decide which multiplexing to use. It
was introduced in 2457293dce. But this is
6y ago and there is no need for this differentiation anymore.
2020-05-28 23:06:01 +02:00
hashicorp-ci cd617dbfa9 update bindata_assetfs.go 2020-05-28 14:39:37 -04:00
hashicorp-ci b3ca11fb0b update bindata_assetfs.go 2020-05-28 14:39:28 -04:00
Daniel Nephin c88fae0aac ci: Add staticcheck and fix most errors
Three of the checks are temporarily disabled to limit the size of the
diff, and allow us to enable all the other checks in CI.

In a follow up we can fix the issues reported by the other checks one
at a time, and enable them.
2020-05-28 11:59:58 -04:00
Daniel Nephin 4f2bff174d
Merge pull request #7963 from hashicorp/dnephin/replace-lib-translate-keys
Replace lib.TranslateKeys with a mapstructure decode hook
2020-05-27 16:51:26 -04:00
Daniel Nephin 6a2d7d77c0 config: use the new HookTranslateKeys instead of lib.TranslateKeys
With the exception of CA provider config, which will be migrated at some
later time.
2020-05-27 16:24:47 -04:00
Daniel Nephin 8ced4300c8 Add alias struct tags for new decode hook 2020-05-27 16:24:47 -04:00
R.B. Boyer 77f2e54618
create lib/stringslice package (#7934) 2020-05-27 11:47:32 -05:00
R.B. Boyer ddd0a13e27
agent: handle re-bootstrapping in a secondary datacenter when WAN federation via mesh gateways is configured (#7931)
The main fix here is to always union the `primary-gateways` list with
the list of mesh gateways in the primary returned from the replicated
federation states list. This will allow any replicated (incorrect) state
to be supplemented with user-configured (correct) state in the config
file. Eventually the game of random selection whack-a-mole will pick a
winning entry and re-replicate the latest federation states from the
primary. If the user-configured state is actually the incorrect one,
then the same eventual correct selection process will work in that case,
too.

The secondary fix is actually to finish making wanfed-via-mgws actually
work as originally designed. Once a secondary datacenter has replicated
federation states for the primary AND managed to stand up its own local
mesh gateways then all of the RPCs from a secondary to the primary
SHOULD go through two sets of mesh gateways to arrive in the consul
servers in the primary (one hop for the secondary datacenter's mesh
gateway, and one hop through the primary datacenter's mesh gateway).
This was neglected in the initial implementation. While everything
works, ideally we should treat communications that go around the mesh
gateways as just provided for bootstrapping purposes.

Now we heuristically use the success/failure history of the federation
state replicator goroutine loop to determine if our current mesh gateway
route is working as intended. If it is, we try using the local gateways,
and if those don't work we fall back on trying the primary via the union
of the replicated state and the go-discover configuration flags.

This can be improved slightly in the future by possibly initializing the
gateway choice to local on startup if we already have replicated state.
This PR does not address that improvement.

Fixes #7339
2020-05-27 11:31:10 -05:00
Raphaël Rondeau 0d2f178b7b
connect: fix endpoints clusterName when using cluster escape hatch (#7319)
```changelog
* fix(connect): fix endpoints clusterName when using cluster escape hatch
```
2020-05-26 10:57:22 +02:00
Pierre Souchay d6649e42af
Stop all watches before shuting down anything dring shutdown. (#7526)
This will prevent watches from being triggered.

```changelog
* fix(agent):  stop all watches before shuting down
```
2020-05-26 10:01:49 +02:00
R.B. Boyer 1b5023cb69
connect: ensure proxy-defaults protocol is used for upstreams (#7938) 2020-05-21 16:08:39 -05:00
Kyle Havlovitz b14696e32a
Standardize support for Tagged and BindAddresses in Ingress Gateways (#7924)
* Standardize support for Tagged and BindAddresses in Ingress Gateways

This updates the TaggedAddresses and BindAddresses behavior for Ingress
to match Mesh/Terminating gateways. The `consul connect envoy` command
now also allows passing an address without a port for tagged/bind
addresses.

* Update command/connect/envoy/envoy.go

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

* PR comments

* Check to see if address is an actual IP address

* Update agent/xds/listeners.go

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

* fix whitespace

Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2020-05-21 09:08:12 -05:00
Daniel Nephin 03291943e1
Merge pull request #7933 from hashicorp/dnephin/state-txn-missing-errors
state: fix unhandled error
2020-05-20 17:00:20 -04:00
Daniel Nephin 04bf0f3490
Update agent/consul/state/catalog.go
Co-authored-by: Hans Hasselberg <me@hans.io>
2020-05-20 16:34:14 -04:00
Seth Hoenig 44ee818d46
grpc: use default resolver scheme for grpc dialing (#7617)
Currently checks of type gRPC will emit log messages such as,

    2020/02/12 13:48:22 [INFO] parsed scheme: ""
    2020/02/12 13:48:22 [INFO] scheme "" not registered, fallback to default scheme

Without adding full support for using custom gRPC schemes (maybe that's
right long-term path) we can just supply the default scheme as provided
by the grpc library.

Fixes https://github.com/hashicorp/consul/issues/7274
and https://github.com/hashicorp/nomad/issues/7415
2020-05-20 22:26:26 +02:00
Daniel Nephin 3f607d9ef0 state: use an error to indicate compare failed
Errors are values. We can use the error value to identify the 'comparison failed' case which makes the function easier to use and should make it harder to miss handle the error case
2020-05-20 12:43:33 -04:00