Commit Graph

9063 Commits (7d89e519a26284bc26d0159e39959b70c87ba232)

Author SHA1 Message Date
John Cowen 7d89e519a2 UI: New ACLs (#4789)
UI to accompany the new ACLs APIs
2018-10-19 08:45:05 -07:00
Hans Hasselberg 54cc0820b4 Update CHANGELOG.md 2018-10-19 08:45:02 -07:00
Hans Hasselberg 296e9fcdce Update CHANGELOG.md 2018-10-19 08:44:18 -07:00
Hans Hasselberg f0ac02e463 website: update sprockets and ffi to dodge CVEs (#4781)
* Update sprockets to dodge CVE-2018-3760 and CVE-2014-7819
* update ffi to dodge CVE-2018-1000201
2018-10-19 08:41:04 -07:00
Pierre Souchay fab55bee2b dns: implements prefix lookups for DNS TTL (#4605)
This will fix https://github.com/hashicorp/consul/issues/4509 and allow forinstance lb-* to match services lb-001 or lb-service-007.
2018-10-19 08:41:04 -07:00
Aestek c4d6d19d8e website: add service weights documentation in API doc (#4776)
Adds weights documentation to the following API routes :
* Agent: List Services
* Agent Register Service
* Health: List Nodes for Service
2018-10-19 08:41:03 -07:00
Jack Pearkes bfa1a912ab website: note Envoy support in Nomad guide (#4787) 2018-10-19 08:41:03 -07:00
Jack Pearkes 8c684db488 New command: consul debug (#4754)
* agent/debug: add package for debugging, host info

* api: add v1/agent/host endpoint

* agent: add v1/agent/host endpoint

* command/debug: implementation of static capture

* command/debug: tests and only configured targets

* agent/debug: add basic test for host metrics

* command/debug: add methods for dynamic data capture

* api: add debug/pprof endpoints

* command/debug: add pprof

* command/debug: timing, wg, logs to disk

* vendor: add gopsutil/disk

* command/debug: add a usage section

* website: add docs for consul debug

* agent/host: require operator:read

* api/host: improve docs and no retry timing

* command/debug: fail on extra arguments

* command/debug: fixup file permissions to 0644

* command/debug: remove server flags

* command/debug: improve clarity of usage section

* api/debug: add Trace for profiling, fix profile

* command/debug: capture profile and trace at the same time

* command/debug: add index document

* command/debug: use "clusters" in place of members

* command/debug: remove address in output

* command/debug: improve comment on metrics sleep

* command/debug: clarify usage

* agent: always register pprof handlers and protect

This will allow us to avoid a restart of a target agent
for profiling by always registering the pprof handlers.

Given this is a potentially sensitive path, it is protected
with an operator:read ACL and enable debug being
set to true on the target agent. enable_debug still requires
a restart.

If ACLs are disabled, enable_debug is sufficient.

* command/debug: use trace.out instead of .prof

More in line with golang docs.

* agent: fix comment wording

* agent: wrap table driven tests in t.run()
2018-10-19 08:41:03 -07:00
Kyle Havlovitz c617326470 re-add Connect multi-dc config changes
This reverts commit 8bcfbaffb6.
2018-10-19 08:41:03 -07:00
John Cowen 297788408b
ui: Fix freetext searching within the nodes page (#4819)
Essentially this was missing a call to `super`. The error unfortuantely
didn't arise in the tests as it only errors when the node list has 4
items are more (the 4 columns), and the acceptence tests by change were
only filling the page with 3 nodes for test purposes.

I've bumped the amount of nodes up to 4 in the tests, which then causes
the tests to fail, made the fix by adding the `super` call, and the
tests now pass.

I also tested the UI/text searching on a 10,000 node system, and
everything now works as expected.
2018-10-19 14:31:22 +01:00
Rebecca Zanzig a06b825a28
Merge pull request #4812 from hashicorp/docs/helm-16
Add additional formatting detail to the Helm `join` command docs
2018-10-18 13:43:13 -07:00
Rebecca Zanzig 58fc4a7cbc Add additional formatting detail to the Helm `join` command
In response to consul-helm issue 16.
2018-10-17 12:53:44 -07:00
R.B. Boyer 307d91934c
fix some test hangs (#4785)
The default http.Client uses infinite timeouts, so if TestHTTPAPI_MethodNotAllowed_OSS experienced anything going wrong about setup it could hang forever.

Switching to hard coding various http.Client timeouts to non-infinite values at least bounds the failure time.
2018-10-16 16:04:51 -05:00
Jack Pearkes 34e9531482 Putting source back into Dev Mode 2018-10-11 13:31:27 -07:00
Jack Pearkes 297d7268e9 website: bump website for 1.3.0 2018-10-11 11:27:54 -07:00
banks 469768ae39 Release v1.3.0 2018-10-11 18:21:46 +00:00
Assel Meher f912bf0fcd typo: s/checks/services (#4783) 2018-10-11 18:48:51 +01:00
Jack Pearkes 8bcfbaffb6 Revert "Connect multi-dc config" (#4784) 2018-10-11 17:32:45 +01:00
Paul Banks ad7451d7cf
Update CHANGELOG.md 2018-10-11 13:35:02 +01:00
danielehc 40017f5e59
Update creating-certificates.html.md (#4780)
In case `verify_server_hostname` is set in the configuration, Consul checks the certificate against  `server.<datacenter>.<domain>`.

The name suggested by the guide generates errors like the following:
```
2018/10/10 12:42:20 [ERR] consul: Failed to confirm peer status for consul-3: rpc error getting client: failed to get conn: x509: certificate is valid for server.node.consul.labs, localhost, not server.consul.labs. Retrying in 16s...
```

Removing the `node` part from the certificate permits them to work also when that option is set.
2018-10-11 14:23:51 +02:00
Aestek 25f04fbd21 [Security] Add finer control over script checks (#4715)
* Add -enable-local-script-checks options

These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.

* Add documentation for the new option

* Nitpick doc wording
2018-10-11 13:22:11 +01:00
Paul Banks 298af6dca7
Quick fix for cache age flakiness in CI 2018-10-11 13:12:19 +01:00
Paul Banks f2d915e92d
Update CHANGELOG.md 2018-10-11 12:56:05 +01:00
Rebecca Zanzig 34e5516834 Support multiple tags for health and catalog http api endpoints (#4717)
* Support multiple tags for health and catalog api endpoints

Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

* Update the api package to support multiple tags

Includes additional tests.

* Update new tests to use the `require` library

* Update HealthConnect check after a bad merge
2018-10-11 12:50:05 +01:00
Pierre Souchay 51b33ef015 [Performance On Large clusters] Reduce updates on large services (#4720)
* [Performance On Large clusters] Checks do update services/nodes only when really modified to avoid too many updates on very large clusters

In a large cluster, when having a few thousands of nodes, the anti-entropy
mechanism performs lots of changes (several per seconds) while
there is no real change. This patch wants to improve this in order
to increase Consul scalability when using many blocking requests on
health for instance.

* [Performance for large clusters] Only updates index of service if service is really modified

* [Performance for large clusters] Only updates index of nodes if node is really modified

* Added comments / ensure IsSame() has clear semantics

* Avoid having modified boolean, return nil directly if stutures are Same

* Fixed unstable unit tests TestLeader_ChangeServerID

* Rewrite TestNode_IsSame() for better readability as suggested by @banks

* Rename ServiceNode.IsSame() into IsSameService() + added unit tests

* Do not duplicate TestStructs_ServiceNode_Conversions() and increase test coverage of IsSameService

* Clearer documentation in IsSameService

* Take into account ServiceProxy into ServiceNode.IsSameService()

* Fixed IsSameService() with all new structures
2018-10-11 12:42:39 +01:00
Paul Banks f0c06a912e
Bump Go version for CI and build to 1.11.1 (#4782) 2018-10-11 12:21:53 +01:00
Paul Banks f951dcfb9d
Be more explicit about rolling upgrade process 2018-10-11 11:44:20 +01:00
Paul Banks 5952804334
Move 1.0.6 upgrade docs that were added in the wrong place 2018-10-11 11:38:55 +01:00
Paul Banks e90a0c35b6
f-envoy branch CHANGELOG entries 2018-10-11 11:21:42 +01:00
Paul Banks 51c0001aad
[WIP] Initial draft of Sidecar Service and Managed Proxy deprecation docs (#4752)
* Initial draft of Sidecar Service and Managed Proxy deprecation docs

* Service definition deprecation notices and sidecar service

* gRPC and sidecar service config options; Deprecate managed proxy options

* Envoy Docs: Basic envoy command; envoy getting started/intro

* Remove change that snuck in

* Envoy custom config example

* Add agent/service API docs; deprecate proxy config endpoint

* Misc grep cleanup for managed proxies; capitalize Envoy

* Updates to getting started guide

* Add missing link

* Refactor Envoy guide into a separate guide and add bootstrap reference notes.

* Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle
2018-10-11 10:44:42 +01:00
John Cowen 852b930739
Update CHANGELOG.md 2018-10-11 10:32:33 +01:00
John Cowen 5f39bfd161
ui: Improve layout of node cards on large and small screens (#4761)
1. The grid based unhealthy cards are now clamped to only four wide
maximum. This means that on larger screen the cards are much wider
meaning you can view more information. Grid gutters are also clamped at
a certain ideal width screen, remaining responsive for anything below
this.

2. The healthy node columns are finally responsive following the same
column rules as unhealthy nodes
2018-10-11 10:19:06 +01:00
John Cowen 461db096ed
ui: Add truncation to the `.Address` (#4760) 2018-10-11 10:17:56 +01:00
John Cowen b75731aaec
ui: Cope with service names that contain slashes (#4756) 2018-10-11 10:14:04 +01:00
John Cowen 3410f5955a
ui: Reduce the breakpoint at which the smaller footer kicks in (#4723) 2018-10-11 10:13:17 +01:00
Matt Keeler be52793aad
Update CHANGELOG.md 2018-10-10 15:52:13 -04:00
Pierre Souchay 251156eb68 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
Kyle Havlovitz 391dbcf8dd
Merge pull request #4687 from hashicorp/connect-multidc-config
Connect multi-dc config
2018-10-10 12:33:00 -07:00
Kyle Havlovitz e4349c5710 connect/ca: more OSS split for multi-dc 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 0da4f2b2e8 connect/ca: split CA initialization logic between oss/enterprise 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 56dc426227 agent: add primary_datacenter and connect replication config options 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 98d95cfa80 connect: add ExternalTrustDomain to CARoot fields 2018-10-10 12:16:47 -07:00
Kyle Havlovitz 46c829b879 docs: deprecate acl_datacenter and replace it with primary_datacenter 2018-10-10 12:16:47 -07:00
Paul Banks c9217c958e merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in 2018-10-10 16:55:34 +01:00
Paul Banks f9c0f00abb cli: envoy command default gRPC port (#4768)
* Default gRPC port; Start on some basic tests for argument and ENV handling; Make Exec test less platform-dependent.

* Allow hot-restarts

* Remove debug
2018-10-10 16:55:34 +01:00
R.B. Boyer c310451b2b cli: avoid passing envoy bootstrap configuration as arguments (#4747)
Play a trick with CLOEXEC to pass the envoy bootstrap configuration as
an open file descriptor to the exec'd envoy process. The file only
briefly touches disk before being unlinked.

We convince envoy to read from this open file descriptor by using the
/dev/fd/$FDNUMBER mechanism to read the open file descriptor as a file.

Because the filename no longer has an extension envoy's sniffing logic
falls back on JSON instead of YAML, so the bootstrap configuration must
be generated as JSON instead.
2018-10-10 16:55:34 +01:00
Paul Banks 161482d2cd Fix up tests broken by master merge; add proxy tests to services command (and fix it!); actually run the proxycfg.Manager 2018-10-10 16:55:34 +01:00
Paul Banks a28e4a33b2 Fix bug in leaf-cert cache type where multiple client tokens collide (#4736)
* Fix bug in leaf-cert cache type where multiple clients with different tokens would share certs and block incorrectly

* Use hash for issued certs key to avoid ambiguity concatenating
2018-10-10 16:55:34 +01:00
Paul Banks dca1303d05 Connect Envoy Command (#4735)
* Plumb xDS server and proxyxfg into the agent startup

* Add `consul connect envoy` command to allow running Envoy as a connect sidecar.

* Add test for help tabs; typos and style fixups from review
2018-10-10 16:55:34 +01:00
Paul Banks 1909a95118 xDS Server Implementation (#4731)
* Vendor updates for gRPC and xDS server

* xDS server implementation for serving Envoy as a Connect proxy

* Address initial review comments

* consistent envoy package aliases; typos fixed; override TLS and authz for custom listeners

* Moar Typos

* Moar typos
2018-10-10 16:55:34 +01:00