Commit Graph

70 Commits (c2a30c5fdd6229ce98d6271b58f12582d7dc9dfe)

Author SHA1 Message Date
Aestek c043de5381 [Security] Allow blocking Write endpoints on Agent using Network Addresses (#4719)
* Add -write-allowed-nets option

* Add documentation for the new write_allowed_nets option
2019-01-10 09:27:26 -05:00
Paul Banks 0638e09b6e
connect: agent leaf cert caching improvements (#5091)
* Add State storage and LastResult argument into Cache so that cache.Types can safely store additional data that is eventually expired.

* New Leaf cache type working and basic tests passing. TODO: more extensive testing for the Root change jitter across blocking requests, test concurrent fetches for different leaves interact nicely with rootsWatcher.

* Add multi-client and delayed rotation tests.

* Typos and cleanup error handling in roots watch

* Add comment about how the FetchResult can be used and change ca leaf state to use a non-pointer state.

* Plumb test override of root CA jitter through TestAgent so that tests are deterministic again!

* Fix failing config test
2019-01-10 12:46:11 +00:00
Matt Keeler 18b29c45c4
New ACLs (#4791)
This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week.
Description

At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers.

On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though.

    Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though.
    All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management.
    Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are:
        A server running the new system must still support other clients using the legacy system.
        A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system.
        The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode.

So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.
2018-10-19 12:04:07 -04:00
Kyle Havlovitz c617326470 re-add Connect multi-dc config changes
This reverts commit 8bcfbaffb6.
2018-10-19 08:41:03 -07:00
Jack Pearkes 8bcfbaffb6 Revert "Connect multi-dc config" (#4784) 2018-10-11 17:32:45 +01:00
Aestek 25f04fbd21 [Security] Add finer control over script checks (#4715)
* Add -enable-local-script-checks options

These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.

* Add documentation for the new option

* Nitpick doc wording
2018-10-11 13:22:11 +01:00
Pierre Souchay 251156eb68 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
Kyle Havlovitz 56dc426227 agent: add primary_datacenter and connect replication config options 2018-10-10 12:17:59 -07:00
Paul Banks c9217c958e merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in 2018-10-10 16:55:34 +01:00
Paul Banks 8336b5e6b9 XDS Server Config (#4730)
* Config for the coming XDS server

* Default gRPC to 8502 for -dev mode; Re-merge the command Info output that shows gRPC.
2018-10-10 16:55:34 +01:00
Paul Banks 1e7eace066 Add SidecarService Syntax sugar to Service Definition (#4686)
* Added new Config for SidecarService in ServiceDefinitions.

* WIP: all the code needed for SidecarService is written... none of it is tested other than config :). Need API updates too.

* Test coverage for the new sidecarServiceFromNodeService method.

* Test API registratrion with SidecarService

* Recursive Key Translation 🤦

* Add tests for nested sidecar defintion arrays to ensure they are translated correctly

* Use dedicated internal state rather than Service Meta for tracking sidecars for deregistration.

Add tests for deregistration.

* API struct for agent register. No other endpoint should be affected yet.

* Additional test cases to cover updates to API registrations
2018-10-10 16:55:34 +01:00
Igal Shprincis e1fe3af37f watch: don't set TLSConfig.Address explicitly (#4727)
Don't set the value of TLSConfig.Address explicitly.

This will make sure env vars like CONSUL_TLS_SERVER_NAME are taken into account for the connection. Fixes #4718.
2018-10-08 22:01:17 +02:00
Siva Prasad ca35d04472
Adds a new command line flag -log-file for file based logging. (#4581)
* Added log-file flag to capture Consul logs in a user specified file

* Refactored code.

* Refactored code. Added flags to rotate logs based on bytes and duration

* Added the flags for log file and log rotation on the webpage

* Fixed TestSantize from failing due to the addition of 3 flags

* Introduced changes : mutex, data-dir log writes, rotation logic

* Added test for logfile and updated the default log destination for docs

* Log name now uses UnixNano

* TestLogFile is now uses t.Parallel()

* Removed unnecessary int64Val function

* Updated docs to reflect default log name for log-file

* No longer writes to data-dir and adds .log if the filename has no extension
2018-08-29 16:56:58 -04:00
Matt Keeler 0e0227792b
Gossip tuneables (#4444)
Expose a few gossip tuneables for both lan and wan interfaces

gossip_nodes
gossip_interval
probe_timeout
probe_interval
retransmit_mult
suspicion_mult
2018-07-26 11:39:49 -04:00
Paul Banks 8dd50d5b2d
Add config option to disable HTTP printable char path check (#4442) 2018-07-26 13:53:39 +01:00
Paul Banks d5e934f9ff
Ooops that was meant to be to a branch no master... EMORECOFFEE
Revert "Add config option to disable HTTP printable char path check"

This reverts commit eebe45a47b.
2018-07-25 15:54:11 +01:00
Paul Banks eebe45a47b
Add config option to disable HTTP printable char path check 2018-07-25 15:52:37 +01:00
Matt Keeler c891e264ca Fix issue with choosing a client addr that is 0.0.0.0 or :: 2018-07-16 16:30:15 -04:00
Matt Keeler 7572ca0f37
Merge pull request #4374 from hashicorp/feature/proxy-env-vars
Setup managed proxy environment with API client env vars
2018-07-12 09:13:54 -04:00
Matt Keeler 0f56ed2d01 Set api.Config’s InsecureSkipVerify to the value of !RuntimeConfig.VerifyOutgoing 2018-07-12 07:49:23 -04:00
Matt Keeler 22e4058893 Use type switch instead of .Network for more reliably detecting UnixAddrs 2018-07-12 07:30:17 -04:00
Matt Keeler 700a275ddf Look specifically for tcp instead of unix
Add runtime -> api.Config tests
2018-07-11 17:25:36 -04:00
Matt Keeler c54b43bef3 PR Updates
Proxy now doesn’t need to know anything about the api as we pass env vars to it instead of the api config.
2018-07-11 09:44:54 -04:00
Matt Keeler 3b6eef8ec6 Pass around an API Config object and convert to env vars for the managed proxy 2018-07-10 12:13:51 -04:00
Pierre Souchay abde81a3e7 Added async-cache with similar behaviour as extend-cache but asynchronously 2018-07-01 23:50:30 +02:00
mkeeler 6813a99081 Merge remote-tracking branch 'connect/f-connect' 2018-06-25 19:42:51 +00:00
Paul Banks c6ef6a61c9 Refactor to use embedded struct. 2018-06-25 12:25:39 -07:00
Paul Banks 9f559da913 Revert telemetry config changes ready for cleaner approach 2018-06-25 12:25:39 -07:00
Paul Banks d83f2e8e21 Expose telemetry config from RuntimeConfig to proxy config endpoint 2018-06-25 12:25:38 -07:00
Paul Banks 8aeb7bd206 Disable TestAgent proxy execution properly 2018-06-25 12:25:38 -07:00
Mitchell Hashimoto f7fc026e18 agent/config: AllowManagedAPIRegistration 2018-06-25 12:25:11 -07:00
Mitchell Hashimoto 4897ca6545 agent/config: add AllowManagedRoot 2018-06-25 12:25:11 -07:00
Paul Banks 0824d1df5f Misc comment cleanups 2018-06-25 12:24:16 -07:00
Paul Banks cdc7cfaa36 Abandon daemonize for simpler solution (preserving history):
Reverts:
  - bdb274852ae469c89092d6050697c0ff97178465
  - 2c689179c4f61c11f0016214c0fc127a0b813bfe
  - d62e25c4a7ab753914b6baccd66f88ffd10949a3
  - c727ffbcc98e3e0bf41e1a7bdd40169bd2d22191
  - 31b4d18933fd0acbe157e28d03ad59c2abf9a1fb
  - 85c3f8df3eabc00f490cd392213c3b928a85aa44
2018-06-25 12:24:10 -07:00
Paul Banks 8cf4b3a6eb Sanity check that we are never trying to self-exec a test binary. Add daemonize bypass for TestAgent so that we don't have to jump through ridiculous self-execution hooks for every package that might possibly invoke a managed proxy 2018-06-25 12:24:09 -07:00
Mitchell Hashimoto aaa2431350
agent: change connect command paths to be slices, not strings
This matches other executable configuration and allows us to cleanly
separate executable from arguments without trying to emulate shell
parsing.
2018-06-14 09:42:08 -07:00
Paul Banks e0e12e165b
TLS watching integrated into Service with some basic tests.
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Kyle Havlovitz daa8dd1779
Add CA config to connect section of agent config 2018-06-14 09:42:05 -07:00
Paul Banks 3e3f0e1f31
HTTP agent registration allows proxy to be defined. 2018-06-14 09:41:57 -07:00
Paul Banks e6071051cf
Added connect proxy config and local agent state setup on boot. 2018-06-14 09:41:57 -07:00
Matt Keeler 6cc0422408 Add configuration entry to control including TXT records for node meta in DNS responses
If set to false, the only way to retrieve TXT records for node meta is to specifically query for TXT records.
2018-06-11 11:49:04 -04:00
Preetha Appan d721da7b67
Also make snapshot interval configurable 2018-05-11 10:43:24 -05:00
Preetha Appan 66f31cd25a
Make raft snapshot commit threshold configurable 2018-05-11 10:43:24 -05:00
Pierre Souchay a680c8e91b Clearer documentation and comments for enabling Prometheus support 2018-04-09 13:16:45 +02:00
Pierre Souchay 93a01b0949 Now use prometheus_retention_time > 0 to enable prometheus support 2018-04-06 14:21:05 +02:00
Pierre Souchay fd98fb1449 Added support exposing metrics in Prometheus format 2018-04-06 09:18:06 +02:00
Preetha a67d27c756
Adds discovery_max_stale (#4004)
Adds a new option to allow service discovery endpoints to return stale results if configured at the agent level.
2018-03-30 10:14:44 -05:00
Josh Soref 94835a2715 Spelling (#3958)
* spelling: another

* spelling: autopilot

* spelling: beginning

* spelling: circonus

* spelling: default

* spelling: definition

* spelling: distance

* spelling: encountered

* spelling: enterprise

* spelling: expands

* spelling: exits

* spelling: formatting

* spelling: health

* spelling: hierarchy

* spelling: imposed

* spelling: independence

* spelling: inspect

* spelling: last

* spelling: latest

* spelling: client

* spelling: message

* spelling: minimum

* spelling: notify

* spelling: nonexistent

* spelling: operator

* spelling: payload

* spelling: preceded

* spelling: prepared

* spelling: programmatically

* spelling: required

* spelling: reconcile

* spelling: responses

* spelling: request

* spelling: response

* spelling: results

* spelling: retrieve

* spelling: service

* spelling: significantly

* spelling: specifies

* spelling: supported

* spelling: synchronization

* spelling: synchronous

* spelling: themselves

* spelling: unexpected

* spelling: validations

* spelling: value
2018-03-19 16:56:00 +00:00
Devin Canterberry c901307a47
🔒 Update supported TLS cipher suites
The list of cipher suites included in this commit are consistent with
the values and precedence in the [Golang TLS documentation](https://golang.org/src/crypto/tls/cipher_suites.go).

> **Note:** Cipher suites with RC4 are still included within the list
> of accepted values for compatibility, but **these cipher suites are
> not safe to use** and should be deprecated with warnings and
> subsequently removed. Support for RC4 ciphers has already been
> removed or disabled by default in many prominent browsers and tools,
> including Golang.
>
> **References:**
>
>  * [RC4 on Wikipedia](https://en.wikipedia.org/wiki/RC4)
>  * [Mozilla Security Blog](https://blog.mozilla.org/security/2015/09/11/deprecating-the-rc4-cipher/)
2018-03-15 10:19:46 -07:00
Pierre Souchay 0b7f620dc6 Allow to control the number of A/AAAA Record returned by DNS
This allows to have randomized resource records (i.e. each
answer contains only one IP, but the IP changes every request) for
A, AAAA records.

It will fix https://github.com/hashicorp/consul/issues/3355 and
https://github.com/hashicorp/consul/issues/3937

See https://github.com/hashicorp/consul/issues/3937#issuecomment-370610509
for details.

It basically add a new option called `a_record_limit` and will not
return more than a_record_limit when performing A, AAAA or ANY DNS
requests.

The existing `udp_answer_limit` option is still working but should
be considered as deprecated since it works only with DNS clients
not supporting EDNS.
2018-03-06 02:07:42 +01:00