Commit Graph

61 Commits (7d89e519a26284bc26d0159e39959b70c87ba232)

Author SHA1 Message Date
Kyle Havlovitz c617326470 re-add Connect multi-dc config changes
This reverts commit 8bcfbaffb6.
2018-10-19 08:41:03 -07:00
Jack Pearkes 8bcfbaffb6 Revert "Connect multi-dc config" (#4784) 2018-10-11 17:32:45 +01:00
Aestek 25f04fbd21 [Security] Add finer control over script checks (#4715)
* Add -enable-local-script-checks options

These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.

* Add documentation for the new option

* Nitpick doc wording
2018-10-11 13:22:11 +01:00
Pierre Souchay 251156eb68 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
Kyle Havlovitz 56dc426227 agent: add primary_datacenter and connect replication config options 2018-10-10 12:17:59 -07:00
Paul Banks 8336b5e6b9 XDS Server Config (#4730)
* Config for the coming XDS server

* Default gRPC to 8502 for -dev mode; Re-merge the command Info output that shows gRPC.
2018-10-10 16:55:34 +01:00
Paul Banks 1e7eace066 Add SidecarService Syntax sugar to Service Definition (#4686)
* Added new Config for SidecarService in ServiceDefinitions.

* WIP: all the code needed for SidecarService is written... none of it is tested other than config :). Need API updates too.

* Test coverage for the new sidecarServiceFromNodeService method.

* Test API registratrion with SidecarService

* Recursive Key Translation 🤦

* Add tests for nested sidecar defintion arrays to ensure they are translated correctly

* Use dedicated internal state rather than Service Meta for tracking sidecars for deregistration.

Add tests for deregistration.

* API struct for agent register. No other endpoint should be affected yet.

* Additional test cases to cover updates to API registrations
2018-10-10 16:55:34 +01:00
Paul Banks b83bbf248c Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Pierre Souchay eddcf228ea Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Siva Prasad ca35d04472
Adds a new command line flag -log-file for file based logging. (#4581)
* Added log-file flag to capture Consul logs in a user specified file

* Refactored code.

* Refactored code. Added flags to rotate logs based on bytes and duration

* Added the flags for log file and log rotation on the webpage

* Fixed TestSantize from failing due to the addition of 3 flags

* Introduced changes : mutex, data-dir log writes, rotation logic

* Added test for logfile and updated the default log destination for docs

* Log name now uses UnixNano

* TestLogFile is now uses t.Parallel()

* Removed unnecessary int64Val function

* Updated docs to reflect default log name for log-file

* No longer writes to data-dir and adds .log if the filename has no extension
2018-08-29 16:56:58 -04:00
Matt Keeler 0e0227792b
Gossip tuneables (#4444)
Expose a few gossip tuneables for both lan and wan interfaces

gossip_nodes
gossip_interval
probe_timeout
probe_interval
retransmit_mult
suspicion_mult
2018-07-26 11:39:49 -04:00
Paul Banks 8dd50d5b2d
Add config option to disable HTTP printable char path check (#4442) 2018-07-26 13:53:39 +01:00
Paul Banks 5635227fa6
Allow config-file based Service Definitions for unmanaged proxies and Connect-natice apps. (#4443) 2018-07-25 19:55:41 +01:00
Paul Banks d5e934f9ff
Ooops that was meant to be to a branch no master... EMORECOFFEE
Revert "Add config option to disable HTTP printable char path check"

This reverts commit eebe45a47b.
2018-07-25 15:54:11 +01:00
Paul Banks eebe45a47b
Add config option to disable HTTP printable char path check 2018-07-25 15:52:37 +01:00
Mitchell Hashimoto f0658a0ede
agent/config: support configuring alias check 2018-07-12 09:36:10 -07:00
Mitchell Hashimoto 6ef28dece0
agent/config: parse upstreams with multiple service definitions 2018-06-28 15:13:33 -05:00
mkeeler 6813a99081 Merge remote-tracking branch 'connect/f-connect' 2018-06-25 19:42:51 +00:00
Paul Banks 9f559da913 Revert telemetry config changes ready for cleaner approach 2018-06-25 12:25:39 -07:00
Paul Banks d83f2e8e21 Expose telemetry config from RuntimeConfig to proxy config endpoint 2018-06-25 12:25:38 -07:00
Mitchell Hashimoto f7fc026e18 agent/config: AllowManagedAPIRegistration 2018-06-25 12:25:11 -07:00
Mitchell Hashimoto 4897ca6545 agent/config: add AllowManagedRoot 2018-06-25 12:25:11 -07:00
Paul Banks 0824d1df5f Misc comment cleanups 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 927b45bf91 agent/config: move ports to `ports` structure, update docs 2018-06-25 12:24:15 -07:00
Matt Keeler 8216816e3f Make filtering out TXT RRs only apply when they would end up in Additional section
ANY queries are no longer affected.
2018-06-19 10:08:16 -04:00
Mitchell Hashimoto 37dde6d64a
agent/config: add managed proxy upstreams config to skip
agent/config will turn [{}] into {} (single element maps into a single
map) to work around HCL issues. These are resolved in HCL2 which I'm
sure Consul will switch to eventually.

This breaks the connect proxy configuration in service definition FILES
since we call this patch function. For now, let's just special-case skip
this. In the future we maybe Consul will adopt HCL2 and fix it, or we
can do something else if we want. This works and is tested.
2018-06-14 09:42:13 -07:00
Mitchell Hashimoto aaa2431350
agent: change connect command paths to be slices, not strings
This matches other executable configuration and allows us to cleanly
separate executable from arguments without trying to emulate shell
parsing.
2018-06-14 09:42:08 -07:00
Kyle Havlovitz daa8dd1779
Add CA config to connect section of agent config 2018-06-14 09:42:05 -07:00
Paul Banks e6071051cf
Added connect proxy config and local agent state setup on boot. 2018-06-14 09:41:57 -07:00
Matt Keeler 6cc0422408 Add configuration entry to control including TXT records for node meta in DNS responses
If set to false, the only way to retrieve TXT records for node meta is to specifically query for TXT records.
2018-06-11 11:49:04 -04:00
Preetha Appan 3ff5fd6ec5
More docs and removed SnapShotInterval from raft timing struct stanza 2018-05-11 10:43:24 -05:00
Preetha Appan d721da7b67
Also make snapshot interval configurable 2018-05-11 10:43:24 -05:00
Preetha Appan 66f31cd25a
Make raft snapshot commit threshold configurable 2018-05-11 10:43:24 -05:00
Kyle Havlovitz 75953273e2
Remove unused retry join structs from config 2018-05-08 16:25:34 -07:00
Kyle Havlovitz ba3971d2c1
Remove deprecated metric names 2018-05-08 16:23:15 -07:00
Kyle Havlovitz b73323aa42
Remove the script field from checks in favor of args 2018-05-08 15:31:53 -07:00
Kyle Havlovitz cc214d45b6
Remove support for EnableTagOverride in config files 2018-05-07 16:19:13 -07:00
Kyle Havlovitz 6461087c25
Remove support for CheckID field in service check definitions 2018-05-07 16:15:08 -07:00
Paul Banks c8db140ff7
Merge pull request #4047 from pierresouchay/added_missing_meta_in_service_definition
[BUGFIX] Added Service Meta support in configuration files
2018-04-25 13:08:53 +01:00
Pierre Souchay eccc223480 Fixed Meta name for JSON + Added unit tests for HCL/JSON 2018-04-24 16:39:43 +02:00
Pierre Souchay d2ab3deacf [BUGFIX] Added Service Meta support in configuration files
Fixes https://github.com/hashicorp/consul/issues/4045

Was not added by mistake in https://github.com/hashicorp/consul/pull/3881
2018-04-18 22:18:58 +02:00
Pierre Souchay 93a01b0949 Now use prometheus_retention_time > 0 to enable prometheus support 2018-04-06 14:21:05 +02:00
Pierre Souchay fd98fb1449 Added support exposing metrics in Prometheus format 2018-04-06 09:18:06 +02:00
Preetha a67d27c756
Adds discovery_max_stale (#4004)
Adds a new option to allow service discovery endpoints to return stale results if configured at the agent level.
2018-03-30 10:14:44 -05:00
Pierre Souchay 0b7f620dc6 Allow to control the number of A/AAAA Record returned by DNS
This allows to have randomized resource records (i.e. each
answer contains only one IP, but the IP changes every request) for
A, AAAA records.

It will fix https://github.com/hashicorp/consul/issues/3355 and
https://github.com/hashicorp/consul/issues/3937

See https://github.com/hashicorp/consul/issues/3937#issuecomment-370610509
for details.

It basically add a new option called `a_record_limit` and will not
return more than a_record_limit when performing A, AAAA or ANY DNS
requests.

The existing `udp_answer_limit` option is still working but should
be considered as deprecated since it works only with DNS clients
not supporting EDNS.
2018-03-06 02:07:42 +01:00
James Phillips 5f31c8d8d3
Changes "TLS" to "GRPCUseTLS" since it only applies to GRPC checks. 2018-02-02 17:29:34 -08:00
Dmytro Kostiuchenko 1a10b08e82 Add gRPC health-check #3073 2018-01-04 16:42:30 -05:00
James Phillips 1e49c157c5
Adds a snapshot agent stub to the config structure.
Fixes #3678
2017-11-10 13:50:45 -08:00
James Phillips 93f68555d0
Adds enable_agent_tls_for_checks configuration option which allows (#3661)
HTTP health checks for services requiring 2-way TLS to be checked
using the agent's credentials.
2017-11-07 18:22:09 -08:00
Frank Schröder 874e350b2f config: add -config-format option (#3626)
* config: refactor ReadPath(s) methods without side-effects

Return the sources instead of modifying the state.

* config: clean data dir before every test

* config: add tests for config-file and config-dir

* config: add -config-format option

Starting with Consul 1.0 all config files must have a '.json' or '.hcl'
extension to make it unambigous how the data should be parsed. Some
automation tools generate temporary files by appending a random string
to the generated file which obfuscates the extension and prevents the
file type detection.

This patch adds a -config-format option which can be used to override
the auto-detection behavior by forcing all config files or all files
within a config directory independent of their extension to be
interpreted as of this format.

Fixes #3620
2017-10-31 17:30:01 -05:00