157 Commits (7013d15a23732179d18ec5d17336e16b26fab5d4)

Author	SHA1	Message	Date
Matt Keeler	8837907de4	Make the Agent Cache more Context aware (#8092) ... Blocking queries issues will still be uncancellable (that cannot be helped until we get rid of net/rpc). However this makes it so that if calling getWithIndex (like during a cache Notify go routine) we can cancell the outer routine. Previously it would keep issuing more blocking queries until the result state actually changed.	5 years ago
Kyle Havlovitz	e9e8c0e730	Ingress Gateways for TCP services (#7509) ... * Implements a simple, tcp ingress gateway workflow This adds a new type of gateway for allowing Ingress traffic into Connect from external services. Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>	5 years ago
Pierre Souchay	c9e01ed0a3	Pre-allocations of DNS meta to avoid several allocations	5 years ago
Daniel Nephin	326453eaa1	dns: Remove a few unused params	5 years ago
Matt Keeler	861f754dad	Properly detect no alt domain set (#7323)	5 years ago
Wim	3a2c865ff6	Fix high cpu usage with IPv6 recursor address. Closes #6120 (#6128)	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130) ... * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Aestek	ba8fd8296f	Add support for dual stack IPv4/IPv6 network (#6640) ... * Use consts for well known tagged adress keys * Add ipv4 and ipv6 tagged addresses for node lan and wan * Add ipv4 and ipv6 tagged addresses for service lan and wan * Use IPv4 and IPv6 address in DNS	5 years ago
Wim	4f5d5020b8	dns: fix memoryleak by upgrading outdated miekg/dns (#6748) ... * Add updated github.com/miekg/dns to go modules * Add updated github.com/miekg/dns to vendor * Fix github.com/miekg/dns api breakage * Decrease size when trimming UDP packets Need more room for the header(?), if we don't decrease the size we get an "overflow unpacking uint32" from the dns library * Fix dns truncate tests with api changes * Make windows build working again. Upgrade x/sys and x/crypto and vendor This upgrade is needed because of API breakage in x/sys introduced by the minimal x/sys dependency of miekg/dns This API breakage has been fixed in commit 855e68c859	5 years ago
Matt Keeler	5934f803bf	Sync of OSS changes to support namespaces (#6909)	5 years ago
Nicolas Benoit	0e9a2e5bd0	Fix dns service SRV lookup when service address is a fqdn ... Refactor dns to have same behavior between A and SRV. Current implementation returns the node name instead of the service address. With this fix when querying for SRV record service address is return in the SRV record. And when performing a simple dns lookup it returns a CNAME to the service address.	5 years ago
hashicorp-ci	43bda6fb76	Merge Consul OSS branch 'master' at commit e91f73f592	5 years ago
Akshay Ganeshen	98a35fbe69	dns: support alt domains for dns resolution (#5940) ... this adds an option for an alt domain to be used with dns while migrating to a new consul domain.	5 years ago
Matt Keeler	f3d9b999ee	Add tagged addresses for services (#5965) ... This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.	6 years ago
Pierre Souchay	4a4c63bda0	Ensure Consul is IPv6 compliant (#5468)	6 years ago
Aestek	f669bb7b0f	Add support for DNS config hot-reload (#4875) ... The DNS config parameters `recursors` and `dns_config.*` are now hot reloaded on SIGHUP or `consul reload` and do not need an agent restart to be modified. Config is stored in an atomic.Value and loaded at the beginning of each request. Reloading only affects requests that start _after_ the reload. Ongoing requests are not affected. To match the current behavior the recursor handler is loaded and unloaded as needed on config reload.	6 years ago
R.B. Boyer	f4a3b9d518	fix typos reported by golangci-lint:misspell (#5434)	6 years ago
Aestek	02f991843f	Fix race condition in DNS when using cache (#5398) ... * Fix race condition in DNS when using cache The healty node filtering was modifying the result from the cache, which caused a crash when multiple queries were made to the same service simultaneously. We now copy the node slice before filtering to ensure we do not modify the data stored in the cache. * Fix wording in dns cache config doc s/dns_max_age/cache_max_age/	6 years ago
Aestek	f1cdfbe40e	Allow DNS interface to use agent cache (#5300) ... Adds two new configuration parameters "dns_config.use_cache" and "dns_config.cache_max_age" controlling how DNS requests use the agent cache when querying servers.	6 years ago
Matt Keeler	1736e24fb3	Don't generate TXT records just to discard them (#5272) ... * Don't generate TXT records just to discard them * Add unit test for formatNodeRecord to ensure it prevents returning TXT records	6 years ago
Pierre Souchay	ae7f88f995	Avoid to have infinite recursion in DNS lookups when resolving CNAMEs (#4918) ... * Avoid to have infinite recursion in DNS lookups when resolving CNAMEs This will avoid killing Consul when a Service.Address is using CNAME to a Consul CNAME that creates an infinite recursion. This will fix https://github.com/hashicorp/consul/issues/4907 * Use maxRecursionLevel = 3 to allow several recursions	6 years ago
Paul Banks	b29bc906ee	bugfix: use ServiceTags to generate cache key hash (#4987) ... * bugfix: use ServiceTags to generate cahce key hash * update unit test * update * remote print log * Update .gitignore * Completely deprecate ServiceTag field internally for clarity * Add explicit test for CacheInfo cases	6 years ago
Pierre Souchay	fab55bee2b	dns: implements prefix lookups for DNS TTL (#4605) ... This will fix https://github.com/hashicorp/consul/issues/4509 and allow forinstance lb-* to match services lb-001 or lb-service-007.	6 years ago
Pierre Souchay	251156eb68	Added SOA configuration for DNS settings. (#4714) ... This will allow to fine TUNE SOA settings sent by Consul in DNS responses, for instance to be able to control negative ttl. Will fix: https://github.com/hashicorp/consul/issues/4713 # Example Override all settings: * min_ttl: 0 => 60s * retry: 600 (10m) => 300s (5 minutes), * expire: 86400 (24h) => 43200 (12h) * refresh: 3600 (1h) => 1800 (30 minutes) ``` consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}' ``` Result: ``` dig +multiline @localhost -p 8600 service.consul ; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557 ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;service.consul. IN A ;; AUTHORITY SECTION: consul. 0 IN SOA ns.consul. hostmaster.consul. ( 1537959133 ; serial 1800 ; refresh (30 minutes) 300 ; retry (5 minutes) 43200 ; expire (12 hours) 60 ; minimum (1 minute) ) ;; Query time: 4 msec ;; SERVER: 127.0.0.1#8600(127.0.0.1) ;; WHEN: Wed Sep 26 12:52:13 CEST 2018 ;; MSG SIZE rcvd: 93 ```	6 years ago
Matt Keeler	d3ee66eed4	Add ECS option to EDNS responses where appropriate (#4647) ... This implements parts of RFC 7871 where Consul is acting as an authoritative name server (or forwarding resolver when recursors are configured) If ECS opt is present in the request we will mirror it back and return a response with a scope of 0 (global) or with the same prefix length as the request (indicating its valid specifically for that subnet). We only mirror the prefix-length (non-global) for prepared queries as those could potentially use nearness checks that could be affected by the subnet. In the future we could get more sophisticated with determining the scope bits and allow for better caching of prepared queries that don’t rely on nearness checks. The other thing this does not do is implement the part of the ECS RFC related to originating ECS headers when acting as a intermediate DNS server (forwarding resolver). That would take a quite a bit more effort and in general provide very little value. Consul will currently forward the ECS headers between recursors and the clients transparently, we just don't originate them for non-ECS clients to get potentially more accurate "location aware" results.	6 years ago
Pierre Souchay	eddcf228ea	Implementation of Weights Data structures (#4468) ... * Implementation of Weights Data structures Adding this datastructure will allow us to resolve the issues #1088 and #4198 This new structure defaults to values: ``` { Passing: 1, Warning: 0 } ``` Which means, use weight of 0 for a Service in Warning State while use Weight 1 for a Healthy Service. Thus it remains compatible with previous Consul versions. * Implemented weights for DNS SRV Records * DNS properly support agents with weight support while server does not (backwards compatibility) * Use Warning value of Weights of 1 by default When using DNS interface with only_passing = false, all nodes with non-Critical healthcheck used to have a weight value of 1. While having weight.Warning = 0 as default value, this is probably a bad idea as it breaks ascending compatibility. Thus, we put a default value of 1 to be consistent with existing behaviour. * Added documentation for new weight field in service description * Better documentation about weights as suggested by @banks * Return weight = 1 for unknown Check states as suggested by @banks * Fixed typo (of -> or) in error message as requested by @mkeeler * Fixed unstable unit test TestRetryJoin * Fixed unstable tests * Fixed wrong Fatalf format in `testrpc/wait.go` * Added notes regarding DNS SRV lookup limitations regarding number of instances * Documentation fixes and clarification regarding SRV records with weights as requested by @banks * Rephrase docs	6 years ago
Siva Prasad	865068a358	DNS : Fixes recursors answering the DNS query to properly return the correct response. (#4461) ... * Fixes the DNS recursor properly resolving the requests * Added a test case for the recursor bug * Refactored code && added a test case for all failing recursors * Inner indentation moved into else if check	6 years ago
Matt Keeler	0fd7e97c2d	Merge remote-tracking branch 'origin/master' into bugfix/prevent-multi-cname	6 years ago
Matt Keeler	cbf8f14451	Ensure TXT RRs always end up in the Additional section except for ANY or TXT queries ... This also changes where the enforcement of the enable_additional_node_meta_txt configuration gets applied. formatNodeRecord returns the main RRs and the meta/TXT RRs in separate slices. Its then up to the caller to add to the appropriate sections or not.	6 years ago
Matt Keeler	e3783a75e7	Refactor to make this much less confusing	6 years ago
Matt Keeler	554035974e	Add a bunch of comments about preventing multi-cname ... Hopefully this a bit clearer as to the reasoning	6 years ago
Matt Keeler	22c2be5bf1	Fix some edge cases and add some tests.	6 years ago
Matt Keeler	9a8500412b	Only allow 1 CNAME when querying for a service. ... This just makes sure that if multiple services are registered with unique service addresses that we don’t blast back multiple CNAMEs for the same service DNS name and keeps us within the DNS specs.	6 years ago
mkeeler	6813a99081	Merge remote-tracking branch 'connect/f-connect'	7 years ago
Kyle Havlovitz	050da22473	connect/ca: undo the interface changes and use sign-self-issued in Vault	7 years ago
Matt Keeler	8216816e3f	Make filtering out TXT RRs only apply when they would end up in Additional section ... ANY queries are no longer affected.	7 years ago
Mitchell Hashimoto	406366c45b	agent: working DNS for Connect queries, I think, but have to ... implement Health endpoints to be sure	7 years ago
Matt Keeler	6cc0422408	Add configuration entry to control including TXT records for node meta in DNS responses ... If set to false, the only way to retrieve TXT records for node meta is to specifically query for TXT records.	7 years ago
Matt Keeler	27fe219918	Merge pull request #4131 from pierresouchay/enable_full_dns_compression ... Enable full dns compression	7 years ago
Pierre Souchay	fa37f262eb	Fixed comments for max DNS records returned as requested by @mkeeler	7 years ago
Wim	d10e6d0292	Do reverse service lookup only if address doesn't match node	7 years ago
Wim	5c04864b28	Add support for reverse lookup of services	7 years ago
Pierre Souchay	486417a0fc	Ensure to never send messages more than 64k	7 years ago
Pierre Souchay	cfa5986df7	Fixed unit tests and updated limits	7 years ago
Pierre Souchay	6e80b6b127	Re-Enable compression while computing Len(), so we can send more answers ... This will fix https://github.com/hashicorp/consul/issues/4071	7 years ago
Kyle Havlovitz	ba3971d2c1	Remove deprecated metric names	7 years ago
Pierre Souchay	076ecf9712	Removed unecessary copy of Extra and index	7 years ago
Pierre Souchay	728c5308df	Fixed sync of Extra in binarySearch	7 years ago
Pierre Souchay	fadfb95e07	Added Unit tests + fixed boudary limit	7 years ago
Pierre Souchay	c838376dfa	Added comment for function dnsBinaryTruncate	7 years ago
Pierre Souchay	94c0bf978a	Perform a binary search to find optimal size of DNS responses ... Will fix https://github.com/hashicorp/consul/issues/4036 Instead of removing one by one the entries, find the optimal size using binary search. For SRV records, with 5k nodes, duration of DNS lookups is divided by 4 or more.	7 years ago
Kyle Havlovitz	af4be34a2a	Update make static-assets goal and run format	7 years ago
Matt Keeler	0619efc254	GH-3798: More PR Updates ... Update docs a little Update/add tests. Make sure all the various ways of determining the source IP work Update X-Forwarded-For header parsing. This can be a comma separated list with the first element being the original IP so we now handle csv data there. Got rid of error return from sourceAddrFromRequest	7 years ago
Matt Keeler	cec8d5145b	GH-3798: A few more PR updates	7 years ago
Matt Keeler	d065d3a6db	GH-3798: Updates for PR ... Allow DNS peer IP as the source IP. Break early when the right node was found for executing the preapred query. Update docs	7 years ago
Matt Keeler	45a537def9	GH-3798: Add near=_ip support for prepared queries	7 years ago
Matt Keeler	27899f9f46	Merge pull request #3948 from pierresouchay/fix_tcp_dns_limit ... [BUGFIX] do not break when TCP DNS answer exceeds 64k	7 years ago
Matt Keeler	ebc6f414ac	Formatting update	7 years ago
Matt Keeler	3facce6bfa	GH-3854: Warn when node name isnt a valid DNS label	7 years ago
Pierre Souchay	93fa1f6f49	Optimize size for SRV records, should improve performance a bit ... Stricter Unit tests that checks if truncation was OK.	7 years ago
Pierre Souchay	ce3f47a75d	Performance optimization for services having more than 2k records	7 years ago
Pierre Souchay	7d59249d96	Avoid issue with compression of DNS messages causing overflow	7 years ago
Pierre Souchay	b77fd5ce9d	64000 max limit to DNS messages since there is overhead ... Added debug log to give information about truncation.	7 years ago
Pierre Souchay	be39fb20cc	[BUGFIX] do not break when TCP DNS answer exceeds 64k ... It will avoid having discovery broken when having large number of instances of a service (works with SRV and A* records). Fixes https://github.com/hashicorp/consul/issues/3850	7 years ago
Pierre Souchay	0b7f620dc6	Allow to control the number of A/AAAA Record returned by DNS ... This allows to have randomized resource records (i.e. each answer contains only one IP, but the IP changes every request) for A, AAAA records. It will fix https://github.com/hashicorp/consul/issues/3355 and https://github.com/hashicorp/consul/issues/3937 See https://github.com/hashicorp/consul/issues/3937#issuecomment-370610509 for details. It basically add a new option called `a_record_limit` and will not return more than a_record_limit when performing A, AAAA or ANY DNS requests. The existing `udp_answer_limit` option is still working but should be considered as deprecated since it works only with DNS clients not supporting EDNS.	7 years ago
Andrei Burd	b608091014	adding human readability for dns requests debug log (#3751)	7 years ago
Veselkov Konstantin	5f38e1148a	fix refactoring	7 years ago
Veselkov Konstantin	7de57ba4de	remove golint warnings	7 years ago
Frank Schröder	350932161d	dns: return NXDOMAIN if datacenter is invalid (#3200) (#3596) ... Queries to the DNS server can contain an optional datacenter name in the query name. You can query for 'foo.service.consul' or 'foo.service.dc.consul' to get a response for either the default or a specific datacenter. Datacenter names cannot have dots, therefore the datacenter name can refer to only one element in the DNS query name. The DNS server allowed extra labels between the optional datacenter name and the domain and returned a valid response instead of returning NXDOMAIN. For example, if the domain is set to '.consul' then 'foo.service.dc1.extra.consul' should return NXDOMAIN because of 'extra' being between the datacenter name 'dc1' and the domain '.consul'. Fixes #3200	7 years ago
Ryan Slade	85e4aea9d1	Replace time.Now().Sub(x) with time.Since(x)	7 years ago
Kyle Havlovitz	c728564994	Update metric names and add a legacy config flag	7 years ago
Patrick Sodré	7501331d13	Implement encodeKVasRFC1464 function	7 years ago
Patrick Sodré	865f087ec9	Turn encodeKVasRFC1464 into a plain function	7 years ago
Patrick Sodré	d5e3b9d843	Refactor formatTxtRecords as encodeKVasRFC1464 ... - Move the logic of rfc1035 out of the encoding function - Left basic version of encodingKV as 'k=v'	7 years ago
Patrick Sodré	655c89ee10	Fix editorial suggestions	7 years ago
Patrick Sodré	afb0c92334	Remove redundant check of Node.Meta size	7 years ago
Patrick Sodré	53e812e759	Return Node.Meta info using the DNS interface	7 years ago
Frank Schröder	12216583a1	New config parser, HCL support, multiple bind addrs (#3480) ... * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt	7 years ago
Kyle Havlovitz	baa501e0c5	Fill in the segment in the QuerySource for prepared query lookups	7 years ago
Frank Schroeder	bf96857b17	dns: replace nameserver lookup with consistent rpc call ... This patch replaces the code which determines the list of servers in the current cluster with an RPC call to get the list of active consul service instances which only run on servers. This replaces the previous implementation which was more complex and relied on serf messages which can provide a different view than the consistent response from the raft log. As a side effect it makes the implementation independent of the server and the agent which means it works consistently across both. Different behavior for server and agent was the root cause for the bug in http://github.com/hashicorp/consul/issue/3047. Fixes #3407	7 years ago
Frank Schroeder	4052c6d2d2	dns: split node lookup from request handling	7 years ago
Frank Schroeder	d4e3d4344a	dns: refactor label by unrolling loop	7 years ago
Frank Schroeder	70be1ab635	dns: move ttl closer to usage	7 years ago
Frank Schroeder	1acff3533e	agent: move agent/consul/structs to agent/structs	7 years ago
James Phillips	7442039c2d	Fixes a vet error.	7 years ago
Kyle Havlovitz	cf02e3bc22	Merge pull request #3369 from hashicorp/metrics-enhancements ... Add support for labels/filters from go-metrics	7 years ago
Frank Schroeder	9fa237ddb6	dns: minor cleanups	7 years ago
Kyle Havlovitz	d5634fe2a8	Add support for labels/filters from go-metrics	7 years ago
Preetha Appan	72ae8c8f33	Go back to using <nodename>.node.dc.consul as the name of the ns record being returned.	7 years ago
Frank Schroeder	8a9653bdf8	dns: keep NS names in consul domain	7 years ago
Frank Schroeder	f17bf78bb1	dns: postmaster -> hostmaster	7 years ago
Frank Schroeder	60608b455d	dns: we do not support zone transfers	7 years ago
Frank Schroeder	76b2538915	dns: drop CNAME for primary name server	7 years ago
Preetha Appan	7f34dc08a5	Added test case with IPV6 bind address for NS records, rewrote tests to use verify library and other code review feedback	7 years ago
Preetha Appan	76319f751d	Added back glue records in NS response, expanded unit test. Also reused same function used in node lookup for adding A/AAAA records in the extra section of the NS response	7 years ago
Preetha Appan	f01f17bda3	Don't add A records for NS requests, because the record being returned already resolves correctly. Also fixed all the unit tests, and ignored hostnames that don't meet valid dns hostname criteria	7 years ago
Frank Schroeder	7ea11c2f45	dns: provide correct SOA and NS responses ... This patch changes the behavior of the DNS server as follows: * The SOA response contains the SOA record in the Answer section instead of the Authority section. It also contains NS records in the Authority and the corresponding A glue records in the Extra section. In addition, CNAMEs are added to the Extra section to make the MNAME of the SOA record resolvable. AAAA glue records are not yet supported. * The NS response returns up to three random servers from the consul cluster in the Answer section and the glue A records in the Extra section. AAAA glue records are not yet supported.	7 years ago
Preetha Appan	824fc4ee20	Unify regex used to identify invalid dns characters	7 years ago
Preetha Appan	37f75a393e	Use sanitized version of node name of server in NS record, and start with "server" rather than "ns"	7 years ago
Preetha Appan	794d1afe44	Removed a copy pasted irrelevant comment, and other code review feedback	7 years ago