171 Commits (47d7df5c912548805db1e527333ca2080bf01254)

Author	SHA1	Message	Date
Matt Keeler	baa8946ea6	cache: Pass through wait query param to the cache.Get (#5203) ... This adds a MaxQueryTime field to the connect ca leaf cache request type and populates it via the wait query param. The cache will then do the right thing and timeout the operation as expected if no new leaf cert is available within that time. Fixes #4462 The reproduction scenario in the original issue now times out appropriately.	6 years ago
Grégoire Seux	4f62a3b528	Implement /v1/agent/health/service/<service name> endpoint (#3551) ... This endpoint aggregates all checks related to <service id> on the agent and return an appropriate http code + the string describing the worst check. This allows to cleanly expose service status to other component, hiding complexity of multiple checks. This is especially useful to use consul to feed a load balancer which would delegate health checking to consul agent. Exposing this endpoint on the agent is necessary to avoid a hit on consul servers and avoid decreasing resiliency (this endpoint will work even if there is no consul leader in the cluster).	6 years ago
Jack Pearkes	a90c29e60d	Doc changes for 1.4 Final (#4870) ... * website: add multi-dc enterprise landing page * website: switch all 1.4.0 alerts/RC warnings * website: connect product wording Co-Authored-By: pearkes <jackpearkes@gmail.com> * website: remove RC notification * commmand/acl: fix usage docs for ACL tokens * agent: remove comment, OperatorRead * website: improve multi-dc docs Still not happy with this but tried to make it slightly more informative. * website: put back acl guide warning for 1.4.0 * website: simplify multi-dc page and respond to feedback * Fix Multi-DC typos on connect index page. * Improve Multi-DC overview. A full guide is a WIP and will be added post-release. * Fixes typo avaiable > available	6 years ago
Jack Pearkes	8c684db488	New command: consul debug (#4754) ... * agent/debug: add package for debugging, host info * api: add v1/agent/host endpoint * agent: add v1/agent/host endpoint * command/debug: implementation of static capture * command/debug: tests and only configured targets * agent/debug: add basic test for host metrics * command/debug: add methods for dynamic data capture * api: add debug/pprof endpoints * command/debug: add pprof * command/debug: timing, wg, logs to disk * vendor: add gopsutil/disk * command/debug: add a usage section * website: add docs for consul debug * agent/host: require operator:read * api/host: improve docs and no retry timing * command/debug: fail on extra arguments * command/debug: fixup file permissions to 0644 * command/debug: remove server flags * command/debug: improve clarity of usage section * api/debug: add Trace for profiling, fix profile * command/debug: capture profile and trace at the same time * command/debug: add index document * command/debug: use "clusters" in place of members * command/debug: remove address in output * command/debug: improve comment on metrics sleep * command/debug: clarify usage * agent: always register pprof handlers and protect This will allow us to avoid a restart of a target agent for profiling by always registering the pprof handlers. Given this is a potentially sensitive path, it is protected with an operator:read ACL and enable debug being set to true on the target agent. enable_debug still requires a restart. If ACLs are disabled, enable_debug is sufficient. * command/debug: use trace.out instead of .prof More in line with golang docs. * agent: fix comment wording * agent: wrap table driven tests in t.run()	6 years ago
Kyle Havlovitz	c617326470	re-add Connect multi-dc config changes ... This reverts commit 8bcfbaffb6.	6 years ago
Jack Pearkes	8bcfbaffb6	Revert "Connect multi-dc config" (#4784)	6 years ago
Aestek	25f04fbd21	[Security] Add finer control over script checks (#4715) ... * Add -enable-local-script-checks options These options allow for a finer control over when script checks are enabled by giving the option to only allow them when they are declared from the local file system. * Add documentation for the new option * Nitpick doc wording	6 years ago
Kyle Havlovitz	56dc426227	agent: add primary_datacenter and connect replication config options	6 years ago
Paul Banks	c9217c958e	merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in	6 years ago
Paul Banks	161482d2cd	Fix up tests broken by master merge; add proxy tests to services command (and fix it!); actually run the proxycfg.Manager	6 years ago
Paul Banks	dca1303d05	Connect Envoy Command (#4735) ... * Plumb xDS server and proxyxfg into the agent startup * Add `consul connect envoy` command to allow running Envoy as a connect sidecar. * Add test for help tabs; typos and style fixups from review	6 years ago
Paul Banks	e812f5516a	Add -sidecar-for and new /agent/service/:service_id endpoint (#4691) ... - A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it: - **supports hash-based blocking** and so; - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config. - It's not proxy specific and so works for any service. - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested). - The built-in proxy now uses the new endpoint exclusively for it's config - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered. - Several fixes for edge cases for SidecarService - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.	6 years ago
Paul Banks	1e7eace066	Add SidecarService Syntax sugar to Service Definition (#4686) ... * Added new Config for SidecarService in ServiceDefinitions. * WIP: all the code needed for SidecarService is written... none of it is tested other than config :). Need API updates too. * Test coverage for the new sidecarServiceFromNodeService method. * Test API registratrion with SidecarService * Recursive Key Translation 🤦 * Add tests for nested sidecar defintion arrays to ensure they are translated correctly * Use dedicated internal state rather than Service Meta for tracking sidecars for deregistration. Add tests for deregistration. * API struct for agent register. No other endpoint should be affected yet. * Additional test cases to cover updates to API registrations	6 years ago
Paul Banks	b83bbf248c	Add Proxy Upstreams to Service Definition (#4639) ... * Refactor Service Definition ProxyDestination. This includes: - Refactoring all internal structs used - Updated tests for both deprecated and new input for: - Agent Services endpoint response - Agent Service endpoint response - Agent Register endpoint - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Register - Unmanaged deprecated field - Unmanaged new fields - Managed deprecated upstreams - Managed new - Catalog Services endpoint response - Catalog Node endpoint response - Catalog Service endpoint response - Updated API tests for all of the above too (both deprecated and new forms of register) TODO: - config package changes for on-disk service definitions - proxy config endpoint - built-in proxy support for new fields * Agent proxy config endpoint updated with upstreams * Config file changes for upstreams. * Add upstream opaque config and update all tests to ensure it works everywhere. * Built in proxy working with new Upstreams config * Command fixes and deprecations * Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts... TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct. * Fix translated keys in API registration. ≈ * Fixes from docs - omit some empty undocumented fields in API - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally. * Documentation updates for Upstreams in service definition * Fixes for tests broken by many refactors. * Enable travis on f-connect branch in this branch too. * Add consistent Deprecation comments to ProxyDestination uses * Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs	6 years ago
Paul O'Connor	6b7f03911e	Fix prometheus error message (#4745)	6 years ago
Pierre Souchay	2fe728c7bd	Ensure that Proxies ARE always cleaned up, event with DeregisterCriticalServiceAfter (#4649) ... This fixes https://github.com/hashicorp/consul/issues/4648	6 years ago
Pierre Souchay	eddcf228ea	Implementation of Weights Data structures (#4468) ... * Implementation of Weights Data structures Adding this datastructure will allow us to resolve the issues #1088 and #4198 This new structure defaults to values: ``` { Passing: 1, Warning: 0 } ``` Which means, use weight of 0 for a Service in Warning State while use Weight 1 for a Healthy Service. Thus it remains compatible with previous Consul versions. * Implemented weights for DNS SRV Records * DNS properly support agents with weight support while server does not (backwards compatibility) * Use Warning value of Weights of 1 by default When using DNS interface with only_passing = false, all nodes with non-Critical healthcheck used to have a weight value of 1. While having weight.Warning = 0 as default value, this is probably a bad idea as it breaks ascending compatibility. Thus, we put a default value of 1 to be consistent with existing behaviour. * Added documentation for new weight field in service description * Better documentation about weights as suggested by @banks * Return weight = 1 for unknown Check states as suggested by @banks * Fixed typo (of -> or) in error message as requested by @mkeeler * Fixed unstable unit test TestRetryJoin * Fixed unstable tests * Fixed wrong Fatalf format in `testrpc/wait.go` * Added notes regarding DNS SRV lookup limitations regarding number of instances * Documentation fixes and clarification regarding SRV records with weights as requested by @banks * Rephrase docs	6 years ago
Matt Keeler	870a6ad6a8	Handle resolving proxy tokens when parsing HTTP requests (#4453) ... Fixes: #4441 This fixes the issue with Connect Managed Proxies + ACLs being broken. The underlying problem was that the token parsed for most http endpoints was sent untouched to the servers via the RPC request. These changes make it so that at the HTTP endpoint when parsing the token we additionally attempt to convert potential proxy tokens into regular tokens before sending to the RPC endpoint. Proxy tokens are only valid on the agent with the managed proxy so the resolution has to happen before it gets forwarded anywhere.	6 years ago
Matt Keeler	3fe5f566f2	Persist proxies from config files ... Also change how loadProxies works. Now it will load all persisted proxies into a map, then when loading config file proxies will look up the previous proxy token in that map.	6 years ago
Paul Banks	17789d4fe3	register TCP check for managed proxies	7 years ago
Paul Banks	420ae3df69	Limit proxy telemetry config to only be visible with authenticated with a proxy token	7 years ago
Paul Banks	c6ef6a61c9	Refactor to use embedded struct.	7 years ago
Paul Banks	38405bd4a9	Allow user override of proxy telemetry config	7 years ago
Paul Banks	7649d630c6	Basic proxy telemetry working; not sure if it's too ugly; need to instrument things we care about	7 years ago
Paul Banks	d83f2e8e21	Expose telemetry config from RuntimeConfig to proxy config endpoint	7 years ago
Paul Banks	43b48bc06b	Get agent cache tests passing without global hit count (which is racy). ... Few other fixes in here just to get a clean run locally - they are all also fixed in other PRs but shouldn't conflict. This should be robust to timing between goroutines now.	7 years ago
Mitchell Hashimoto	e9e6514c9b	agent: disallow deregistering a managed proxy directly	7 years ago
Mitchell Hashimoto	66a573e496	agent: deregister service deregisters the proxy along with it	7 years ago
Mitchell Hashimoto	0d6dcbd2f1	agent: disallow API registration with managed proxy if not enabled	7 years ago
Paul Banks	e21723a891	Persist proxy state through agent restart	7 years ago
Mitchell Hashimoto	eb2a6952ba	address comment feedback	7 years ago
Mitchell Hashimoto	cd39f09693	agent: leaf endpoint accepts name, not service ID ... This change is important so that requests can made representing a service that may not be registered with the same local agent.	7 years ago
Mitchell Hashimoto	1906fe1c0d	agent: address feedback	7 years ago
Mitchell Hashimoto	54ac5adb08	agent: comments to point to differing logic	7 years ago
Paul Banks	c58d47ba59	Fix broken api test for service Meta (logical conflict rom OSS). Add test that would make this much easier to catch in future.	7 years ago
Paul Banks	4aeab3897c	Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes.	7 years ago
Paul Banks	1722734313	Verify trust domain on /authorize calls	7 years ago
Mitchell Hashimoto	e54e69d11f	agent: verify local proxy tokens for CA leaf + tests	7 years ago
Mitchell Hashimoto	a099c27b07	agent: verify proxy token for ProxyConfig endpoint + tests	7 years ago
Mitchell Hashimoto	171bf8d599	agent: clean up defaulting of proxy configuration ... This cleans up and unifies how proxy settings defaults are applied.	7 years ago
Mitchell Hashimoto	3d3eee2f6e	agent: resolve some conflicts and fix tests	7 years ago
Paul Banks	e0e12e165b	TLS watching integrated into Service with some basic tests. ... There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.	7 years ago
Paul Banks	90c574ebaa	Wire up agent leaf endpoint to cache framework to support blocking.	7 years ago
Paul Banks	cd88b2a351	Basic `watch` support for connect proxy config and certificate endpoints. ... - Includes some bug fixes for previous `api` work and `agent` that weren't tested - Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches. - Integration into `connect` is partially done here but still WIP	7 years ago
Mitchell Hashimoto	6c01e402e0	agent: augment /v1/connect/authorize to cache intentions	7 years ago
Mitchell Hashimoto	6902d721d6	agent: initialize the cache and cache the CA roots	7 years ago
Paul Banks	36dbd878c9	Adds `api` client code and tests for new Proxy Config endpoint, registering with proxy and seeing proxy config in /agent/services list.	7 years ago
Paul Banks	730da74369	Fix various test failures and vet warnings. ... Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.	7 years ago
Paul Banks	1e72ad66f5	Refactor localBlockingQuery to use memdb.WatchSet. Much simpler and correct as a bonus!	7 years ago
Paul Banks	d73f079d0f	Add X-Consul-ContentHash header; implement removing all proxies; add load/unload test.	7 years ago
Paul Banks	2a69663448	Agent Connect Proxy config endpoint with hash-based blocking	7 years ago
Paul Banks	3e3f0e1f31	HTTP agent registration allows proxy to be defined.	7 years ago
Mitchell Hashimoto	95da20ffd7	agent: rename authorize param ClientID to ClientCertURI	7 years ago
Mitchell Hashimoto	6e57233913	agent: add TODO for verification	7 years ago
Mitchell Hashimoto	5a47a53c70	acl: IntentionDefault => IntentionDefaultAllow	7 years ago
Mitchell Hashimoto	ac72a0c5fd	agent: ACL checks for authorize, default behavior	7 years ago
Mitchell Hashimoto	6dc2db94ea	agent/structs: String format for Intention, used for logging	7 years ago
Mitchell Hashimoto	fb7bccc690	agent: bolster commenting for clearer understandability	7 years ago
Mitchell Hashimoto	9a987d6452	agent: default deny on connect authorize endpoint	7 years ago
Mitchell Hashimoto	86a8ce45b9	agent: /v1/agent/connect/authorize is functional, with tests	7 years ago
Mitchell Hashimoto	70d1d5bf06	agent: get rid of method checks since they're done in the http layer	7 years ago
Mitchell Hashimoto	d28ee70a56	agent: implement an always-200 authorize endpoint	7 years ago
Mitchell Hashimoto	c2588262b7	agent: /v1/connect/ca/leaf/:service_id	7 years ago
Mitchell Hashimoto	578db06600	agent/consul: tests for CA endpoints	7 years ago
Mitchell Hashimoto	e7536e5485	agent: /v1/connect/ca/roots	7 years ago
Mitchell Hashimoto	714026dfb7	agent: validate service entry on register	7 years ago
Paul Banks	c8db140ff7	Merge pull request #4047 from pierresouchay/added_missing_meta_in_service_definition ... [BUGFIX] Added Service Meta support in configuration files	7 years ago
Pierre Souchay	26388503e0	Removed Nanoseconds cast as requested by @banks	7 years ago
Pierre Souchay	62a68a008d	Removed content negotiation of Prometheus as requested by @banks	7 years ago
Pierre Souchay	c152cb7bdf	Added Missing Service Meta synchronization and field	7 years ago
Pierre Souchay	27362320e8	Enable compression / automatic Mime-Type detection for Prometheus endpoint	7 years ago
Pierre Souchay	93a01b0949	Now use prometheus_retention_time > 0 to enable prometheus support	7 years ago
Pierre Souchay	fd98fb1449	Added support exposing metrics in Prometheus format	7 years ago
Yoann	0f6e05d4c1	Add support for compression in http api ... The need has been spotted in issue https://github.com/hashicorp/consul/issues/3687. Using "NYTimes/gziphandler", the http api responses can now be compressed if required. The Go API requires compressed response if possible and handle the compressed response. We here change only the http api (not the UI for instance).	7 years ago
Preetha Appan	c7581d68c6	Renames agent API layer for service metadata to "meta" for consistency	7 years ago
Preetha	daa61c5803	Merge pull request #3881 from pierresouchay/service_metadata ... Feature Request: Support key-value attributes for services	7 years ago
Pierre Souchay	980189a33f	Added validation of ServiceMeta in Catalog ... Fixed Error Message when ServiceMeta is not valid Added Unit test for adding a Service with badly formatted ServiceMeta	7 years ago
Guido Iaquinti	8cd11d5888	Add package name to log output	7 years ago
Josh Soref	94835a2715	Spelling (#3958) ... * spelling: another * spelling: autopilot * spelling: beginning * spelling: circonus * spelling: default * spelling: definition * spelling: distance * spelling: encountered * spelling: enterprise * spelling: expands * spelling: exits * spelling: formatting * spelling: health * spelling: hierarchy * spelling: imposed * spelling: independence * spelling: inspect * spelling: last * spelling: latest * spelling: client * spelling: message * spelling: minimum * spelling: notify * spelling: nonexistent * spelling: operator * spelling: payload * spelling: preceded * spelling: prepared * spelling: programmatically * spelling: required * spelling: reconcile * spelling: responses * spelling: request * spelling: response * spelling: results * spelling: retrieve * spelling: service * spelling: significantly * spelling: specifies * spelling: supported * spelling: synchronization * spelling: synchronous * spelling: themselves * spelling: unexpected * spelling: validations * spelling: value	7 years ago
Preetha	c87699abf2	Merge pull request #3885 from eddsteel/support-options-requests ... Support OPTIONS requests	7 years ago
Paul Banks	de58eb1820	Fixes #3891: agent monitor no longer unresponsive before logs stream. ... The root cause is actually that the agent's streaming HTTP API didn't flush until the first log line was found which commonly was pretty soon since the default level is INFO. In cases where there were no logs immediately due to level for instance, the client gets stuck in the HTTP code waiting on a response packet from the server before we enter the loop that checks the shutdown channel from the signal handler. This fix flushes the initial status immediately on the streaming endpoint which lets the client code get into it's expected state where it's listening for shutdown or log lines.	7 years ago
Edd Steel	77f19f7505	Support OPTIONS requests ... - register endpoints with supported methods - support OPTIONS requests, indicating supported methods - extract method validation (error 405) from individual endpoints - on 405 where multiple methods are allowed, create a single Allow header with comma-separated values, not multiple Allow headers.	7 years ago
Pierre Souchay	b259b1609c	Merge remote-tracking branch 'origin/master' into service_metadata	7 years ago
James Phillips	c2a59f1e6c	Addresses additional state mutations. ... Did a sweep of 84d6ac2d51 and checked them all.	7 years ago
James Phillips	1c6de1d623	Fixes all the racy output-side updates to tags.	7 years ago
Pierre Souchay	80dde5465b	Added support for Service Metadata	7 years ago
James Phillips	93fd6bfeb4	Moves the coordinate fetch after the ACL check.	7 years ago
James Phillips	9509aa6c4b	Adds the NodeID field back to the /v1/agent/self Config block. ... Fixes #3778	7 years ago
James Phillips	29367cd5ae	Moves ACL disabled response logic down into endpoints. ... This lets us make the registration of endpoints less fancy, on the road to adding a registration mechanism.	7 years ago
Frank Schroeder	8f145559d8	Decouple the code that executes checks from the agent	7 years ago
Frank Schroeder	b803bf3091	local state: tests compile	7 years ago
Frank Schroeder	29e18c7494	agent: decouple anti-entropy from local state ... The anti-entropy code manages background synchronizations of the local state on a regular basis or on demand when either the state has changed or a new consul server has been added. This patch moves the anti-entropy code into its own package and decouples it from the local state code since they are performing two different functions. To simplify code-review this revision does not make any optimizations, renames or refactorings. This will happen in subsequent commits.	7 years ago
Frank Schroeder	58b0e153f9	Revert "agent: decouple anti-entropy from local state" ... This reverts commit a842dc9c2b.	7 years ago
Frank Schroeder	9ed4b2d631	Revert "local state: tests compile" ... This reverts commit 1af52bf7be.	7 years ago
Frank Schroeder	1af52bf7be	local state: tests compile	7 years ago
Frank Schroeder	a842dc9c2b	agent: decouple anti-entropy from local state ... The anti-entropy code manages background synchronizations of the local state on a regular basis or on demand when either the state has changed or a new consul server has been added. This patch moves the anti-entropy code into its own package and decouples it from the local state code since they are performing two different functions. To simplify code-review this revision does not make any optimizations, renames or refactorings. This will happen in subsequent commits.	7 years ago
preetapan	77c972f594	Fixes agent error handling when check definition is invalid. Distingu… (#3560) ... * Fixes agent error handling when check definition is invalid. Distinguishes between empty checks vs invalid checks * Made CheckTypes return Checks from service definition struct rather than a new copy, and other changes from code review. This also errors when json payload contains empty structs * Simplify and improve validate method, and make sure that CheckTypes always returns a new copy of validated check definitions * Tweaks some small style things and error messages. * Updates the change log.	7 years ago
Frank Schröder	759ef8a1d4	config: add generic method to translate between CamelCase and snake_case (#3557) ... * doc: document discrepancy between id and CheckID * doc: document enable_tag_override change * config: add TranslateKeys helper TranslateKeys makes it easier to map between different representations of internal structures. It allows to recursively map alias keys to canonical keys in structured maps. * config: use TranslateKeys for config file This also adds support for 'enabletagoverride' and removes the need for a separate CheckID alias field. * config: remove dead code * agent: use TranslateKeys for FixupCheckType * agent: translate enable_tag_override during service registration * doc: add '.hcl' as valid extension * config: map ScriptArgs to args * config: add comment for TranslateKeys	7 years ago
Frank Schröder	ce887a0c45	Provide stable config for agent/self (#3532) ... * config: provide stable config for /v1/agent/self (#3530) This patch adds a stable subset of the previous Config struct to the agent/self response. The actual runtime configuration is moved into DebugConfig and will be documented to change. Fixes #3530 * config: fix tests * doc: update api documentation for /v1/agent/self	7 years ago
Frank Schröder	21118cafeb	Recursive sanitize (#3505) ... * vendor: add github.com/sergi/go-diff/diffmatchpatch for diff'ing test output * config: refactor Sanitize to recursively clean runtime config and format complex fields * Removes an extra int cast. * Adds a top-level check test case for sanitization.	7 years ago