1706 Commits (0687e2fe934e1313eb0bb0b4bc003b44fde2e762)

Author	SHA1	Message	Date
R.B. Boyer	38d76c624e	Allow for both snake_case and CamelCase for config entries written with 'consul config write'. (#6044) ... This also has the added benefit of fixing an issue with passing time.Duration fields through config entries.	6 years ago
Hans Hasselberg	a82e6a7fd3	Release v1.5.2	6 years ago
Hans Hasselberg	33a7df3330	tls: auto_encrypt enables automatic RPC cert provisioning for consul clients (#5597)	6 years ago
R.B. Boyer	6a52f9f9fb	initial version of L7 config entry compiler (#5994) ... With this you should be able to fetch all of the relevant discovery chain config entries from the state store in one query and then feed them into the compiler outside of a transaction. There are a lot of TODOs scattered through here, but they're mostly around handling fun edge cases and can be deferred until more of the plumbing works completely.	6 years ago
R.B. Boyer	ceef44bbc9	adding new config entries for L7 discovery chain (unused) (#5987)	6 years ago
Todd Radel	a18b6d5ab9	connect: store signingKeyId instead of authorityKeyId (#6005)	6 years ago
R.B. Boyer	f7fdf18335	fix test that was failing after #6013 (#6026)	6 years ago
Aestek	81f8092a42	acl: allow service deregistration with node write permission (#5217) ... With ACLs enabled if an agent is wiped and restarted without a leave it can no longer deregister the services it had previously registered because it no longer has the tokens the services were registered with. To remedy that we allow service deregistration from tokens with node write permission.	6 years ago
Akshay Ganeshen	98a35fbe69	dns: support alt domains for dns resolution (#5940) ... this adds an option for an alt domain to be used with dns while migrating to a new consul domain.	6 years ago
hashicorp-ci	f4304e2e5b	Merge Consul OSS branch 'master' at commit 4eb73973b6	6 years ago
Pierre Souchay	4eb73973b6	agent: added metadata information about servers into consul service description (#5455) ... This allows have information about servers from HTTP APIs without using the command line.	6 years ago
Sarah Christoff	d3d92d76f3	ui: modify content path (#5950) ... * Add ui-content-path flag * tests complete, regex validator on string, index.html updated * cleaning up debugging stuff * ui: Enable ember environment configuration to be set via the go binary at runtime (#5934) * ui: Only inject {{.ContentPath}} if we are makeing a prod build... ...otherwise we just use the current rootURL This gets injected into a <base /> node which solves the assets path problem but not the ember problem * ui: Pull out the <base href=""> value and inject it into ember env See previous commit: The <base href=""> value is 'sometimes' injected from go at index serve time. We pass this value down to ember by overwriting the ember config that is injected via a <meta> tag. This has to be done before ember bootup. Sometimes (during testing and development, basically not production) this is injected with the already existing value, in which case this essentially changes nothing. The code here is slightly abstracted away from our specific usage to make it easier for anyone else to use, and also make sure we can cope with using this same method to pass variables down from the CLI through to ember in the future. * ui: We can't use <base /> move everything to javascript (#5941) Unfortuantely we can't seem to be able to use <base> and rootURL together as URL paths will get doubled up (`ui/ui/`). This moves all the things that we need to interpolate with .ContentPath to the `startup` javascript so we can conditionally print out `{{.ContentPath}}` in lots of places (now we can't use base) * fixed when we serve index.html * ui: For writing a ContentPath, we also need to cope with testing... (#5945) ...and potentially more environments Testing has more additional things in a separate index.html in `tests/` This make the entire thing a little saner and uses just javascriopt template literals instead of a pseudo handbrake synatx for our templating of these files. Intead of just templating the entire file this way, we still only template `{{content-for 'head'}}` and `{{content-for 'body'}}` in this way to ensure we support other plugins/addons * build: Loosen up the regex for retrieving the CONSUL_VERSION (#5946) * build: Loosen up the regex for retrieving the CONSUL_VERSION 1. Previously the `sed` replacement was searching for the CONSUL_VERSION comment at the start of a line, it no longer does this to allow for indentation. 2. Both `grep` and `sed` where looking for the omment at the end of the line. We've removed this restriction here. We don't need to remove it right now, but if we ever put the comment followed by something here the searching would break. 3. Added `xargs` for trimming the resulting version string. We aren't using this already in the rest of the scripts, but we are pretty sure this is available on most systems. * ui: Fix erroneous variable, and also force an ember cache clean on build 1. We referenced a variable incorrectly here, this fixes that. 2. We also made sure that every `make` target clears ember's `tmp` cache to ensure that its not using any caches that have since been edited everytime we call a `make` target. * added docs, fixed encoding * fixed go fmt * Update agent/config/config.go Co-Authored-By: R.B. Boyer <public@richardboyer.net> * Completed Suggestions * run gofmt on http.go * fix testsanitize * fix fullconfig/hcl by setting correct 'want' * ran gofmt on agent/config/runtime_test.go * Update website/source/docs/agent/options.html.md Co-Authored-By: Hans Hasselberg <me@hans.io> * Update website/source/docs/agent/options.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * remove contentpath from redirectFS struct	6 years ago
Pierre Souchay	0e907f5aa8	Support for maximum size for Output of checks (#5233) ... * Support for maximum size for Output of checks This PR allows users to limit the size of output produced by checks at the agent and check level. When set at the agent level, it will limit the output for all checks monitored by the agent. When set at the check level, it can override the agent max for a specific check but only if it is lower than the agent max. Default value is 4k, and input must be at least 1.	6 years ago
hashicorp-ci	4d185baf55	Merge Consul OSS branch 'master' at commit 88b15d84f9 ... skip-checks: true	6 years ago
Matt Keeler	813e009a2d	Prepare for having different service kinds that are all generic… (#6013) ... Default to internal error when service kind is unknown	6 years ago
Matt Keeler	43c5ba0304	New Cache Types (#5995) ... * Add a cache type for the Catalog.ListServices endpoint * Add a cache type for the Catalog.ListDatacenters endpoint	6 years ago
Matt Keeler	19e70c46bf	Ensure that looking for services by addreses works with Tagged Addresses (#5984)	6 years ago
Matt Keeler	6cc1451895	Update some tests to fix ContentHash broken by the tagged service addresses (#5996)	6 years ago
Hans Hasselberg	f13fe4b304	agent: transfer leadership when establishLeadership fails (#5247)	6 years ago
Aestek	b839f52195	kv: do not trigger watches when setting the same value (#5885) ... If a KVSet is performed but does not update the entry, do not trigger watches for this key. This avoids releasing blocking queries for KV values that did not actually changed.	6 years ago
Aestek	24a0f2bba2	ae: use stale requests when performing full sync (#5873) ... Read requests performed during anti antropy full sync currently target the leader only. This generates a non-negligible load on the leader when the DC is large enough and can be offloaded to the followers following the "eventually consistent" policy for the agent state. We switch the AE read calls to use stale requests with a small (2s) MaxStaleDuration value and make sure we do not read too fast after a write.	6 years ago
Matt Keeler	f3d9b999ee	Add tagged addresses for services (#5965) ... This allows addresses to be tagged at the service level similar to what we allow for nodes already. The address translation that can be enabled with the `translate_wan_addrs` config was updated to take these new addresses into account as well.	6 years ago
Matt Keeler	2557d7a6cc	Fix CAS operations on Services (#5971) ... * Fix CAS operations on services * Update agent/consul/state/catalog_test.go Co-Authored-By: R.B. Boyer <public@richardboyer.net>	6 years ago
Paul Banks	acfcc7daf4	Add rate limiting to RPCs sent within a server instance too (#5927)	6 years ago
Paul Banks	ffcfdf29fc	Upgrade xDS (go-control-plane) API to support Envoy 1.10. (#5872) ... * Upgrade xDS (go-control-plane) API to support Envoy 1.10. This includes backwards compatibility shim to work around the ext_authz package rename in 1.10. It also adds integration test support in CI for 1.10.0. * Fix go vet complaints * go mod vendor * Update Envoy version info in docs * Update website/source/docs/connect/proxies/envoy.md	6 years ago
Pierre Souchay	4a4c63bda0	Ensure Consul is IPv6 compliant (#5468)	6 years ago
Matt Keeler	2ba6c3ac00	Update links to envoy docs on xDS protocol (#5871)	6 years ago
R.B. Boyer	40336fd353	agent: fix several data races and bugs related to node-local alias checks (#5876) ... The observed bug was that a full restart of a consul datacenter (servers and clients) in conjunction with a restart of a connect-flavored application with bring-your-own-service-registration logic would very frequently cause the envoy sidecar service check to never reflect the aliased service. Over the course of investigation several bugs and unfortunate interactions were corrected: (1) local.CheckState objects were only shallow copied, but the key piece of data that gets read and updated is one of the things not copied (the underlying Check with a Status field). When the stock code was run with the race detector enabled this highly-relevant-to-the-test-scenario field was found to be racy. Changes: a) update the existing Clone method to include the Check field b) copy-on-write when those fields need to change rather than incrementally updating them in place. This made the observed behavior occur slightly less often. (2) If anything about how the runLocal method for node-local alias check logic was ever flawed, there was no fallback option. Those checks are purely edge-triggered and failure to properly notice a single edge transition would leave the alias check incorrect until the next flap of the aliased check. The change was to introduce a fallback timer to act as a control loop to double check the alias check matches the aliased check every minute (borrowing the duration from the non-local alias check logic body). This made the observed behavior eventually go away when it did occur. (3) Originally I thought there were two main actions involved in the data race: A. The act of adding the original check (from disk recovery) and its first health evaluation. B. The act of the HTTP API requests coming in and resetting the local state when re-registering the same services and checks. It took awhile for me to realize that there's a third action at work: C. The goroutines associated with the original check and the later checks. The actual sequence of actions that was causing the bad behavior was that the API actions result in the original check to be removed and re-added _without waiting for the original goroutine to terminate_. This means for brief windows of time during check definition edits there are two goroutines that can be sending updates for the alias check status. In extremely unlikely scenarios the original goroutine sees the aliased check start up in `critical` before being removed but does not get the notification about the nearly immediate update of that check to `passing`. This is interlaced wit the new goroutine coming up, initializing its base case to `passing` from the current state and then listening for new notifications of edge triggers. If the original goroutine "finishes" its update, it then commits one more write into the local state of `critical` and exits leaving the alias check no longer reflecting the underlying check. The correction here is to enforce that the old goroutines must terminate before spawning the new one for alias checks.	6 years ago
Freddy	6b31482333	Increase reliability of TestResetSessionTimerLocked_Renew	6 years ago
Pierre Souchay	e892981418	agent: Improve startup message to avoid confusing users when no error occurs (#5896) ... * Improve startup message to avoid confusing users when no error occurs Several times, some users not very familiar with Consul get confused by error message at startup: `[INFO] agent: (LAN) joined: 1 Err: <nil>` Having `Err: <nil>` seems weird to many users, I propose to have the following instead: * Success: `[INFO] agent: (LAN) joined: 1` * Error: `[WARN] agent: (LAN) couldn't join: %d Err: ERROR`	6 years ago
Freddy	17e74985b0	Run TestServer_Expect on its own (#5890)	6 years ago
Freddy	6c19cacd42	Flaky test: ACLReplication_Tokens (#5891) ... * Exclude non-go workflows while testing * Wait for s2 global-management policy * Revert "Exclude non-go workflows while testing" This reverts commit 47a83cbe9f.	6 years ago
Freddy	d4ea163b0b	Add retries to StatsFetcherTest (#5892)	6 years ago
Jack Pearkes	40cec98468	Release v1.5.1	6 years ago
freddygv	40b809bce3	Wait for s2 global-management policy	6 years ago
Freddy	e9259ca97a	Change log line used for verification	6 years ago
Freddy	d1c315fad9	Stop running TestLeader_ChangeServerID in parallel	6 years ago
Sarah Christoff	32b5992d0f	Add retries around `obj`	6 years ago
Sarah Christoff	73d73e0e20	Add retries to all `obj`	6 years ago
Sarah Christoff	2a018e5e0a	Update agent/coordinate_endpoint_test.go ... Co-Authored-By: Freddy <freddygv@users.noreply.github.com>	6 years ago
Sarah Christoff	b96d9b01bd	Update type assertion logic ... Logic updated to evaluate with a boolean after the type assertion. This allows us to check if the type assertion succeeded and be more clear with errors.	6 years ago
Kyle Havlovitz	31bb9d67df	Set the dead node reclaim timer at 30s	6 years ago
Kyle Havlovitz	29eb83c9c2	Merge branch 'master' into change-node-id	6 years ago
Jack Pearkes	34eff659dc	Release v1.5.0	6 years ago
Matt Keeler	dbc48ea3f7	Fixes race condition in Agent Cache (#5796) ... * Fix race condition during a cache get Check the entry we pulled out of the cache while holding the lock had Fetching set. If it did then we should use the existing Waiter instead of calling fetch. The reason this is better than just calling fetch is that fetch re-gets the entry out of the entries map and the previous fetch may have finished. Therefore this prevents erroneously starting a new fetch because we just missed the last update. * Fix race condition fully The first commit still allowed for the following scenario: • No entry existing when checked in getWithIndex while holding the read lock • Then by time we had reached fetch it had been created and finished. * always use ok when returning * comment mentioning the reading from entries. * use cacheHit consistently	6 years ago
Matt Keeler	dbf0a0f6c0	Copy the proxy config instead of direct assignment (#5786) ... This prevents modifying the data in the state store which is supposed to be immutable.	6 years ago
R.B. Boyer	20eefeea11	acl: a role binding rule for a role that does not exist should be ignored (#5778) ... I wrote the docs under this assumption but completely forgot to actually enforce it.	6 years ago
R.B. Boyer	b4371bcccd	acl: enforce that you cannot persist tokens and roles with missing links except during replication (#5779)	6 years ago
Matt Keeler	42d32db817	Fix ConfigEntryResponse binary marshaller and ensure we watch the chan in ConfigEntry.Get even when no entry exists. (#5773)	6 years ago
Paul Banks	6a58527cd8	Fix previous accidental master push 🤦 (#5771) ... * Fix previous accidental master push 🤦 * Fix ACL test	6 years ago
Paul Banks	6c81f9da0d	Fix panic in Resolving service config when proxy-defaults isn't defined yet (#5769)	6 years ago
Paul Banks	8f5b16ebaf	Fix uint8 conversion issues for service config response maps.	6 years ago
Paul Banks	0cfb6051ea	Add integration test for central config; fix central config WIP (#5752) ... * Add integration test for central config; fix central config WIP * Add integration test for central config; fix central config WIP * Set proxy protocol correctly and begin adding upstream support * Add upstreams to service config cache key and start new notify watcher if they change. This doesn't update the tests to pass though. * Fix some merging logic get things working manually with a hack (TODO fix properly) * Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway * Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does * Fix up service manageer and API test failures * Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally * Remove version.go hack - will make integration test fail until release * Remove unused code from commands and upstream merge * Re-bump version to 1.5.0	6 years ago
Matt Keeler	69f902608c	Update to use a consulent build tag instead of just ent (#5759)	6 years ago
Matt Keeler	3145bf5230	Centralized Config CLI (#5731) ... * Add HTTP endpoints for config entry management * Finish implementing decoding in the HTTP Config entry apply endpoint * Add CAS operation to the config entry apply endpoint Also use this for the bootstrapping and move the config entry decoding function into the structs package. * First pass at the API client for the config entries * Fixup some of the ConfigEntry APIs Return a singular response object instead of a list for the ConfigEntry.Get RPC. This gets plumbed through the HTTP API as well. Dont return QueryMeta in the JSON response for the config entry listing HTTP API. Instead just return a list of config entries. * Minor API client fixes * Attempt at some ConfigEntry api client tests These don’t currently work due to weak typing in JSON * Get some of the api client tests passing * Implement reflectwalk magic to correct JSON encoding a ProxyConfigEntry Also added a test for the HTTP endpoint that exposes the problem. However, since the test doesn’t actually do the JSON encode/decode its still failing. * Move MapWalk magic into a binary marshaller instead of JSON. * Add a MapWalk test * Get rid of unused func * Get rid of unused imports * Fixup some tests now that the decoding from msgpack coerces things into json compat types * Stub out most of the central config cli Fully implement the config read command. * Basic config delete command implementation * Implement config write command * Implement config list subcommand Not entirely sure about the output here. Its basically the read output indented with a line specifying the kind/name of each type which is also duplicated in the indented output. * Update command usage * Update some help usage formatting * Add the connect enable helper cli command * Update list command output * Rename the config entry API client methods. * Use renamed apis * Implement config write tests Stub the others with the noTabs tests. * Change list output format Now just simply output 1 line per named config * Add config read tests * Add invalid args write test. * Add config delete tests * Add config list tests * Add connect enable tests * Update some CLI commands to use CAS ops This also modifies the HTTP API for a write op to return a boolean indicating whether the value was written or not. * Fix up the HTTP API CAS tests as I realized they weren’t testing what they should. * Update config entry rpc tests to properly test CAS * Fix up a few more tests * Fix some tests that using ConfigEntries.Apply * Update config_write_test.go * Get rid of unused import	6 years ago
Matt Keeler	f665695b6b	Ensure ServiceName is populated correctly for agent service checks ... Also update some snapshot agent docs * Enforce correct permissions when registering a check Previously we had attempted to enforce service:write for a check associated with a service instead of node:write on the agent but due to how we decoded the health check from the request it would never do it properly. This commit fixes that. * Update website/source/docs/commands/snapshot/agent.html.markdown.erb Co-Authored-By: mkeeler <mkeeler@users.noreply.github.com>	6 years ago
Matt Keeler	d0f410cd84	Make a few config entry endpoints return 404s and allow for snake_case and lowercase key names. (#5748)	6 years ago
Freddy	44e3dd79ff	go fmt runtime_test.go	6 years ago
Freddy	d19eb36085	Restrict config file extensions read	6 years ago
Matt Keeler	4daa1585b0	ACL Token ID Initialization (#5307)	6 years ago
Paul Banks	a12810664f	Modify ConfigEntry bootstrapping syntax more generic (#5744) ... * Modify ConfigEntry bootstrapping syntax to be generic and compatible with other CLI config syntax. Refs #5743 * Fix gofmt issues.	6 years ago
Kyle Havlovitz	aba54cec55	Add HTTP endpoints for config entry management (#5718)	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
R.B. Boyer	c6722fc43d	Merge pull request #5617 from hashicorp/f-acl-ux ... Secure ACL Introduction for Kubernetes	6 years ago
Aestek	21a776e202	Fix: fail messages after a node rename replace the new node definition (#5520) ... When receiving a serf faild message for a node which is not in the catalog, do not perform a register request to set is serf heath to critical as it could overwrite the node information and services if it was renamed. Fixes : #5518	6 years ago
R.B. Boyer	e47d7eeddb	acl: adding support for kubernetes auth provider login (#5600) ... * auth providers * binding rules * auth provider for kubernetes * login/logout	6 years ago
R.B. Boyer	cc1aa3f973	acl: adding Roles to Tokens (#5514) ... Roles are named and can express the same bundle of permissions that can currently be assigned to a Token (lists of Policies and Service Identities). The difference with a Role is that it not itself a bearer token, but just another entity that can be tied to a Token. This lets an operator potentially curate a set of smaller reusable Policies and compose them together into reusable Roles, rather than always exploding that same list of Policies on any Token that needs similar permissions. This also refactors the acl replication code to be semi-generic to avoid 3x copypasta.	6 years ago
R.B. Boyer	7928305279	making ACLToken.ExpirationTime a *time.Time value instead of time.Time (#5663) ... This is mainly to avoid having the API return "0001-01-01T00:00:00Z" as a value for the ExpirationTime field when it is not set. Unfortunately time.Time doesn't respect the json marshalling "omitempty" directive.	6 years ago
R.B. Boyer	db43fc3a20	acl: ACL Tokens can now be assigned an optional set of service identities (#5390) ... These act like a special cased version of a Policy Template for granting a token the privileges necessary to register a service and its connect proxy, and read upstreams from the catalog.	6 years ago
R.B. Boyer	2144bd7fbd	acl: tokens can be created with an optional expiration time (#5353)	6 years ago
Matt Keeler	15e80e4e76	Implement bootstrapping proxy defaults from the config file (#5714)	6 years ago
Matt Keeler	5befe0f5d5	Implement config entry replication (#5706)	6 years ago
Matt Keeler	222afeae4c	Move the watch package into the api module (#5664) ... * Move the watch package into the api module It was already just a thin wrapper around the API anyways. The biggest change was to the testing. Instead of using a test agent directly from the agent package it now uses the binary on the PATH just like the other API tests. The other big changes were to fix up the connect based watch tests so that we didn’t need to pull in the connect package (and therefore all of Consul)	6 years ago
Alvin Huang	8ceca2ace3	Add fmt and vet (#5671) ... * add go fmt and vet * go fmt fixes	6 years ago
Kyle Havlovitz	d8f8400fe1	Merge pull request #5700 from hashicorp/service-reg-manager ... Use centralized service config on agent service registrations	6 years ago
Kyle Havlovitz	43bfc20dc8	Test an index=0 value in cache.Notify	6 years ago
Freddy	f2213f60e0	Update alias checks on local add and remove	6 years ago
Aestek	f669bb7b0f	Add support for DNS config hot-reload (#4875) ... The DNS config parameters `recursors` and `dns_config.*` are now hot reloaded on SIGHUP or `consul reload` and do not need an agent restart to be modified. Config is stored in an atomic.Value and loaded at the beginning of each request. Reloading only affects requests that start _after_ the reload. Ongoing requests are not affected. To match the current behavior the recursor handler is loaded and unloaded as needed on config reload.	6 years ago
Kyle Havlovitz	cba47aa0ca	Fix a race in the ready logic	6 years ago
Kyle Havlovitz	c269369760	Make central service config opt-in and rework the initial registration	6 years ago
Kyle Havlovitz	b58572afbd	Fix a race in the service updates	6 years ago
Kyle Havlovitz	88e1d8ce03	Fill out the service manager functionality and fix tests	6 years ago
Kyle Havlovitz	7c25869e67	Add the service registration manager to the agent	6 years ago
Kyle Havlovitz	b186c3020c	Merge pull request #5615 from hashicorp/config-entry-rpc ... Add RPC endpoints for config entry operations	6 years ago
Kyle Havlovitz	fed7595d45	Rename config entry ACL methods	6 years ago
R.B. Boyer	6269d1f130	tests: actually have TestSessionTTLRenew sleep during execution (#5669) ... Due to an unintended order of operations issue with integer division TestSessionTTLRenew was sleeping for 0s every time. Also add explicit failures for when the various session renewal returns nil unexpectedly.	6 years ago
kaitlincarter-hc	7dcc727b4d	[docs] Server Performance (#5627) ... * Moving server performance guide to docs. * fixing broken links * updating broken link * fixing broken links	6 years ago
Freddy	eebc788959	Filter non-passing nodes without modifying cache	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579) ... Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Freddy	b2c12550d7	Remove old UI, option to use it, and its build processes	6 years ago
Kyle Havlovitz	690e9dd2c0	Move the ACL logic into the ConfigEntry interface	6 years ago
Kyle Havlovitz	f2ed482680	Add RPC endpoints for config entry operations	6 years ago
Alvin Huang	f45e495e38	Merge pull request #5376 from hashicorp/fix-tests ... Fix tests in prep for CircleCI Migration	6 years ago
Kyle Havlovitz	5f569fb2ac	Merge pull request #5539 from hashicorp/service-config ... Service config state model	6 years ago
Kyle Havlovitz	a2fa9a0019	Cleaned up some error handling/comments around config entries	6 years ago
Kyle Havlovitz	c2da314eeb	Merge pull request #5553 from hashicorp/txn-check-serialization ... Use the correct check duration fields when converting transaction ops	6 years ago
Hans Hasselberg	ac45b17482	fix remaining CI failures after Go 1.12.1 Upgrade (#5576)	6 years ago
Kyle Havlovitz	d16be2e269	Encode config entry FSM messages in a generic type	6 years ago
Kyle Havlovitz	f6df5c9b3b	Clean up service config state store methods	6 years ago
R.B. Boyer	0d1b496a52	acl: memdb filter of tokens-by-policy was inverted (#5575) ... The inversion wasn't noticed because the parallel execution of TokenList tests was operating incorrectly due to variable shadowing.	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Kyle Havlovitz	716a20d8a6	Re-add logic to handle the undocumented duration fields	6 years ago
Kyle Havlovitz	3f5e20452e	http: use the correct check duration fields when converting txn ops	6 years ago
Paul Banks	89fa5ec3ba	Connect: Fix Envoy getting stuck during load (#5499) ... * Connect: Fix Envoy getting stuck during load Also in this PR: - Enabled outlier detection on upstreams which will mark instances unhealthy after 5 failures (using Envoy's defaults) - Enable weighted load balancing where DNS weights are configured * Fix empty load assignments in the right place * Fix import names from review * Move millisecond parse to a helper function	6 years ago
Kyle Havlovitz	e199c37ee4	Add some basic normalize/validation logic for config entries	6 years ago
Paul Banks	d2e68a900a	Connect: Make Connect health queries unblock correctly (#5508) ... * Make Connect health queryies unblock correctly in all cases and use optimal number of watch chans. Fixes #5506. * Node check test cases and clearer bug test doc * Comment update	6 years ago
Kyle Havlovitz	d92577c16b	Fix fsm serialization and add snapshot/restore	6 years ago
Hans Hasselberg	ea5210a30e	Release v1.4.4	6 years ago
Kyle Havlovitz	17aa6a5a34	Fill out state store/FSM functions and add tests	6 years ago
R.B. Boyer	02b2cb1d15	agent: ensure the TLS hostname verification knows about the currently configured domain (#5513)	6 years ago
Kyle Havlovitz	9d07add047	Add config types and state store table	6 years ago
Kyle Havlovitz	aa4e26d102	Condense some test logic and add a comment about renaming	6 years ago
Hans Hasselberg	e7134a0dab	agent: only use TestAgent when appropriate (#5502)	6 years ago
Paul Banks	0b5a078b95	Optimize health watching to single chan/goroutine. (#5449) ... Refs #4984. Watching chans for every node we touch in a health query is wasteful. In #4984 it shows that if there are more than 682 service instances we always fallback to watching all services which kills performance. We already have a record in MemDB that is reliably update whenever the service health result should change thanks to per-service watch indexes. So in general, provided there is at least one service instances and we actually have a service index for it (we always do now) we only ever need to watch a single channel. This saves us from ever falling back to the general index and causing the performance cliff in #4984, but it also means fewer goroutines and work done for every blocking health query. It also saves some allocations made during the query because we no longer have to populate a WatchSet with 3 chans per service instance which saves the internal map allocation. This passes all state store tests except the one that explicitly checked for the fallback behaviour we've now optimized away and in general seems safe.	6 years ago
Pierre Souchay	88d4383410	Ensure we remove Connect proxy before deregistering the service itself (#5482) ... This will fix https://github.com/hashicorp/consul/issues/5351	6 years ago
Valentin Fritz	21f149de8b	Fix checks removal when removing service (#5457) ... Fix my recently discovered issue described here: #5456	6 years ago
R.B. Boyer	cd96af4fc0	acl: reduce complexity of token resolution process with alternative singleflighting (#5480) ... acl: reduce complexity of token resolution process with alternative singleflighting Switches acl resolution to use golang.org/x/sync/singleflight. For the identity/legacy lookups this is a drop-in replacement with the same overall approach to request coalescing. For policies this is technically a change in behavior, but when considered holistically is approximately performance neutral (with the benefit of less code). There are two goals with this blob of code (speaking specifically of policy resolution here): 1) Minimize cross-DC requests. 2) Minimize client-to-server LAN requests. The previous iteration of this code was optimizing for the case of many possibly different tokens being resolved concurrently that have a significant overlap in linked policies such that deduplication would be worth the complexity. While this is laudable there are some things to consider that can help to adjust expectations: 1) For v1.4+ policies are always replicated, and once a single policy shows up in a secondary DC the replicated data is considered authoritative for requests made in that DC. This means that our earlier concerns about minimizing cross-DC requests are irrelevant because there will be no cross-DC policy reads that occur. 2) For Server nodes the in-memory ACL policy cache is capped at zero, meaning it has no caching. Only Client nodes run with a cache. This means that instead of having an entire DC's worth of tokens (what a Server might see) that can have policy resolutions coalesced these nodes will only ever be seeing node-local token resolutions. In a reasonable worst-case scenario where a scheduler like Kubernetes has "filled" a node with Connect services, even that will only schedule ~100 connect services per node. If every service has a unique token there will only be 100 tokens to coalesce and even then those requests have to occur concurrently AND be hitting an empty consul cache. Instead of seeing a great coalescing opportunity for cutting down on redundant Policy resolutions, in practice it's far more likely given node densities that you'd see requests for the same token concurrently than you would for two tokens sharing a policy concurrently (to a degree that would warrant the overhead of the current variation of singleflighting. Given that, this patch switches the Policy resolution process to only singleflight by requesting token (but keeps the cache as by-policy).	6 years ago
Kyle Havlovitz	8ae6547934	Update state store test for changing node ID	6 years ago
Kyle Havlovitz	6932b06f97	Add a test for changing a failed node's ID	6 years ago
Hans Hasselberg	7e11dd82aa	agent: enable reloading of tls config (#5419) ... This PR introduces reloading tls configuration. Consul will now be able to reload the TLS configuration which previously required a restart. It is not yet possible to turn TLS ON or OFF with these changes. Only when TLS is already turned on, the configuration can be reloaded. Most importantly the certificates and CAs.	6 years ago
R.B. Boyer	2e175be41b	acl: correctly extend the cache for acl identities during resolution (#5475)	6 years ago
Aestek	4bea29f15a	[catalog] Update the node's services indexes on update (#5458) ... Node updates were not updating the service indexes, which are used for service related queries. This caused the X-Consul-Index to stay the same after a node update as seen from a service query even though the node data is returned in heath queries. If that happened in between queries the client would miss this change. We now update the indexes of the services on the node when it is updated. Fixes: #5450	6 years ago
Kyle Havlovitz	1e4523f55e	Add logic to allow changing a failed node's ID	6 years ago
Alvin Huang	8cb8108b1b	fix typos	6 years ago
R.B. Boyer	f4a3b9d518	fix typos reported by golangci-lint:misspell (#5434)	6 years ago
R.B. Boyer	2ffbea41c8	improve flaky LANReap tests by expliciting configuring the tombstone timeout ... In TestServer_LANReap autopilot is running, so the alternate flow through the serf reaping function is possible. In that situation the ReconnectTimeout is not relevant so for parity also override the TombstoneTimeout value as well. For additional parity update the TestServer_WANReap and TestClient_LANReap versions of this test in the same way even though autopilot is irrelevant here .	6 years ago
R.B. Boyer	5bea49ecb0	tests: avoid leaking child processes from agent/proxyprocess package	6 years ago
Matt Keeler	567e41ff6b	Release v1.4.3	6 years ago
Matt Keeler	90040f8bff	Fixes for CVE-2019-8336 ... Fix error in detecting raft replication errors. Detect redacted token secrets and prevent attempting to insert. Add a Redacted field to the TokenBatchRead and TokenRead RPC endpoints This will indicate whether token secrets have been redacted. Ensure any token with a redacted secret in secondary datacenters is removed. Test that redacted tokens cannot be replicated.	6 years ago
Hans Hasselberg	d35824b1fa	default to tls 1.2 as promised. (#5340)	6 years ago
Aestek	2aac4d5168	Register and deregisters services and their checks atomically in the local state (#5012) ... Prevent race between register and deregister requests by saving them together in the local state on registration. Also adds more cleaning in case of failure when registering services / checks.	6 years ago
Matt Keeler	6e6910ea11	Dont modify memdb owned token data for get/list requests of tokens (#5412) ... Previously we were fixing up the token links directly on the *ACLToken returned by memdb. This invalidated some assumptions that a snapshot is immutable as well as potentially being able to cause a crash. The fix here is to give the policy link fixing function copy on write semantics. When no fixes are necessary we can return the memdb object directly, otherwise we copy it and create a new list of links. Eventually we might find a better way to keep those policy links in sync but for now this fixes the issue.	6 years ago
Aestek	02f991843f	Fix race condition in DNS when using cache (#5398) ... * Fix race condition in DNS when using cache The healty node filtering was modifying the result from the cache, which caused a crash when multiple queries were made to the same service simultaneously. We now copy the node slice before filtering to ensure we do not modify the data stored in the cache. * Fix wording in dns cache config doc s/dns_max_age/cache_max_age/	6 years ago
Matt Keeler	200c0fb3e9	Call RemoveServer for reap events (#5317) ... This ensures that servers are removed from RPC routing when they are reaped.	6 years ago
R.B. Boyer	409c901f8e	test: fix concurrent map access when setting up test vault	6 years ago
R.B. Boyer	6955186239	fix ignored errors in state store internals as reported by errcheck	6 years ago
R.B. Boyer	c7067645dd	fix a few leap-year related clock math inaccuracies and failing tests	6 years ago
Matt Keeler	118adbb123	ACL Token Persistence and Reloading (#5328) ... This PR adds two features which will be useful for operators when ACLs are in use. 1. Tokens set in configuration files are now reloadable. 2. If `acl.enable_token_persistence` is set to `true` in the configuration, tokens set via the `v1/agent/token` endpoint are now persisted to disk and loaded when the agent starts (or during configuration reload) Note that token persistence is opt-in so our users who do not want tokens on the local disk will see no change. Some other secondary changes: * Refactored a bunch of places where the replication token is retrieved from the token store. This token isn't just for replicating ACLs and now it is named accordingly. * Allowed better paths in the `v1/agent/token/` API. Instead of paths like: `v1/agent/token/acl_replication_token` the path can now be just `v1/agent/token/replication`. The old paths remain to be valid. * Added a couple new API functions to set tokens via the new paths. Deprecated the old ones and pointed to the new names. The names are also generally better and don't imply that what you are setting is for ACLs but rather are setting ACL tokens. There is a minor semantic difference there especially for the replication token as again, its no longer used only for ACL token/policy replication. The new functions will detect 404s and fallback to using the older token paths when talking to pre-1.4.3 agents. * Docs updated to reflect the API additions and to show using the new endpoints. * Updated the ACL CLI set-agent-tokens command to use the non-deprecated APIs.	6 years ago
Kyle Havlovitz	f07e928afc	Merge pull request #5325 from hashicorp/consul-ca-panic ... connect/ca: fix a potential panic in the Consul provider	6 years ago
Hans Hasselberg	80e7d63fc2	Centralise tls configuration part 2 (#5374) ... This PR is based on #5366 and continues to centralise the tls configuration in order to be reloadable eventually! This PR is another refactoring. No tests are changed, beyond calling other functions or cosmetic stuff. I added a bunch of tests, even though they might be redundant.	6 years ago
Hans Hasselberg	786b3b1095	Centralise tls configuration part 1 (#5366) ... In order to be able to reload the TLS configuration, we need one way to generate the different configurations. This PR introduces a `tlsutil.Configurator` which holds a `tlsutil.Config`. Afterwards it is responsible for rendering every `tls.Config`. In this particular PR I moved `IncomingHTTPSConfig`, `IncomingTLSConfig`, and `OutgoingTLSWrapper` into `tlsutil.Configurator`. This PR is a pure refactoring - not a single feature added. And not a single test added. I only slightly modified existing tests as necessary.	6 years ago
Aestek	f1cdfbe40e	Allow DNS interface to use agent cache (#5300) ... Adds two new configuration parameters "dns_config.use_cache" and "dns_config.cache_max_age" controlling how DNS requests use the agent cache when querying servers.	6 years ago
Alvin Huang	77eecf1046	add wait to TestClient_JoinLAN	6 years ago
Alvin Huang	136df63e2c	add retry to TestResetSessionTimerLocked	6 years ago
Alvin Huang	a7180f715a	add serf check to testDNSServiceLookupResponseLimits, checkDNSService	6 years ago
Alvin Huang	d10b5a396b	add wait to TestOperator_AutopilotCASConfiguration	6 years ago
Alvin Huang	dc200daf21	add wait to TestSnapshot	6 years ago
Alvin Huang	c2a19e5090	add wait to TestAgent_RPCPing	6 years ago
Alvin Huang	c23eb91262	fix TestAgent_CheckCriticalTime and better error output	6 years ago
Alvin Huang	6c9b516a29	skip TestCheckTCPPassing on CircleCI	6 years ago
R.B. Boyer	c2a30c5fdd	fix incorrect body of TestACLEndpoint_PolicyBatchRead ... Lifted from PR #5307 as it was an unrelated drive-by fix on that PR anyway. s/token/policy/	6 years ago
R.B. Boyer	b569f222f9	update agent/agent_endpoint_test.go to use V2 tokens with attached policies	6 years ago
Nicholas Jackson	99fe9dabce	Envoy config cluster (#5308) ... * Start adding tests for cluster override * Refactor tests for clusters * Passing tests for custom upstream cluster override * Added capability to customise local app cluster * Rename config for local cluster override	6 years ago
Kainoa Seto	b2af8862c7	Deferred updating response meta with consul headers (#5355)	6 years ago
R.B. Boyer	ef8258cd4e	test: switch test file from assert -> require for consistency ... Also in acl_endpoint_test.go: * convert logical blocks in some token tests to subtests * remove use of require.New This removes a lot of noise in a later PR.	6 years ago
Matt Keeler	766d771017	Pass a testing.T into NewTestAgent and TestAgent.Start (#5342) ... This way we can avoid unnecessary panics which cause other tests not to run. This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent	6 years ago
R.B. Boyer	adbe8ed370	correct some typos	6 years ago
R.B. Boyer	88bb53d001	ensure that we plumb our configured logger into all parts of the raft library	6 years ago
R.B. Boyer	2c983902be	reduce the local scope of variable	6 years ago
R.B. Boyer	de0f585583	agent: only enable TLS on gRPC if the HTTPS API port is enabled (#5287) ... Currently the gRPC server assumes that if you have configured TLS certs on the agent (for RPC) that you want gRPC to be encrypted. If gRPC is bound to localhost this can be overkill. For the API we let the user choose to offer HTTP or HTTPS API endpoints independently of the TLS cert configuration for a similar reason. This setting will let someone encrypt RPC traffic with TLS but avoid encrypting local gRPC traffic if that is what they want to do by only enabling TLS on gRPC if the HTTPS API port is enabled.	6 years ago
R.B. Boyer	f2ed3a3777	clarify the ACL.PolicyDelete endpoint (#5337) ... There was an errant early-return in PolicyDelete() that bypassed the rest of the function. This was ok because the only caller of this function ignores the results. This removes the early-return making it structurally behave like TokenDelete() and for both PolicyDelete and TokenDelete clarify the lone callers to indicate that the return values are ignored. We may wish to avoid the entire return value as well, but this patch doesn't go that far.	6 years ago
R.B. Boyer	324ba5df17	update TestStateStore_ACLBootstrap to not rely upon request mutation (#5335)	6 years ago
Matt Keeler	7073ba4ed2	Move autopilot initialization to prevent race (#5322) ... `establishLeadership` invoked during leadership monitoring may use autopilot to do promotions etc. There was a race with doing that and having autopilot initialized and this fixes it.	6 years ago
Kyle Havlovitz	29e4c17b07	connect/ca: fix a potential panic in the Consul provider	6 years ago
Matt Keeler	acfd87c673	Improve Connect with Prepared Queries (#5291) ... Given a query like: ``` { "Name": "tagged-connect-query", "Service": { "Service": "foo", "Tags": ["tag"], "Connect": true } } ``` And a Consul configuration like: ``` { "services": [ "name": "foo", "port": 8080, "connect": { "sidecar_service": {} }, "tags": ["tag"] ] } ``` If you executed the query it would always turn up with 0 results. This was because the sidecar service was being created without any tags. You could instead make your config look like: ``` { "services": [ "name": "foo", "port": 8080, "connect": { "sidecar_service": { "tags": ["tag"] } }, "tags": ["tag"] ] } ``` However that is a bit redundant for most cases. This PR ensures that the tags and service meta of the parent service get copied to the sidecar service. If there are any tags or service meta set in the sidecar service definition then this copying does not take place. After the changes, the query will now return the expected results. A second change was made to prepared queries in this PR which is to allow filtering on ServiceMeta just like we allow for filtering on NodeMeta.	6 years ago
R.B. Boyer	e1e4249e90	testutil: redirect some test agent logs to testing.T.Logf (#5304) ... When tests fail, only the logs for the failing run are dumped to the console which helps in diagnosis. This is easily added to other test scenarios as they come up.	6 years ago
R.B. Boyer	db8a871309	Merge pull request #5237 from hashicorp/term-grpc-stream-on-token-failure ... Check ACLs more often for xDS endpoints.	6 years ago
mkeeler	c97c712e96	Release v1.4.2	6 years ago
Kyle Havlovitz	7118f42950	Fix failing TestAgent_PurgeCheckOnDuplicate after merge	6 years ago
Matt Keeler	1736e24fb3	Don't generate TXT records just to discard them (#5272) ... * Don't generate TXT records just to discard them * Add unit test for formatNodeRecord to ensure it prevents returning TXT records	6 years ago
Kyle Havlovitz	928b7ec60d	Merge branch 'healthcheck-duration-fix'	6 years ago
Kyle Havlovitz	1a4978fb94	Re-add ReadableDuration types to health check definition ... This is to fix the backwards-incompatible change made in 1.4.1 by changing these fields to time.Duration.	6 years ago
R.B. Boyer	e9a2eab316	speed up TestHTTPAPI_MethodNotAllowed_OSS from 11s -> 0.5s (#5268)	6 years ago
Hans Hasselberg	552e150536	correct name	6 years ago
Hans Hasselberg	aebb50d47d	simpler fix	6 years ago
Hans Hasselberg	5db185a7e4	do not export that type	6 years ago
Hans Hasselberg	7f44100101	fix marshalling	6 years ago
Hans Hasselberg	d4790b2827	demo nomad problem	6 years ago
banks	65d2c9b51d	Release v1.4.1	6 years ago
Matt Keeler	d5a3ba6cda	Disregard rules when set on a management token (#5261) ... * Disregard rules when set on a management token * Add unit test for legacy mgmt token with rules	6 years ago
Kyle Havlovitz	88c044759f	connect: Forward intention RPCs if this isn't the primary	6 years ago
Kyle Havlovitz	6b28434f8a	Merge pull request #5249 from hashicorp/ca-fixes-oss ... Minor CA fixes	6 years ago
Kyle Havlovitz	5bdf130767	Merge pull request #4869 from hashicorp/txn-checks ... Add node/service/check operations to transaction api	6 years ago
Kyle Havlovitz	a28ba4687d	connect/ca: return a better error message if the CA isn't fully initialized when signing	6 years ago
Matt Keeler	579a8b32ed	Fix several ACL token/policy resolution issues. (#5246) ... * Fix 2 remote ACL policy resolution issues 1 - Use the right method to fire async not found errors when the ACL.PolicyResolve RPC returns that error. This was previously accidentally firing a token result instead of a policy result which would have effectively done nothing (unless there happened to be a token with a secret id == the policy id being resolved. 2. When concurrent policy resolution is being done we single flight the requests. The bug before was that for the policy resolution that was going to piggy back on anothers RPC results it wasn’t waiting long enough for the results to come back due to looping with the wrong variable. * Fix a handful of other edge case ACL scenarios The main issue was that token specific issues (not able to access a particular policy or the token being deleted after initial fetching) were poisoning the policy cache. A second issue was that for concurrent token resolutions, the first resolution to get started would go fetch all the policies. If before the policies were retrieved a second resolution request came in, the new request would register watchers for those policies but then never block waiting for them to complete. This resulted in using the default policy when it shouldn't have.	6 years ago
Paul Banks	ef9f27cbc8	connect: tame thundering herd of CSRs on CA rotation (#5228) ... * Support rate limiting and concurrency limiting CSR requests on servers; handle CA rotations gracefully with jitter and backoff-on-rate-limit in client * Add CSR rate limiting docs * Fix config naming and add tests for new CA configs	6 years ago
R.B. Boyer	d3eb781384	Check ACLs more often for xDS endpoints. ... For established xDS gRPC streams recheck ACLs for each DiscoveryRequest or DiscoveryResponse. If more than 5 minutes has elapsed since the last ACL check, recheck even without an incoming DiscoveryRequest or DiscoveryResponse. ACL failures will terminate the stream.	6 years ago
Kyle Havlovitz	ddc4a8d848	oss: add the enterprise server stub for intention replication check	6 years ago
R.B. Boyer	2dea3e2bd7	Fix some test typos.	6 years ago
Matt Keeler	7e6b3e6a0c	Implement prepared query upstreams watching for envoy (#5224) ... Fixes #4969 This implements non-blocking request polling at the cache layer which is currently only used for prepared queries. Additionally this enables the proxycfg manager to poll prepared queries for use in envoy proxy upstreams.	6 years ago
Kyle Havlovitz	21380021af	txn: update existing txn api docs with new operations	6 years ago
Matt Keeler	1ec5f2a27f	Store leaf cert indexes in raft and use for the ModifyIndex on the returned certs (#5211) ... * Store leaf cert indexes in raft and use for the ModifyIndex on the returned certs This ensures that future certificate signings will have a strictly greater ModifyIndex than any previous certs signed.	6 years ago
Aestek	4afbe792df	Improve blocking queries on services that do not exist (#4810) ... ## Background When making a blocking query on a missing service (was never registered, or is not registered anymore) the query returns as soon as any service is updated. On clusters with frequent updates (5~10 updates/s in our DCs) these queries virtually do not block, and clients with no protections againt this waste ressources on the agent and server side. Clients that do protect against this get updates later than they should because of the backoff time they implement between requests. ## Implementation While reducing the number of unnecessary updates we still want : * Clients to be notified as soon as when the last instance of a service disapears. * Clients to be notified whenever there's there is an update for the service. * Clients to be notified as soon as the first instance of the requested service is added. To reduce the number of unnecessary updates we need to block when a request to a missing service is made. However in the following case : 1. Client `client1` makes a query for service `foo`, gets back a node and X-Consul-Index 42 2. `foo` is unregistered 3. `client1`  makes a query for `foo` with `index=42` -> `foo` does not exist, the query blocks and `client1` is not notified of the change on `foo` We could store the last raft index when each service was last alive to know wether we should block on the incoming query or not, but that list could grow indefinetly. We instead store the last raft index when a service was unregistered and use it when a query targets a service that does not exist. When a service `srv` is unregistered this "missing service index" is always greater than any X-Consul-Index held by the clients while `srv` was up, allowing us to immediatly notify them. 1. Client `client1` makes a query for service `foo`, gets back a node and `X-Consul-Index: 42` 2. `foo` is unregistered, we set the "missing service index" to 43 3. `client1` makes a blocking query for `foo` with `index=42` -> `foo` does not exist, we check against the "missing service index" and return immediatly with `X-Consul-Index: 43` 4. `client1` makes a blocking query for `foo` with `index=43` -> we block 5. Other changes happen in the cluster, but foo still doesn't exist and "missing service index" hasn't changed, the query is still blocked 6. `foo` is registered again on index 62 -> `foo` exists and its index is greater than 43, we unblock the query	6 years ago
Matt Keeler	baa8946ea6	cache: Pass through wait query param to the cache.Get (#5203) ... This adds a MaxQueryTime field to the connect ca leaf cache request type and populates it via the wait query param. The cache will then do the right thing and timeout the operation as expected if no new leaf cert is available within that time. Fixes #4462 The reproduction scenario in the original issue now times out appropriately.	6 years ago
Aestek	c043de5381	[Security] Allow blocking Write endpoints on Agent using Network Addresses (#4719) ... * Add -write-allowed-nets option * Add documentation for the new write_allowed_nets option	6 years ago
Matt Keeler	1048f3d5e7	acl: Prevent tokens from deleting themselves (#5210) ... Fixes #4897 Also apparently token deletion could segfault in secondary DCs when attempting to delete non-existant tokens. For that reason both checks are wrapped within the non-nil check.	6 years ago
Paul Banks	0638e09b6e	connect: agent leaf cert caching improvements (#5091) ... * Add State storage and LastResult argument into Cache so that cache.Types can safely store additional data that is eventually expired. * New Leaf cache type working and basic tests passing. TODO: more extensive testing for the Root change jitter across blocking requests, test concurrent fetches for different leaves interact nicely with rootsWatcher. * Add multi-client and delayed rotation tests. * Typos and cleanup error handling in roots watch * Add comment about how the FetchResult can be used and change ca leaf state to use a non-pointer state. * Plumb test override of root CA jitter through TestAgent so that tests are deterministic again! * Fix failing config test	6 years ago
Kyle Havlovitz	c07c5446a8	txn: clean up some state store/acl code	6 years ago
Hans Hasselberg	067027230b	connect: add tls config for vault connect ca provider (#5125) ... * add tlsconfig for vault connect ca provider. * add options to the docs * add tests for new configuration	6 years ago
Alejandro Guirao Rodríguez	9f33353c14	agent/config: Fix typo in comment (#5202)	6 years ago
Paul Banks	bb7145f27d	agent: add default weights to service in local state to prevent AE churn (#5126) ... * Add default weights when adding a service with no weights to local state to prevent constant AE re-sync. This fix was contributed by @42wim in https://github.com/hashicorp/consul/pull/5096 but was merged against the wrong base. This adds it to master and adds a test to cover the behaviour. * Fix tests that broke due to comparing internal state which now has default weights	6 years ago
Paul Banks	0589525ae9	agent: Don't leave old errors around in cache (#5094) ... * Fixes #4480. Don't leave old errors around in cache that can be hit in specific circumstances. * Move error reset to cover extreme edge case of nil Value, nil err Fetch	6 years ago
Pierre Souchay	ae7f88f995	Avoid to have infinite recursion in DNS lookups when resolving CNAMEs (#4918) ... * Avoid to have infinite recursion in DNS lookups when resolving CNAMEs This will avoid killing Consul when a Service.Address is using CNAME to a Consul CNAME that creates an infinite recursion. This will fix https://github.com/hashicorp/consul/issues/4907 * Use maxRecursionLevel = 3 to allow several recursions	6 years ago
Paul Banks	b29bc906ee	bugfix: use ServiceTags to generate cache key hash (#4987) ... * bugfix: use ServiceTags to generate cahce key hash * update unit test * update * remote print log * Update .gitignore * Completely deprecate ServiceTag field internally for clarity * Add explicit test for CacheInfo cases	6 years ago
Aestek	8709213d6e	Prevent status flap when re-registering a check (#4904) ... Fixes point `#2` of: https://github.com/hashicorp/consul/issues/4903 When registering a service each healthcheck status is saved and restored (https://github.com/hashicorp/consul/blob/master/agent/agent.go#L1914) to avoid unnecessary flaps in health state. This change extends this feature to single check registration by moving this protection in `AddCheck()` so that both `PUT /v1/agent/service/register` and `PUT /v1/agent/check/register` behave in the same idempotent way. #### Steps to reproduce 1. Register a check : ``` curl -X PUT \ http://127.0.0.1:8500/v1/agent/check/register \ -H 'Content-Type: application/json' \ -d '{ "Name": "my_check", "ServiceID": "srv", "Interval": "10s", "Args": ["true"] }' ``` 2. The check will initialize and change to `passing` 3. Run the same request again 4. The check status will quickly go from `critical` to `passing` (the delay for this transission is determined by https://github.com/hashicorp/consul/blob/master/agent/checks/check.go#L95)	6 years ago
Mitchell Hashimoto	f76022fa63	CA Provider Plugins (#4751) ... This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This **does not** wire this up in any way to Consul itself, so this will not enable using these plugins yet. ## Why? We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security. ### go-plugin The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself. The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin. `go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature. ## Implementation Details ### `map[string]interface{}` The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software. Note that this is just the transport protocol, the end software views it fully decoded. ### `x509.Certificate` and `CertificateRequest` We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire. ### Testing `go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC. ### Vendoring This PR doesn't introduce vendoring for two reasons: 1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict. 2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors). ## Next Steps With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.	6 years ago
Grégoire Seux	4f62a3b528	Implement /v1/agent/health/service/<service name> endpoint (#3551) ... This endpoint aggregates all checks related to <service id> on the agent and return an appropriate http code + the string describing the worst check. This allows to cleanly expose service status to other component, hiding complexity of multiple checks. This is especially useful to use consul to feed a load balancer which would delegate health checking to consul agent. Exposing this endpoint on the agent is necessary to avoid a hit on consul servers and avoid decreasing resiliency (this endpoint will work even if there is no consul leader in the cluster).	6 years ago
Aestek	5960974db1	[Fix] Services sometimes not being synced with acl_enforce_version_8 = false (#4771) ... Fixes: https://github.com/hashicorp/consul/issues/3676 This fixes a bug were registering an agent with a non-existent ACL token can prevent other services registered with a good token from being synced to the server when using `acl_enforce_version_8 = false`. ## Background When `acl_enforce_version_8` is off the agent does not check the ACL token validity before storing the service in its state. When syncing a service registered with a missing ACL token we fall into the default error handling case (https://github.com/hashicorp/consul/blob/master/agent/local/state.go#L1255) and stop the sync (https://github.com/hashicorp/consul/blob/master/agent/local/state.go#L1082) without setting its Synced property to true like in the permission denied case. This means that the sync will always stop at the faulty service(s). The order in which the services are synced is random since we iterate on a map. So eventually all services with good ACL tokens will be synced, this can however take some time and is influenced by the cluster size, the bigger the slower because retries are less frequent. Having a service in this state also prevent all further sync of checks as they are done after the services. ## Changes This change modify the sync process to continue even if there is an error. This fixes the issue described above as well as making the sync more error tolerant: if the server repeatedly refuses a service (the ACL token could have been deleted by the time the service is synced, the servers were upgraded to a newer version that has more strict checks on the service definition...). Then all services and check that can be synced will, and those that don't will be marked as errors in the logs instead of blocking the whole process.	6 years ago
Hans Hasselberg	0b4a879203	ui: serve /robots.txt when UI is enabled. (#5089) ... * serve /robots.txt * robots.txt: disallow everything	6 years ago
Kyle Havlovitz	995e728ea0	txn: fix an issue with querying nodes by name instead of ID	6 years ago
Pierre Souchay	f4dc8b42e0	[Travis][UnstableTests] Fixed unstable tests in travis (#5013) ... * [Travis][UnstableTests] Fixed unstable tests in travis as seen in https://travis-ci.org/hashicorp/consul/jobs/460824602 * Fixed unstable tests in https://travis-ci.org/hashicorp/consul/jobs/460857687	6 years ago
Kyle Havlovitz	67bac7a815	api: add support for new txn operations	6 years ago
Kyle Havlovitz	de4dbf583e	txn: add tests for RPC endpoint	6 years ago
Kyle Havlovitz	6a512e5c0f	txn: add ACL enforcement/validation to new txn ops	6 years ago
Kyle Havlovitz	9467067432	state: add tests for new txn ops	6 years ago
Kyle Havlovitz	7759e9ea8b	txn: add service operations	6 years ago
Kyle Havlovitz	ab58986ac3	txn: add node operations	6 years ago
Kyle Havlovitz	01e1b5b1df	txn: add pre-check operations to txn endpoint	6 years ago
Kyle Havlovitz	b371ea8783	Add check operations to transaction api	6 years ago
Kyle Havlovitz	c7e0d3b919	Merge pull request #5061 from hashicorp/blank-ca-fix ... connect/ca: prevent blank CA config in snapshot	6 years ago
Kyle Havlovitz	4f2715d4e2	connect/ca: prevent blank CA config in snapshot ... This PR both prevents a blank CA config from being written out to a snapshot and allows Consul to gracefully recover from a snapshot with an invalid CA config. Fixes #4954.	6 years ago
Jack Pearkes	b64e8b262f	Documentation and changes for `verify_server_hostname` (#5069) ... * verify_server_hostname implies verify_outgoing * mention CVE in the docs.	6 years ago
R.B. Boyer	c1eccfd1db	agent: remove some stray fmt.Print* calls (#5015)	6 years ago
Pierre Souchay	c5ae9caa28	Fixed another list of unstable unit tests in travis (#4915) ... * Fixed another list of unstable unit tests in travis Fixed failing tests in https://travis-ci.org/hashicorp/consul/jobs/451357061 * Fixed another list of unstable unit tests in travis. Fixed failing tests in https://travis-ci.org/hashicorp/consul/jobs/451357061	6 years ago
banks	0bddfa23a2	Release v1.4.0	6 years ago
Kyle Havlovitz	76f102a1e0	Merge pull request #4952 from hashicorp/test-version ... tests: Bump test server version to 1.4.0	6 years ago
R.B. Boyer	934fae659f	acl: add stub hooks to support some plumbing in enterprise (#4951)	6 years ago
Kyle Havlovitz	269354c61d	oss: bump test server version to 1.4.0	6 years ago
Aestek	4942e66440	Fix catalog tag filter backward compat (#4944) ... Fix catalog service node filtering (ex /v1/catalog/service/srv?tag=tag1) between agent version <=v1.2.3 and server >=v1.3.0. New server version did not account for the old field when filtering hence request made from old agent were not tag-filtered.	6 years ago
Jack Pearkes	a90c29e60d	Doc changes for 1.4 Final (#4870) ... * website: add multi-dc enterprise landing page * website: switch all 1.4.0 alerts/RC warnings * website: connect product wording Co-Authored-By: pearkes <jackpearkes@gmail.com> * website: remove RC notification * commmand/acl: fix usage docs for ACL tokens * agent: remove comment, OperatorRead * website: improve multi-dc docs Still not happy with this but tried to make it slightly more informative. * website: put back acl guide warning for 1.4.0 * website: simplify multi-dc page and respond to feedback * Fix Multi-DC typos on connect index page. * Improve Multi-DC overview. A full guide is a WIP and will be added post-release. * Fixes typo avaiable > available	6 years ago
Paul Banks	54c2ff6aca	connect: remove additional trust-domain validation (#4934) ... * connct: Remove additional trust-domain validation * Comment typos * Update connect_ca.go	6 years ago
Kyle Havlovitz	4a73a59d70	Merge pull request #4917 from hashicorp/replication-token-cleanup ... Use acl replication_token for connect	6 years ago
Kyle Havlovitz	972177071d	update non-voting server test to fix enterprise diff	6 years ago
Kyle Havlovitz	643bd13aed	oss: do a proper check-and-set on the CA roots/config fsm operation	6 years ago
R.B. Boyer	e30cc73b1d	Update agent tests to wait a bit longer for the /v1/agent/self endpoint (#4937)	6 years ago
R.B. Boyer	2afc2a3c3b	acl: fixes ACL replication for legacy tokens without AccessorIDs (#4885)	6 years ago
Kyle Havlovitz	e8dd89359a	agent: fix formatting	6 years ago
Kyle Havlovitz	62691ebc82	config: remote connect replication_token	6 years ago
R.B. Boyer	9211d2701d	fix comment typos (#4890)	6 years ago
Kyle Havlovitz	8337e3d8c0	Merge pull request #4872 from hashicorp/node-snapshot-fix ... Node ID/datacenter snapshot fix	6 years ago
Matt Keeler	db2cf01406	Adds documentation for the new ACL APIs (#4851) ... * Update the ACL API docs * Add a CreateTime to the anon token Also require acl:read permissions at least to perform rule translation. Don’t want someone DoSing the system with an open endpoint that actually does a bit of work. * Fix one place where I was referring to id instead of AccessorID * Add godocs for the API package additions. * Minor updates: removed some extra commas and updated the acl intro paragraph * minor tweaks * Updated the language to be clearer * Updated the language to be clearer for policy page * I was also confused by that! Your updates are much clearer. Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Sounds much better. Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Updated sidebar layout and deprecated warning	6 years ago
Matt Keeler	f9cf0eb36e	Remaining ACL Unit Tests (#4852) ... * Add leader token upgrade test and fix various ACL enablement bugs * Update the leader ACL initialization tests. * Add a StateStore ACL tests for ACLTokenSet and ACLTokenGetBy* functions * Advertise the agents acl support status with the agent/self endpoint. * Make batch token upsert CAS’able to prevent consistency issues with token auto-upgrade * Finish up the ACL state store token tests * Finish the ACL state store unit tests Also rename some things to make them more consistent. * Do as much ACL replication testing as I can.	6 years ago
Kyle Havlovitz	bd6d0e598f	fsm: update snapshot/restore test to include ID and datacenter	6 years ago
Kyle Havlovitz	6483356329	fsm: add missing ID/datacenter to persistNodes	6 years ago
Matt Keeler	d238cb181c	New ACL API Tests (#4848) ... * A few API mods and unit tests. * Update the unit tests to verify query/write metadata and to fix the rules endpoint tests. * Make sure the full information for the replication status is in the api packge	6 years ago
Matt Keeler	790cf90ee5	Fix the NonVoter Bootstrap test (#4786)	6 years ago
banks	1757fbc0aa	Release v1.4.0-rc1	6 years ago
Kyle Havlovitz	819566f6b7	fsm: add Intention operations to transactions for internal use	6 years ago
Matt Keeler	34b53e7099	A few misc fixes found by go vet	6 years ago