1825 Commits (2587a32688ff05e88b6ef211816d008c1c4a47ed)

Author	SHA1	Message	Date
Freddy	c24c73bc59	Do not trigger update check when in dev mode	6 years ago
Paul Banks	0cfb6051ea	Add integration test for central config; fix central config WIP (#5752) ... * Add integration test for central config; fix central config WIP * Add integration test for central config; fix central config WIP * Set proxy protocol correctly and begin adding upstream support * Add upstreams to service config cache key and start new notify watcher if they change. This doesn't update the tests to pass though. * Fix some merging logic get things working manually with a hack (TODO fix properly) * Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway * Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does * Fix up service manageer and API test failures * Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally * Remove version.go hack - will make integration test fail until release * Remove unused code from commands and upstream merge * Re-bump version to 1.5.0	6 years ago
R.B. Boyer	c649243f7c	docs: add documentation for all secure acl introduction work (#5640)	6 years ago
Matt Keeler	3145bf5230	Centralized Config CLI (#5731) ... * Add HTTP endpoints for config entry management * Finish implementing decoding in the HTTP Config entry apply endpoint * Add CAS operation to the config entry apply endpoint Also use this for the bootstrapping and move the config entry decoding function into the structs package. * First pass at the API client for the config entries * Fixup some of the ConfigEntry APIs Return a singular response object instead of a list for the ConfigEntry.Get RPC. This gets plumbed through the HTTP API as well. Dont return QueryMeta in the JSON response for the config entry listing HTTP API. Instead just return a list of config entries. * Minor API client fixes * Attempt at some ConfigEntry api client tests These don’t currently work due to weak typing in JSON * Get some of the api client tests passing * Implement reflectwalk magic to correct JSON encoding a ProxyConfigEntry Also added a test for the HTTP endpoint that exposes the problem. However, since the test doesn’t actually do the JSON encode/decode its still failing. * Move MapWalk magic into a binary marshaller instead of JSON. * Add a MapWalk test * Get rid of unused func * Get rid of unused imports * Fixup some tests now that the decoding from msgpack coerces things into json compat types * Stub out most of the central config cli Fully implement the config read command. * Basic config delete command implementation * Implement config write command * Implement config list subcommand Not entirely sure about the output here. Its basically the read output indented with a line specifying the kind/name of each type which is also duplicated in the indented output. * Update command usage * Update some help usage formatting * Add the connect enable helper cli command * Update list command output * Rename the config entry API client methods. * Use renamed apis * Implement config write tests Stub the others with the noTabs tests. * Change list output format Now just simply output 1 line per named config * Add config read tests * Add invalid args write test. * Add config delete tests * Add config list tests * Add connect enable tests * Update some CLI commands to use CAS ops This also modifies the HTTP API for a write op to return a boolean indicating whether the value was written or not. * Fix up the HTTP API CAS tests as I realized they weren’t testing what they should. * Update config entry rpc tests to properly test CAS * Fix up a few more tests * Fix some tests that using ConfigEntries.Apply * Update config_write_test.go * Get rid of unused import	6 years ago
Matt Keeler	4daa1585b0	ACL Token ID Initialization (#5307)	6 years ago
R.B. Boyer	b4727fbe18	cli: actually allow the 'connect envoy' and 'watch' subcommands to work with -token-file (#5733)	6 years ago
Matt Keeler	f67e12eb6f	Enabling "service" watch handler to accept a slice of tags ... Originally from PR #5347	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
R.B. Boyer	e47d7eeddb	acl: adding support for kubernetes auth provider login (#5600) ... * auth providers * binding rules * auth provider for kubernetes * login/logout	6 years ago
R.B. Boyer	cc1aa3f973	acl: adding Roles to Tokens (#5514) ... Roles are named and can express the same bundle of permissions that can currently be assigned to a Token (lists of Policies and Service Identities). The difference with a Role is that it not itself a bearer token, but just another entity that can be tied to a Token. This lets an operator potentially curate a set of smaller reusable Policies and compose them together into reusable Roles, rather than always exploding that same list of Policies on any Token that needs similar permissions. This also refactors the acl replication code to be semi-generic to avoid 3x copypasta.	6 years ago
R.B. Boyer	7928305279	making ACLToken.ExpirationTime a *time.Time value instead of time.Time (#5663) ... This is mainly to avoid having the API return "0001-01-01T00:00:00Z" as a value for the ExpirationTime field when it is not set. Unfortunately time.Time doesn't respect the json marshalling "omitempty" directive.	6 years ago
R.B. Boyer	db43fc3a20	acl: ACL Tokens can now be assigned an optional set of service identities (#5390) ... These act like a special cased version of a Policy Template for granting a token the privileges necessary to register a service and its connect proxy, and read upstreams from the catalog.	6 years ago
R.B. Boyer	2144bd7fbd	acl: tokens can be created with an optional expiration time (#5353)	6 years ago
Matt Keeler	222afeae4c	Move the watch package into the api module (#5664) ... * Move the watch package into the api module It was already just a thin wrapper around the API anyways. The biggest change was to the testing. Instead of using a test agent directly from the agent package it now uses the binary on the PATH just like the other API tests. The other big changes were to fix up the connect based watch tests so that we didn’t need to pull in the connect package (and therefore all of Consul)	6 years ago
Alvin Huang	8ceca2ace3	Add fmt and vet (#5671) ... * add go fmt and vet * go fmt fixes	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579) ... Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Matt Keeler	f88d1ccc36	Handle rules translation when coming from the JSON compat HCL (#5662) ... We were not handling some object keys when they were strings instead of identifiers. Now both are handled. Fixes #5493	6 years ago
Alvin Huang	f45e495e38	Merge pull request #5376 from hashicorp/fix-tests ... Fix tests in prep for CircleCI Migration	6 years ago
tristan-weil	fe938e6e28	tls: allow to add ip addresses as Subject Alternative Names when creating certificates (#5602)	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago
Alvin Huang	8cb8108b1b	fix typos	6 years ago
R.B. Boyer	f4a3b9d518	fix typos reported by golangci-lint:misspell (#5434)	6 years ago
R.B. Boyer	c7067645dd	fix a few leap-year related clock math inaccuracies and failing tests	6 years ago
Matt Keeler	118adbb123	ACL Token Persistence and Reloading (#5328) ... This PR adds two features which will be useful for operators when ACLs are in use. 1. Tokens set in configuration files are now reloadable. 2. If `acl.enable_token_persistence` is set to `true` in the configuration, tokens set via the `v1/agent/token` endpoint are now persisted to disk and loaded when the agent starts (or during configuration reload) Note that token persistence is opt-in so our users who do not want tokens on the local disk will see no change. Some other secondary changes: * Refactored a bunch of places where the replication token is retrieved from the token store. This token isn't just for replicating ACLs and now it is named accordingly. * Allowed better paths in the `v1/agent/token/` API. Instead of paths like: `v1/agent/token/acl_replication_token` the path can now be just `v1/agent/token/replication`. The old paths remain to be valid. * Added a couple new API functions to set tokens via the new paths. Deprecated the old ones and pointed to the new names. The names are also generally better and don't imply that what you are setting is for ACLs but rather are setting ACL tokens. There is a minor semantic difference there especially for the replication token as again, its no longer used only for ACL token/policy replication. The new functions will detect 404s and fallback to using the older token paths when talking to pre-1.4.3 agents. * Docs updated to reflect the API additions and to show using the new endpoints. * Updated the ACL CLI set-agent-tokens command to use the non-deprecated APIs.	6 years ago
Alvin Huang	31af6be26a	add serf check to TestLockCommand	6 years ago
Matt Keeler	766d771017	Pass a testing.T into NewTestAgent and TestAgent.Start (#5342) ... This way we can avoid unnecessary panics which cause other tests not to run. This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent	6 years ago
R.B. Boyer	adbe8ed370	correct some typos	6 years ago
R.B. Boyer	de50bc3295	cli: fix typo in help text for 'consul acl role read' (#5311)	6 years ago
R.B. Boyer	5165874318	incorrect examples for 'consul acl policy' commands (#5303)	6 years ago
Matt Keeler	9715e1a08e	Basic TLS Command Tests (#5259) ... * Add tls ca create tests * Add a basic tls cert create test	6 years ago
Matt Keeler	1f2d1d4f75	Fix typo that prevented using the default ca domain for tls cert creation (#5258)	6 years ago
Hans Hasselberg	174099593a	agent: display messages from serf in cli (#5236) ... * display messages from serf in cli	6 years ago
Grégoire Seux	4f62a3b528	Implement /v1/agent/health/service/<service name> endpoint (#3551) ... This endpoint aggregates all checks related to <service id> on the agent and return an appropriate http code + the string describing the worst check. This allows to cleanly expose service status to other component, hiding complexity of multiple checks. This is especially useful to use consul to feed a load balancer which would delegate health checking to consul agent. Exposing this endpoint on the agent is necessary to avoid a hit on consul servers and avoid decreasing resiliency (this endpoint will work even if there is no consul leader in the cluster).	6 years ago
Boris Popovschi	b4eca8fcd7	Fixed gziping function for debug archive (#5184)	6 years ago
Hans Hasselberg	acc458d7a4	Builtin tls helper (#5078) ... * command: add tls subcommand * website: update docs and guide	6 years ago
Jack Pearkes	a90c29e60d	Doc changes for 1.4 Final (#4870) ... * website: add multi-dc enterprise landing page * website: switch all 1.4.0 alerts/RC warnings * website: connect product wording Co-Authored-By: pearkes <jackpearkes@gmail.com> * website: remove RC notification * commmand/acl: fix usage docs for ACL tokens * agent: remove comment, OperatorRead * website: improve multi-dc docs Still not happy with this but tried to make it slightly more informative. * website: put back acl guide warning for 1.4.0 * website: simplify multi-dc page and respond to feedback * Fix Multi-DC typos on connect index page. * Improve Multi-DC overview. A full guide is a WIP and will be added post-release. * Fixes typo avaiable > available	6 years ago
Paul Banks	37d88cad29	Allow ACL legacy migration via CLI (#4882) ... * Adds a flag to `consul acl token update` that allows legacy ACLs to be upgraded via the CLI. Also fixes a bug where descriptions are deleted if not specified. * Remove debug	6 years ago
R.B. Boyer	57dd160f40	command/debug: make better use of atomic operations to write out the debug snapshots to disk	6 years ago
R.B. Boyer	9211d2701d	fix comment typos (#4890)	6 years ago
Paul Banks	33ae0149ea	Doc and whitespace fixes for translate-rules command (#4877)	6 years ago
Martin Halder	988ceb697a	website: fix minor typo in documentation (#4864)	6 years ago
Matt Keeler	a02a6be6b9	Implement CLI token cloning & special ID handling (#4827) ... * Implement CLI token cloning & special ID handling * Update a couple CLI commands to take some alternative options. * Document the CLI. * Update the policy list and set-agent-token synopsis	6 years ago
Dhi Aurrahman	649082ab35	connect: Fix comment DYNAMIC_DNS to LOGICAL_DNS (#4799) ... LOGICAL_DNS is one of the supported service discovery types [1]. [1] https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/service_discovery#arch-overview-service-discovery-types	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791) ... This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Jack Pearkes	8c684db488	New command: consul debug (#4754) ... * agent/debug: add package for debugging, host info * api: add v1/agent/host endpoint * agent: add v1/agent/host endpoint * command/debug: implementation of static capture * command/debug: tests and only configured targets * agent/debug: add basic test for host metrics * command/debug: add methods for dynamic data capture * api: add debug/pprof endpoints * command/debug: add pprof * command/debug: timing, wg, logs to disk * vendor: add gopsutil/disk * command/debug: add a usage section * website: add docs for consul debug * agent/host: require operator:read * api/host: improve docs and no retry timing * command/debug: fail on extra arguments * command/debug: fixup file permissions to 0644 * command/debug: remove server flags * command/debug: improve clarity of usage section * api/debug: add Trace for profiling, fix profile * command/debug: capture profile and trace at the same time * command/debug: add index document * command/debug: use "clusters" in place of members * command/debug: remove address in output * command/debug: improve comment on metrics sleep * command/debug: clarify usage * agent: always register pprof handlers and protect This will allow us to avoid a restart of a target agent for profiling by always registering the pprof handlers. Given this is a potentially sensitive path, it is protected with an operator:read ACL and enable debug being set to true on the target agent. enable_debug still requires a restart. If ACLs are disabled, enable_debug is sufficient. * command/debug: use trace.out instead of .prof More in line with golang docs. * agent: fix comment wording * agent: wrap table driven tests in t.run()	6 years ago
Aestek	25f04fbd21	[Security] Add finer control over script checks (#4715) ... * Add -enable-local-script-checks options These options allow for a finer control over when script checks are enabled by giving the option to only allow them when they are declared from the local file system. * Add documentation for the new option * Nitpick doc wording	6 years ago
Paul Banks	c9217c958e	merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in	6 years ago
Paul Banks	f9c0f00abb	cli: envoy command default gRPC port (#4768) ... * Default gRPC port; Start on some basic tests for argument and ENV handling; Make Exec test less platform-dependent. * Allow hot-restarts * Remove debug	6 years ago
R.B. Boyer	c310451b2b	cli: avoid passing envoy bootstrap configuration as arguments (#4747) ... Play a trick with CLOEXEC to pass the envoy bootstrap configuration as an open file descriptor to the exec'd envoy process. The file only briefly touches disk before being unlinked. We convince envoy to read from this open file descriptor by using the /dev/fd/$FDNUMBER mechanism to read the open file descriptor as a file. Because the filename no longer has an extension envoy's sniffing logic falls back on JSON instead of YAML, so the bootstrap configuration must be generated as JSON instead.	6 years ago