consul

Commit Graph

Author	SHA1	Message	Date
Matt Keeler	8f0ab0129e	Miscellaneous Fixes (#6896 ) Ensure we close the Sentinel Evaluator so as not to leak go routines Fix a bunch of test logging so that various warnings when starting a test agent go to the ltest logger and not straight to stdout. Various canned ent meta types always return a valid pointer (no more nils). This allows us to blindly deref + assign in various places. Update ACL index tracking to ensure oss -> ent upgrades will work as expected. Update ent meta parsing to include function to disallow wildcarding.	5 years ago
Matt Keeler	deb91f3d3c	[Feature] API: Add a internal endpoint to query for ACL authori… (#6888 ) * Implement endpoint to query whether the given token is authorized for a set of operations * Updates to allow for remote ACL authorization via RPC This is only used when making an authorization request to a different datacenter.	5 years ago
Matt Keeler	79f78632e1	Update the ACL Resolver to allow for Consul Enterprise specific hooks. (#6687 )	5 years ago
Matt Keeler	e4ea9b0a96	Updates to allow for Namespacing ACL resources in Consul Enterp… (#6675 ) Main Changes: • method signature updates everywhere to account for passing around enterprise meta. • populate the EnterpriseAuthorizerContext for all ACL related authorizations. • ACL resource listings now operate like the catalog or kv listings in that the returned entries are filtered down to what the token is allowed to see. With Namespaces its no longer all or nothing. • Modified the acl.Policy parsing to abstract away basic decoding so that enterprise can do it slightly differently. Also updated method signatures so that when parsing a policy it can take extra ent metadata to use during rules validation and policy creation. Secondary Changes: • Moved protobuf encoding functions out of the agentpb package to eliminate circular dependencies. • Added custom JSON unmarshalers for a few ACL resource types (to support snake case and to get rid of mapstructure) • AuthMethod validator cache is now an interface as these will be cached per-namespace for Consul Enterprise. • Added checks for policy/role link existence at the RPC API so we don’t push the request through raft to have it fail internally. • Forward ACL token delete request to the primary datacenter when the secondary DC doesn’t have the token. • Added a bunch of ACL test helpers for inserting ACL resource test data.	5 years ago
Matt Keeler	973341a592	ACL Authorizer overhaul (#6620 ) * ACL Authorizer overhaul To account for upcoming features every Authorization function can now take an extra acl.EnterpriseAuthorizerContext. These are unused in OSS and will always be nil. Additionally the acl package has received some thorough refactoring to enable all of the extra Consul Enterprise specific authorizations including moving sentinel enforcement into the stubbed structs. The Authorizer funcs now return an acl.EnforcementDecision instead of a boolean. This improves the overall interface as it makes multiple Authorizers easily chainable as they now indicate whether they had an authoritative decision or should use some other defaults. A ChainedAuthorizer was added to handle this Authorizer enforcement chain and will never itself return a non-authoritative decision. Include stub for extra enterprise rules in the global management policy * Allow for an upgrade of the global-management policy	5 years ago
Jack Pearkes	36ebca1fd0	Fix to prevent allowing recursive KV deletions when we shouldn’t	6 years ago
Matt Keeler	f88d1ccc36	Handle rules translation when coming from the JSON compat HCL (#5662 ) We were not handling some object keys when they were strings instead of identifiers. Now both are handled. Fixes #5493	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791 ) This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Matt Keeler	fbb1a7a52b	Rewrite all of acl_test.go This is now using table driven testing. In addition to conversion of old tests I also implemented several new tests for the acl fixes in my previous commit. In particular the issues I saw with ACLs for prepared queries, keyring and operator all have tests for those and comments indicating that they would have previously failed.	6 years ago
Matt Keeler	883c5dd001	Fix ACL enforcement This creates one function that takes a rule and the required permissions and returns whether it should be allowed and whether to leave the decision to the parent acl. Then this function is used everywhere. This makes acl enforcement consistent. There were several places where a default allow policy with explicit deny rules wasnt being handled and several others where it wasn’t using the parent acl appropriately but would lump no policy in with a deny policy. All of that has been fixed.	6 years ago
Mitchell Hashimoto	5a47a53c70	acl: IntentionDefault => IntentionDefaultAllow	7 years ago
Mitchell Hashimoto	ac72a0c5fd	agent: ACL checks for authorize, default behavior	7 years ago
Mitchell Hashimoto	a621afe72c	agent/consul: convert intention ACLs to testify/assert	7 years ago
Mitchell Hashimoto	193f93107a	acl: implement IntentionRead/Write methods on ACL interface	7 years ago
Mitchell Hashimoto	437cc76af5	acl: parsing intentions in service block	7 years ago
Josh Soref	94835a2715	Spelling (#3958 ) * spelling: another * spelling: autopilot * spelling: beginning * spelling: circonus * spelling: default * spelling: definition * spelling: distance * spelling: encountered * spelling: enterprise * spelling: expands * spelling: exits * spelling: formatting * spelling: health * spelling: hierarchy * spelling: imposed * spelling: independence * spelling: inspect * spelling: last * spelling: latest * spelling: client * spelling: message * spelling: minimum * spelling: notify * spelling: nonexistent * spelling: operator * spelling: payload * spelling: preceded * spelling: prepared * spelling: programmatically * spelling: required * spelling: reconcile * spelling: responses * spelling: request * spelling: response * spelling: results * spelling: retrieve * spelling: service * spelling: significantly * spelling: specifies * spelling: supported * spelling: synchronization * spelling: synchronous * spelling: themselves * spelling: unexpected * spelling: validations * spelling: value	7 years ago
James Phillips	575d70aaa7	Cleans up some drift between the OSS and Enterprise trees.	7 years ago
Preetha Appan	26accb3b8a	Only allow 'list' policies within 'key' policy definitions. Consolidated two similar tests into one and fixed alignment.	7 years ago
Preetha Appan	51a04ec87d	Introduces new 'list' permission that applies to KV store recursive reads, and enforced only when opted in.	7 years ago
Preetha Appan	d7e27e67c1	Introduce Code Policy validation via sentinel, with a noop implementation	7 years ago
Frank Schröder	a3934c263c	acl: consolidate error handling (#3401 ) The error handling of the ACL code relies on the presence of certain magic error messages. Since the error values are sent via RPC between older and newer consul agents we cannot just replace the magic values with typed errors and switch to type checks since this would break compatibility with older clients. Therefore, this patch moves all magic ACL error messages into the acl package and provides default error values and helper functions which determine the type of error.	7 years ago
游远	ffcd2b1fc8	fix UnitTest in acl	7 years ago
James Phillips	022baeea13	Adds support to the ACL package for agent policies.	8 years ago
James Phillips	60d4322c49	Adds support to ACL package for session policies.	8 years ago
James Phillips	7fa4ab3fd1	Adds support to ACL package for node policies.	8 years ago
James Phillips	9b4f316b21	Sorts all the ACl policy handlers for easier navigation (no functional changes).	8 years ago
James Phillips	c01a3871c9	Adds support for snapshots and restores. (#2396 ) * Updates Raft library to get new snapshot/restore API. * Basic backup and restore working, but need some cleanup. * Breaks out a snapshot module and adds a SHA256 integrity check. * Adds snapshot ACL and fills in some missing comments. * Require a consistent read for snapshots. * Make sure snapshot works if ACLs aren't enabled. * Adds a bit of package documentation. * Returns an empty response from restore to avoid EOF errors. * Adds API client support for snapshots. * Makes internal file names match on-disk file snapshots. * Adds DC and token coverage for snapshot API test. * Adds missing documentation. * Adds a unit test for the snapshot client endpoint. * Moves the connection pool out of the client for easier testing. * Fixes an incidental issue in the prepared query unit test. I realized I had two servers in bootstrap mode so this wasn't a good setup. * Adds a half close to the TCP stream and fixes panic on error. * Adds client and endpoint tests for snapshots. * Moves the pool back into the snapshot RPC client. * Adds a TLS test and fixes half-closes for TLS connections. * Tweaks some comments. * Adds a low-level snapshot test. This is independent of Consul so we can pull this out into a library later if we want to. * Cleans up snapshot and archive and completes archive tests. * Sends a clear error for snapshot operations in dev mode. Snapshots require the Raft snapshots to be readable, which isn't supported in dev mode. Send a clear error instead of a deep-down Raft one. * Adds docs for the snapshot endpoint. * Adds a stale mode and index feedback for snapshot saves. This gives folks a way to extract data even if the cluster has no leader. * Changes the internal format of a snapshot from zip to tgz. * Pulls in Raft fix to cancel inflight before a restore. * Pulls in new Raft restore interface. * Adds metadata to snapshot saves and a verify function. * Adds basic save and restore snapshot CLI commands. * Gets rid of tarball extensions and adds restore message. * Fixes an incidental bad link in the KV docs. * Adds documentation for the snapshot CLI commands. * Scuttle any request body when a snapshot is saved. * Fixes archive unit test error message check. * Allows for nil output writers in snapshot RPC handlers. * Renames hash list Decode to DecodeAndVerify. * Closes the client connection for snapshot ops. * Lowers timeout for restore ops. * Updates Raft vendor to get new Restore signature and integrates with Consul. * Bounces the leader's internal state when we do a restore.	8 years ago
James Phillips	e5850d8a26	Adds new consul operator endpoint, CLI, and ACL and some basic Raft commands.	8 years ago
James Phillips	ae1cd5b47d	Switches all ACL caches to 2Q.	8 years ago
James Phillips	e831727923	Activates fallback to replicated ACLs.	8 years ago
James Phillips	483898abe5	Renames "prepared_query" ACL policy to "query".	9 years ago
James Phillips	899dcfe053	Completes switch of prepared_query ACLs to govern query names.	9 years ago
James Phillips	67de77482e	Creates new "prepared-query" ACL type and new token capture behavior. Prior to this change, prepared queries had the following behavior for ACLs, which will need to change to support templates: 1. A management token, or a token with read access to the service being queried needed to be provided in order to create a prepared query. 2. The token used to create the prepared query was stored with the query in the state store and used to execute the query. 3. A management token, or the token used to create the query needed to be supplied to perform and CRUD operations on an existing prepared query. This was pretty subtle and complicated behavior, and won't work for templates since the service name is computed at execution time. To solve this, we introduce a new "prepared-query" ACL type, where the prefix applies to the query name for static prepared query types and to the prefix for template prepared query types. With this change, the new behavior is: 1. A management token, or a token with "prepared-query" write access to the query name or (soon) the given template prefix is required to do any CRUD operations on a prepared query, or to list prepared queries (the list is filtered by this ACL). 2. You will no longer need a management token to list prepared queries, but you will only be able to see prepared queries that you have access to (you get an empty list instead of permission denied). 3. When listing or getting a query, because it was easy to capture management tokens given the past behavior, this will always blank out the "Token" field (replacing the contents as <hidden>) for all tokens unless a management token is supplied. Going forward, we should discourage people from binding tokens for execution unless strictly necessary. 4. No token will be captured by default when a prepared query is created. If the user wishes to supply an execution token then can pass it in via the "Token" field in the prepared query definition. Otherwise, this field will default to empty. 5. At execution time, we will use the captured token if it exists with the prepared query definition, otherwise we will use the token that's passed in with the request, just like we do for other RPCs (or you can use the agent's configured token for DNS). 6. Prepared queries with no name (accessible only by ID) will not require ACLs to create or modify (execution time will depend on the service ACL configuration). Our argument here is that these are designed to be ephemeral and the IDs are as good as an ACL. Management tokens will be able to list all of these. These changes enable templates, but also enable delegation of authority to manage the prepared query namespace.	9 years ago
James Phillips	ce0881a99a	Adds a new management ACL for prepared queries.	9 years ago
Dale Wijnand	5a28ebcaa3	Fix a bunch of typos.	9 years ago
Ryan Uber	58c26497a9	acl: adding negative tests for bad policy	10 years ago
Ryan Uber	02b49058a2	acl: more keyring tests	10 years ago
Ryan Uber	7e50a457d9	acl: allow omitting keyring policy, add tests	10 years ago
Ryan Uber	47a33e3f1a	acl: keyring policy uses a flat string	10 years ago
Ryan Uber	1b8051a783	acl: initial pass at keyring ACLs	10 years ago
Ryan Uber	90f5eb8f69	acl: fix spelling in tests	10 years ago
Ryan Uber	0c624350eb	acl: support for user events	10 years ago
Maciej Bryński	11425734d5	Consul prefix services ACLs	10 years ago
Veres Lajos	3b1068387a	typofixes - https://github.com/vlajos/misspell_fixer	10 years ago
Armon Dadgar	8ff08819c8	acl: Expose service policy checks	10 years ago
Armon Dadgar	3695f65292	acl: Support for service policies	10 years ago
Armon Dadgar	05900f35c2	acl: Test parsing JSON	10 years ago
Armon Dadgar	34e018e471	acl: Updating for HCL changes	10 years ago
Armon Dadgar	9bababf872	acl: Avoid shared cache with different parents	10 years ago
Armon Dadgar	705c6cdb86	acl: Support checking write permissions on a prefix	10 years ago

1 2

62 Commits (2ad0831b349fef104618070dfe7ddc881473546a)