consul

Commit Graph

Author	SHA1	Message	Date
R.B. Boyer	b6f686fecb	uiserver: upstream refactors done elsewhere (#8891 )	4 years ago
Paul Banks	332477f24c	Add /v1/internal/ui/metrics-proxy API endpoint that proxies to a configured metrics provider backend.	4 years ago
Kit Patella	7fe2f80b4b	add config flag to disable 1.9 metrics backwards compatibility. Add warnings on start and reload on default value	4 years ago
R.B. Boyer	a2a8e9c783	connect: intentions are now managed as a new config entry kind "service-intentions" (#8834 ) - Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older copies of consul will not see new config entries it doesn't understand replicate down. - Add shim conversion code so that the old API/CLI method of interacting with intentions will continue to work so long as none of these are edited via config entry endpoints. Almost all of the read-only APIs will continue to function indefinitely. - Add new APIs that operate on individual intentions without IDs so that the UI doesn't need to implement CAS operations. - Add a new serf feature flag indicating support for intentions-as-config-entries. - The old line-item intentions way of interacting with the state store will transparently flip between the legacy memdb table and the config entry representations so that readers will never see a hiccup during migration where the results are incomplete. It uses a piece of system metadata to control the flip. - The primary datacenter will begin migrating intentions into config entries on startup once all servers in the datacenter are on a version of Consul with the intentions-as-config-entries feature flag. When it is complete the old state store representations will be cleared. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up. - The secondary datacenters continue to run the old intentions replicator until all servers in the secondary DC and primary DC support intentions-as-config-entries (via serf flag). Once this condition it met the old intentions replicator ceases. - The secondary datacenters replicate the new config entries as they are migrated in the primary. When they detect that the primary has zeroed it's old state store table it waits until all config entries up to that point are replicated and then zeroes its own copy of the old state store table. We also record a piece of system metadata indicating this has occurred. We use this metadata to skip ALL of this code the next time the leader starts up.	4 years ago
Paul Banks	e4db845246	Refactor uiserver to separate package, cleaner Reloading	4 years ago
Paul Banks	f6d55e1d25	Fix reload test; address other PR feedback	4 years ago
Paul Banks	54a33efa4b	Fix JSON encoding of metrics options which broke the index but didn't break tests. Also add tests that do catch that error.	4 years ago
Paul Banks	526bab6164	Add config changes for UI metrics	4 years ago
Daniel Nephin	282fbdfa75	api: rename HTTPServer to HTTPHandlers Resolves a TODO about naming. This type is a set of handlers for an http.Server, it is not itself a Server. It provides http.Handler functions.	4 years ago
Paul Banks	2ae5230851	Update UI Config passing to not use an inline script (#8645 ) * Update UI Config passing to not use an inline script * Update agent/http.go * Fix incorrect placeholder name	4 years ago
Daniel Nephin	6ca45e1a61	agent: add apiServers type for managing HTTP servers Remove Server field from HTTPServer. The field is no longer used.	4 years ago
Paul Coignet	1d75a8fb50	Fix tests	4 years ago
Paul Coignet	b8b939b98a	Keep both metrics	4 years ago
Paul Coignet	cf68577bef	Use method and path as labels	4 years ago
Daniel Nephin	5247ef4c70	Remove ACLsEnabled from delegate interface In all cases (oss/ent, client/server) this method was returning a value from config. Since the value is consistent, it doesn't need to be part of the delegate interface.	4 years ago
Daniel Nephin	a5e45defb1	agent/http: un-embed the HTTPServer The embedded HTTPServer struct is not used by the large HTTPServer struct. It is used by tests and the agent. This change is a small first step in the process of removing that field. The eventual goal is to reduce the scope of HTTPServer making it easier to test, and split into separate packages.	4 years ago
gitforbit	808f632346	agent-http: cleanup: return nil instead of err (#8043 ) Since err is already checked, it should return `nil`	4 years ago
Hans Hasselberg	c7e6c9ebec	http: use default minsize for gzip handler. (#7354 ) Fixes #6306	5 years ago
Jono Sosulska	c554ba9e10	Replace whitelist/blacklist terminology with allowlist/denylist (#7971 ) * Replace whitelist/blacklist terminology with allowlist/denylist	5 years ago
Daniel Nephin	4f738b462b	Handle error from template.Execute Refactored the function to make the problem more obvious, by using a guard.	5 years ago
Daniel Nephin	c662f0f0de	Fix a number of problems found by staticcheck Some of these problems are minor (unused vars), but others are real bugs (ignored errors). Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>	5 years ago
Kit Patella	ad1d4d4d07	http: migrate from instrumentation in s.wrap() to an s.enterpriseHandler()	5 years ago
Daniel Nephin	ff0d894101	agent: remove deadcode that called lib.TranslateKeys Move the last remaining function from agent/config.go to the one place it was called.	5 years ago
Kit Patella	ccece5cd21	http: rename paresTokenResolveProxy to parseTokenWithDefault	5 years ago
Kit Patella	927f584761	agent: stub out auditing functionality in OSS	5 years ago
John Cowen	e83fb1882c	Adds http_config.response_headers to the UI headers plus tests (#7369 )	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130 ) * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Matt Keeler	9d801d1dc2	ui: feature support templating for index.html (#6921 )	5 years ago
Matt Keeler	a704ebe639	Add Namespace support to the API module and the CLI commands (#6874 ) Also update the Docs and fixup the HTTP API to return proper errors when someone attempts to use Namespaces with an OSS agent. Add Namespace HTTP API docs Make all API endpoints disallow unknown fields	5 years ago
Matt Keeler	7b471f6bf8	OSS Modifications necessary for sessions namespacing	5 years ago
Ferenc Fabian	c90e838495	Case sensitive Authorization header with lower-cased scheme in… (#6724 )	5 years ago
Sarah Adams	78ad8203a4	Use encoding/json as JSON decoder instead of mapstructure (#6680 ) Fixes #6147	5 years ago
Matt Keeler	e4ea9b0a96	Updates to allow for Namespacing ACL resources in Consul Enterp… (#6675 ) Main Changes: • method signature updates everywhere to account for passing around enterprise meta. • populate the EnterpriseAuthorizerContext for all ACL related authorizations. • ACL resource listings now operate like the catalog or kv listings in that the returned entries are filtered down to what the token is allowed to see. With Namespaces its no longer all or nothing. • Modified the acl.Policy parsing to abstract away basic decoding so that enterprise can do it slightly differently. Also updated method signatures so that when parsing a policy it can take extra ent metadata to use during rules validation and policy creation. Secondary Changes: • Moved protobuf encoding functions out of the agentpb package to eliminate circular dependencies. • Added custom JSON unmarshalers for a few ACL resource types (to support snake case and to get rid of mapstructure) • AuthMethod validator cache is now an interface as these will be cached per-namespace for Consul Enterprise. • Added checks for policy/role link existence at the RPC API so we don’t push the request through raft to have it fail internally. • Forward ACL token delete request to the primary datacenter when the secondary DC doesn’t have the token. • Added a bunch of ACL test helpers for inserting ACL resource test data.	5 years ago
Matt Keeler	973341a592	ACL Authorizer overhaul (#6620 ) * ACL Authorizer overhaul To account for upcoming features every Authorization function can now take an extra acl.EnterpriseAuthorizerContext. These are unused in OSS and will always be nil. Additionally the acl package has received some thorough refactoring to enable all of the extra Consul Enterprise specific authorizations including moving sentinel enforcement into the stubbed structs. The Authorizer funcs now return an acl.EnforcementDecision instead of a boolean. This improves the overall interface as it makes multiple Authorizers easily chainable as they now indicate whether they had an authoritative decision or should use some other defaults. A ChainedAuthorizer was added to handle this Authorizer enforcement chain and will never itself return a non-authoritative decision. Include stub for extra enterprise rules in the global management policy * Allow for an upgrade of the global-management policy	5 years ago
Matt Keeler	76cf54068b	Expand the QueryOptions and QueryMeta interfaces (#6545 ) In a previous PR I made it so that we had interfaces that would work enough to allow blockingQueries to work. However to complete this we need all fields to be settable and gettable. Notes: • If Go ever gets contracts/generics then we could get rid of all the Getters/Setters • protoc / protoc-gen-gogo are going to generate all the getters for us. • I copied all the getters/setters from the protobuf funcs into agent/structs/protobuf_compat.go • Also added JSON marshaling funcs that use jsonpb for protobuf types.	5 years ago
Mike Morris	65be58703c	connect: remove managed proxies (#6220 ) * connect: remove managed proxies implementation and all supporting config options and structs * connect: remove deprecated ProxyDestination * command: remove CONNECT_PROXY_TOKEN env var * agent: remove entire proxyprocess proxy manager * test: remove all managed proxy tests * test: remove irrelevant managed proxy note from TestService_ServerTLSConfig * test: update ContentHash to reflect managed proxy removal * test: remove deprecated ProxyDestination test * telemetry: remove managed proxy note * http: remove /v1/agent/connect/proxy endpoint * ci: remove deprecated test exclusion * website: update managed proxies deprecation page to note removal * website: remove managed proxy configuration API docs * website: remove managed proxy note from built-in proxy config * website: add note on removing proxy subdirectory of data_dir	5 years ago
Christian Muehlhaeuser	7753b97cc7	Simplified code in various places (#6176 ) All these changes should have no side-effects or change behavior: - Use bytes.Buffer's String() instead of a conversion - Use time.Since and time.Until where fitting - Drop unnecessary returns and assignment	5 years ago
Sarah Christoff	d3d92d76f3	ui: modify content path (#5950 ) * Add ui-content-path flag * tests complete, regex validator on string, index.html updated * cleaning up debugging stuff * ui: Enable ember environment configuration to be set via the go binary at runtime (#5934) * ui: Only inject {{.ContentPath}} if we are makeing a prod build... ...otherwise we just use the current rootURL This gets injected into a <base /> node which solves the assets path problem but not the ember problem * ui: Pull out the <base href=""> value and inject it into ember env See previous commit: The <base href=""> value is 'sometimes' injected from go at index serve time. We pass this value down to ember by overwriting the ember config that is injected via a <meta> tag. This has to be done before ember bootup. Sometimes (during testing and development, basically not production) this is injected with the already existing value, in which case this essentially changes nothing. The code here is slightly abstracted away from our specific usage to make it easier for anyone else to use, and also make sure we can cope with using this same method to pass variables down from the CLI through to ember in the future. * ui: We can't use <base /> move everything to javascript (#5941) Unfortuantely we can't seem to be able to use <base> and rootURL together as URL paths will get doubled up (`ui/ui/`). This moves all the things that we need to interpolate with .ContentPath to the `startup` javascript so we can conditionally print out `{{.ContentPath}}` in lots of places (now we can't use base) * fixed when we serve index.html * ui: For writing a ContentPath, we also need to cope with testing... (#5945) ...and potentially more environments Testing has more additional things in a separate index.html in `tests/` This make the entire thing a little saner and uses just javascriopt template literals instead of a pseudo handbrake synatx for our templating of these files. Intead of just templating the entire file this way, we still only template `{{content-for 'head'}}` and `{{content-for 'body'}}` in this way to ensure we support other plugins/addons * build: Loosen up the regex for retrieving the CONSUL_VERSION (#5946) * build: Loosen up the regex for retrieving the CONSUL_VERSION 1. Previously the `sed` replacement was searching for the CONSUL_VERSION comment at the start of a line, it no longer does this to allow for indentation. 2. Both `grep` and `sed` where looking for the omment at the end of the line. We've removed this restriction here. We don't need to remove it right now, but if we ever put the comment followed by something here the searching would break. 3. Added `xargs` for trimming the resulting version string. We aren't using this already in the rest of the scripts, but we are pretty sure this is available on most systems. * ui: Fix erroneous variable, and also force an ember cache clean on build 1. We referenced a variable incorrectly here, this fixes that. 2. We also made sure that every `make` target clears ember's `tmp` cache to ensure that its not using any caches that have since been edited everytime we call a `make` target. * added docs, fixed encoding * fixed go fmt * Update agent/config/config.go Co-Authored-By: R.B. Boyer <public@richardboyer.net> * Completed Suggestions * run gofmt on http.go * fix testsanitize * fix fullconfig/hcl by setting correct 'want' * ran gofmt on agent/config/runtime_test.go * Update website/source/docs/agent/options.html.md Co-Authored-By: Hans Hasselberg <me@hans.io> * Update website/source/docs/agent/options.html.md Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * remove contentpath from redirectFS struct	5 years ago
Matt Keeler	d0f410cd84	Make a few config entry endpoints return 404s and allow for snake_case and lowercase key names. (#5748 )	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579 ) Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Freddy	b2c12550d7	Remove old UI, option to use it, and its build processes	6 years ago
Kyle Havlovitz	1a4978fb94	Re-add ReadableDuration types to health check definition This is to fix the backwards-incompatible change made in 1.4.1 by changing these fields to time.Duration.	6 years ago
Paul Banks	ef9f27cbc8	connect: tame thundering herd of CSRs on CA rotation (#5228 ) * Support rate limiting and concurrency limiting CSR requests on servers; handle CA rotations gracefully with jitter and backoff-on-rate-limit in client * Add CSR rate limiting docs * Fix config naming and add tests for new CA configs	6 years ago
Aestek	c043de5381	[Security] Allow blocking Write endpoints on Agent using Network Addresses (#4719 ) * Add -write-allowed-nets option * Add documentation for the new write_allowed_nets option	6 years ago
Grégoire Seux	4f62a3b528	Implement /v1/agent/health/service/<service name> endpoint (#3551 ) This endpoint aggregates all checks related to <service id> on the agent and return an appropriate http code + the string describing the worst check. This allows to cleanly expose service status to other component, hiding complexity of multiple checks. This is especially useful to use consul to feed a load balancer which would delegate health checking to consul agent. Exposing this endpoint on the agent is necessary to avoid a hit on consul servers and avoid decreasing resiliency (this endpoint will work even if there is no consul leader in the cluster).	6 years ago
Hans Hasselberg	0b4a879203	ui: serve /robots.txt when UI is enabled. (#5089 ) * serve /robots.txt * robots.txt: disallow everything	6 years ago
Kyle Havlovitz	67bac7a815	api: add support for new txn operations	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791 ) This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Jack Pearkes	8c684db488	New command: consul debug (#4754 ) * agent/debug: add package for debugging, host info * api: add v1/agent/host endpoint * agent: add v1/agent/host endpoint * command/debug: implementation of static capture * command/debug: tests and only configured targets * agent/debug: add basic test for host metrics * command/debug: add methods for dynamic data capture * api: add debug/pprof endpoints * command/debug: add pprof * command/debug: timing, wg, logs to disk * vendor: add gopsutil/disk * command/debug: add a usage section * website: add docs for consul debug * agent/host: require operator:read * api/host: improve docs and no retry timing * command/debug: fail on extra arguments * command/debug: fixup file permissions to 0644 * command/debug: remove server flags * command/debug: improve clarity of usage section * api/debug: add Trace for profiling, fix profile * command/debug: capture profile and trace at the same time * command/debug: add index document * command/debug: use "clusters" in place of members * command/debug: remove address in output * command/debug: improve comment on metrics sleep * command/debug: clarify usage * agent: always register pprof handlers and protect This will allow us to avoid a restart of a target agent for profiling by always registering the pprof handlers. Given this is a potentially sensitive path, it is protected with an operator:read ACL and enable debug being set to true on the target agent. enable_debug still requires a restart. If ACLs are disabled, enable_debug is sufficient. * command/debug: use trace.out instead of .prof More in line with golang docs. * agent: fix comment wording * agent: wrap table driven tests in t.run()	6 years ago
Paul Banks	dca1303d05	Connect Envoy Command (#4735 ) * Plumb xDS server and proxyxfg into the agent startup * Add `consul connect envoy` command to allow running Envoy as a connect sidecar. * Add test for help tabs; typos and style fixups from review	6 years ago

1 2 3

108 Commits (262fb3a0c382ff5434d2787dbbce007942532b47)