consul/agent
Dhia Ayachi 83720e5737
add a rate limiter to config auto-reload (#12490)
* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* quit timer loop if ctx is canceled

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-04-04 11:31:39 -04:00
..
ae sdk: add TestLogLevel for setting log level in tests 2022-02-03 13:42:28 -05:00
auto-config Bump go-control-plane 2022-03-30 13:11:27 -04:00
cache bulk rewrite using this script 2022-01-20 10:46:23 -06:00
cache-types bulk rewrite using this script 2022-01-20 10:46:23 -06:00
checks remove DualStack field from check TCP #12629 (#12630) 2022-03-29 14:56:01 -04:00
config add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
configentry server: suppress spurious blocking query returns where multiple config entries are involved (#12362) 2022-02-25 15:46:34 -06:00
connect Avoid using sys/mounts to enable namespaces (#12655) 2022-03-31 23:35:38 -07:00
consul Add expanded token read flag and endpoint option 2022-03-31 10:49:49 -07:00
debug bulk rewrite using this script 2022-01-20 10:46:23 -06:00
dns lib: add validation package + DNS label validation (#12535) 2022-03-17 18:31:28 -07:00
exec re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
grpc Bump go-control-plane 2022-03-30 13:11:27 -04:00
local auto-reload configuration when config files change (#12329) 2022-03-31 15:11:49 -04:00
metadata partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
mock
pool Restructure gRPC server setup (#12586) 2022-03-22 12:40:24 +00:00
proxycfg Implement Lambda Patching in the Serverless Plugin 2022-03-31 16:45:32 -04:00
router sdk: add TestLogLevel for setting log level in tests 2022-02-03 13:42:28 -05:00
routine-leak-checker Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
rpc/middleware polish rpc.service.call metric behavior (#12624) 2022-03-31 10:49:37 -07:00
rpcclient/health Bump go-control-plane 2022-03-30 13:11:27 -04:00
structs Merge pull request #12670 from hashicorp/token-read-expanded 2022-03-31 12:24:11 -07:00
submatview Bump go-control-plane 2022-03-30 13:11:27 -04:00
systemd
token agent/token: rename `agent_master` to `agent_recovery` (internally) (#11744) 2021-12-07 12:12:47 +00:00
uiserver auto-updated agent/uiserver/bindata_assetfs.go from commit 73b6687c5 2022-02-21 12:27:52 +00:00
xds Merge pull request #12681 from hashicorp/lambda-patching-tweaks 2022-04-01 19:59:30 -04:00
acl.go Bulk acl message fixup oss (#12470) 2022-03-10 18:48:27 -08:00
acl_endpoint.go Add expanded token read flag and endpoint option 2022-03-31 10:49:49 -07:00
acl_endpoint_legacy.go
acl_endpoint_legacy_test.go agent: Ensure partition is considered in agent endpoints (#11427) 2021-10-26 15:20:57 -04:00
acl_endpoint_test.go Add expanded token read flag and endpoint option 2022-03-31 10:49:49 -07:00
acl_oss.go agent: support `X-Consul-Results-Filtered-By-ACLs` header in agent-local endpoints (#11610) 2021-12-03 20:36:28 +00:00
acl_test.go sdk: add TestLogLevel for setting log level in tests 2022-02-03 13:42:28 -05:00
agent.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
agent_endpoint.go Restructure gRPC server setup (#12586) 2022-03-22 12:40:24 +00:00
agent_endpoint_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
agent_endpoint_test.go acl: return a resposne from ResolveToken that includes the ACLIdentity 2022-01-22 14:33:09 -05:00
agent_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
agent_test.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
apiserver.go
apiserver_test.go
catalog_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
catalog_endpoint_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
catalog_endpoint_test.go catalog: compare node names case insensitively in more places (#12444) 2022-02-24 16:54:47 -06:00
check.go
config_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
config_endpoint_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
connect_auth.go Bulk acl message fixup oss (#12470) 2022-03-10 18:48:27 -08:00
connect_ca_endpoint.go add root_cert_ttl option for consul connect, vault ca providers (#11428) 2021-11-02 11:02:10 -07:00
connect_ca_endpoint_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
coordinate_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
coordinate_endpoint_test.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
delegate_mock_test.go acl: remove unused methods on fakes, and add changelog 2022-01-31 17:53:53 -05:00
denylist.go
denylist_test.go
discovery_chain_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
discovery_chain_endpoint_test.go server: ensure that service-defaults meta is incorporated into the discovery chain response (#12511) 2022-03-30 10:04:18 -05:00
dns.go dns: add endpoint for querying service virtual IPs 2021-12-02 16:40:28 -08:00
dns_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
dns_test.go Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
enterprise_delegate_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
event_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
event_endpoint_test.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
federation_state_endpoint.go URL-encode/decode resource names for HTTP API part 3 (#12103) 2022-01-26 13:12:42 -05:00
health_endpoint.go URL-encode/decode resource names for HTTP API part 5 (#12297) 2022-02-14 10:47:06 -05:00
health_endpoint_test.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
http.go mark disable_compat_1.9 to deprecate in 1.13, change default to true (#12675) 2022-04-01 10:35:56 -07:00
http_decode_test.go
http_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
http_oss_test.go Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
http_register.go oss: Add overview UI internal endpoint 2022-03-22 17:05:09 -07:00
http_test.go testing: fix test failures caused by new log level 2022-02-03 17:07:39 -05:00
intentions_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
intentions_endpoint_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
intentions_endpoint_test.go
keyring.go Allows keyring operations on client agents 2022-02-24 17:24:57 +00:00
keyring_test.go Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
kvs_endpoint.go Revert "[Docs] Agent configuration hierarchy " 2022-03-15 16:13:58 -07:00
kvs_endpoint_test.go
metrics.go agent: move agent tls metric monitor to a more appropriate place 2021-10-27 16:26:09 -04:00
metrics_test.go mark disable_compat_1.9 to deprecate in 1.13, change default to true (#12675) 2022-04-01 10:35:56 -07:00
nodeid.go
nodeid_test.go
notify.go
notify_test.go
operator_endpoint.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
operator_endpoint_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
operator_endpoint_test.go
prepared_query_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
prepared_query_endpoint_test.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
reload.go
remote_exec.go
remote_exec_test.go Remove references to "master" ACL tokens in tests (#11751) 2021-12-07 12:48:50 +00:00
retry_join.go agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
retry_join_test.go
service_checks_test.go
service_manager.go
service_manager_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
session_endpoint.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
session_endpoint_test.go
setup.go auto-reload configuration when config files change (#12329) 2022-03-31 15:11:49 -04:00
setup_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
sidecar_service.go
sidecar_service_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
signal_unix.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
signal_windows.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
snapshot_endpoint.go
snapshot_endpoint_test.go
status_endpoint.go
status_endpoint_test.go
streaming_test.go regenerate expired certs (#11462) 2021-11-01 11:40:16 -04:00
testagent.go add a rate limiter to config auto-reload (#12490) 2022-04-04 11:31:39 -04:00
testagent_test.go
translate_addr.go
txn_endpoint.go Revert "[Docs] Agent configuration hierarchy " 2022-03-15 16:13:58 -07:00
txn_endpoint_test.go Change error-handling across handlers. (#12225) 2022-01-31 11:17:35 -05:00
ui_endpoint.go oss: Add overview UI internal endpoint 2022-03-22 17:05:09 -07:00
ui_endpoint_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
ui_endpoint_test.go Support for connect native services in topology view. (#12098) 2022-02-16 16:51:54 -05:00
user_event.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
user_event_test.go
util.go Remove some usage of md5 from the system (#11491) 2021-11-04 13:07:54 -07:00
util_test.go Remove some usage of md5 from the system (#11491) 2021-11-04 13:07:54 -07:00
watch_handler.go
watch_handler_test.go