John Murret
e027c94b52
* server: add placeholder glue for rate limit handler This commit adds a no-op implementation of the rate-limit handler and adds it to the `consul.Server` struct and setup code. This allows us to start working on the net/rpc and gRPC interceptors and config logic. * Add handler errors * Set the global read and write limits * fixing multilimiter moving packages * Fix typo * Simplify globalLimit usage * add multilimiter and tests * exporting LimitedEntity * Apply suggestions from code review Co-authored-by: John Murret <john.murret@hashicorp.com> * add config update and rename config params * add doc string and split config * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * use timer to avoid go routine leak and change the interface * add comments to tests * fix failing test * add prefix with config edge, refactor tests * Apply suggestions from code review Co-authored-by: Dan Upton <daniel@floppy.co> * refactor to apply configs for limiters under a prefix * add fuzz tests and fix bugs found. Refactor reconcile loop to have a simpler logic * make KeyType an exported type * split the config and limiter trees to fix race conditions in config update * rename variables * fix race in test and remove dead code * fix reconcile loop to not create a timer on each loop * add extra benchmark tests and fix tests * fix benchmark test to pass value to func * server: add placeholder glue for rate limit handler This commit adds a no-op implementation of the rate-limit handler and adds it to the `consul.Server` struct and setup code. This allows us to start working on the net/rpc and gRPC interceptors and config logic. * Set the global read and write limits * fixing multilimiter moving packages * add server configuration for global rate limiting. * remove agent test * remove added stuff from handler * remove added stuff from multilimiter * removing unnecessary TODOs * Removing TODO comment from handler * adding in defaulting to infinite * add disabled status in there * adding in documentation for disabled mode. * make disabled the default. * Add mock and agent test * addig documentation and missing mock file. * Fixing test TestLoad_IntegrationWithFlags * updating docs based on PR feedback. * Updating Request Limits mode to use int based on PR feedback. * Adding RequestLimits struct so we have a nested struct in ReloadableConfig. * fixing linting references * Update agent/consul/rate/handler.go Co-authored-by: Dan Upton <daniel@floppy.co> * Update agent/consul/config.go Co-authored-by: Dan Upton <daniel@floppy.co> * removing the ignore of the request limits in JSON. addingbuilder logic to convert any read rate or write rate less than 0 to rate.Inf * added conversion function to convert request limits object to handler config. * Updating docs to reflect gRPC and RPC are rate limit and as a result, HTTP requests are as well. * Updating values for TestLoad_FullConfig() so that they were different and discernable. * Updating TestRuntimeConfig_Sanitize * Fixing TestLoad_IntegrationWithFlags test * putting nil check in place * fixing rebase * removing change for missing error checks. will put in another PR * Rebasing after default multilimiter config change * resolving rebase issues * updating reference for incomingRPCLimiter to use interface * updating interface * Updating interfaces * Fixing mock reference Co-authored-by: Daniel Upton <daniel@floppy.co> Co-authored-by: Dhia Ayachi <dhia@hashicorp.com> |
2 years ago | |
---|---|---|
.. | ||
README.md | docs: simplify agent docs slugs | 3 years ago |
checklist-adding-config-fields.md | adding config for request_limits (#15531) | 2 years ago |
README.md
Agent Configuration
The Agent Configuration is the primary mechanism for configuring Consul. Agent Configuration also allows for specifying Config Entries, Services, and Checks that will be loaded when the agent starts.
Most configuration comes from hcl or json
files, but some configuration can also be
specified using command line flags, and some can be loaded with Auto-Config.
See also the checklist for adding a new field to the configuration.
Code
The Agent Configuration is implemented in agent/config, and the primary entrypoint is Load. Config loading is performed in phases:
- Command line flags are used to create a
config.LoadOpts
and passed toLoad
. Load
reads all the config files and builds an ordered list ofconfig.Source
.- Each
config.Source
is read to produce aconfig.Config
. - Each
config.Config
is merged ontop the previous. - A
config.RuntimeConfig
is produced from the mergedconfig.Config
- The
config.RuntimeConfig
is validated. - Finally a result is returned with the
RuntimeConfig
and any warnings, or an error.
If Auto-Config is enabled, when it receives the config from the server, the
entire process is repeated a second time with the addition config provided as another
config.Source
.
Default values can be specified in one of the default sources or set when
converting from Config
to RuntimeConfig
in builder.build. Hopefully in the future we
should remove one of those ways of setting default values.
Auto-Config
Auto-Config is enabled by the auto_config field in an Agent Configuration file. It is implemented in a couple packages.
- the server RPC endpoint is in agent/consul/auto_config_endpoint.go
- the client that receives and applies the config is implemented in agent/auto-config