10 Commits (9278bf534fe4b05ae0904c456c4e36a5758e321c)

Author	SHA1	Message	Date
Nick Cellino	5fb6ab6a3a	Move HCP Manager lifecycle management out of Link controller (#20401) ... * Add function to get update channel for watching HCP Link * Add MonitorHCPLink function This function can be called in a goroutine to manage the lifecycle of the HCP manager. * Update HCP Manager config in link monitor before starting This updates HCPMonitorLink so it updates the HCP manager with an HCP client and management token when a Link is upserted. * Let MonitorHCPManager handle lifecycle instead of link controller * Remove cleanup from Link controller and move it to MonitorHCPLink Previously, the Link Controller was responsible for cleaning up the HCP-related files on the file system. This change makes it so MonitorHCPLink handles this cleanup. As a result, we are able to remove the PlacementEachServer placement strategy for the Link controller because it no longer needs to do this per-node cleanup. * Remove HCP Manager dependency from Link Controller The Link controller does not need to have HCP Manager as a dependency anymore, so this removes that dependency in order to simplify the design. * Add Linked prefix to Linked status variables This is in preparation for adding a new status type to the Link resource. * Add new "validated" status type to link resource The link resource controller will now set a "validated" status in addition to the "linked" status. This is needed so that other components (eg the HCP manager) know when the Link is ready to link with HCP. * Fix tests * Handle new 'EndOfSnapshot' WatchList event * Fix watch test * Remove unnecessary config from TestAgent_scadaProvider Since the Scada provider is now started on agent startup regardless of whether a cloud config is provided, this removes the cloud config override from the relevant test. This change is not exactly related to the changes from this PR, but rather is something small and sort of related that was noticed while working on this PR. * Simplify link watch test and remove sleep from link watch This updates the link watch test so that it uses more mocks and does not require setting up the infrastructure for the HCP Link controller. This also removes the time.Sleep delay in the link watcher loop in favor of an error counter. When we receive 10 consecutive errors, we shut down the link watcher loop. * Add better logging for link validation. Remove EndOfSnapshot test. * Refactor link monitor test into a table test * Add some clarifying comments to link monitor * Simplify link watch test * Test a bunch more errors cases in link monitor test * Use exponential backoff instead of errorCounter in LinkWatch * Move link watch and link monitor into a single goroutine called from server.go * Refactor HCP link watcher to use single go-routine. Previously, if the WatchClient errored, we would've never recovered because we never retry to create the stream. With this change, we have a single goroutine that runs for the life of the server agent and if the WatchClient stream ever errors, we retry the creation of the stream with an exponential backoff.	10 months ago
Nick Ethier	9d4ad74a63	internal/hcp: prevent write loop on telemetrystate resource updates (#20435) ... * internal/hcp: prevent write loop on telemetrystate resource updates * Update controller.go Co-authored-by: Nick Cellino <nick.cellino@hashicorp.com> * internal/hcp: add assertion for looping controller --------- Co-authored-by: Nick Cellino <nick.cellino@hashicorp.com>	10 months ago
Nick Ethier	383d92e9ab	hcp.v2.TelemetryState resource and controller implementation (#20257) ... * pbhcp: add TelemetryState resource * agent/hcp: add GetObservabilitySecrets to client * internal/hcp: add TelemetryState controller logic * hcp/telemetry-state: added config options for hcp sdk and debug key to skip deletion during reconcile * pbhcp: update proto documentation * hcp: address PR feedback, additional validations and code cleanup * internal/hcp: fix type sig change in test * update testdata/v2-resource-dependencies	10 months ago
Melissa Kam	b0e87dbe13	[CC-7049] Stop the HCP manager when link is deleted (#20351) ... * Add Stop method to telemetry provider Stop the main loop of the provider and set the config to disabled. * Add interface for telemetry provider Added for easier testing. Also renamed Run to Start, which better fits with Stop. * Add Stop method to HCP manager * Add manager interface, rename implementation Add interface for easier testing, rename existing Manager to HCPManager. * Stop HCP manager in link Finalizer * Attempt to cleanup if resource has been deleted The link should be cleaned up by the finalizer, but there's an edge case in a multi-server setup where the link is fully deleted on one server before the other server reconciles. This will cover the case where the reconcile happens after the resource is deleted. * Add a delete mananagement token function Passes a function to the HCP manager that deletes the management token that was initially created by the manager. * Delete token as part of stopping the manager * Lock around disabling config, remove descriptions	10 months ago
Melissa Kam	3b9bb8d6f9	[CC-7044] Start HCP manager as part of link creation (#20312) ... * Check for ACL write permissions on write Link eventually will be creating a token, so require acl:write. * Convert Run to Start, only allow to start once * Always initialize HCP components at startup * Support for updating config and client * Pass HCP manager to controller * Start HCP manager in link resource Start as part of link creation rather than always starting. Update the HCP manager with values from the link before starting as well. * Fix metrics sink leaked goroutine * Remove the hardcoded disabled hostname prefix The HCP metrics sink will always be enabled, so the length of sinks will always be greater than zero. This also means that we will also always default to prefixing metrics with the hostname, which is what our documentation states is the expected behavior anyway. * Add changelog * Check and set running status in one method * Check for primary datacenter, add back test * Clarify merge reasoning, fix timing issue in test * Add comment about controller placement * Expand on breaking change, fix typo in changelog	10 months ago
Nick Cellino	ec0df00fc1	Add finalizer to link resource (#20321) ... * Add finalizer to link resource * Update internal/hcp/internal/controllers/link/controller.go Co-authored-by: Semir Patel <semir.patel@hashicorp.com> * Address PR style feedback --------- Co-authored-by: Semir Patel <semir.patel@hashicorp.com>	10 months ago
Melissa Kam	7900544249	[CC-7063] Fetch HCP agent bootstrap config in Link reconciler (#20306) ... * Move config-dependent methods to separate package In order to reuse the fetching and file creation part of the bootstrap package, move the code that would cause cyclical dependencies to a different package. * Export needed bootstrap methods and variables Also add back validating persisted config and update tests. * Add support to check for just management token Add a new method that fetches the bootstrap configuration only if there isn't a valid management token file instead of checking for all the hcp-config files. * Pass data dir as a dependency to link controller The link controller needs to check the data directory for the hcp-config files. * Fetch bootstrap config for token in controller Load the management token when reconciling a link resource, which will fetch the agent boostrap configuration if the token is not already persisted locally. Skip this step if the cluster is in read-only mode. * Validate resource ID format in link creation * Handle unauthorized and forbidden errors Check for 401 and 403s when making GNM requests, exit bootstrap fetch loop and return specific failure statuses for link. * Move test function to a testing file * Log load and status write errors	10 months ago
Melissa Kam	98c9702ba3	[CC-7031] Add initialization support to resource controllers (#20138) ... * Add Initializer to the controller The Initializer adds support for running any required initialization steps when the controller is first started. * Implement HCP Link initializer The link initializer will create a Link resource if the cloud configuration has been set. * Simplify retry logic and testing * Remove internal retry, replace with logging logic	10 months ago
Nick Cellino	fe678e9da1	Sync cluster attributes from GNM to Link resource (#20158) ... * Add 'GetCluster' function to HCP client * Sync cluster data inside Link controller * Add access mode to HCP Link * Sync AccessLevel property * Fix imports and remove outdated comments * Switch accessMode to access level * Add comment around HCPClientFn * Fix spacing in link.proto * Add helper for writing status. Fix reconciliation loop	10 months ago
Nick Cellino	0deebaf637	Add Link resource type and controller skeleton (#19788) ... * Add HCCLink resource type * Register HCCLink resource type with basic validation * Add validation for required fields * Add test for default ACLs * Add no-op controller for HCCLink * Add resource-apis semantic validation check in hcclink controller * Add copyright headers * Rename HCCLink to Link * Add hcp_cluster_url to link proto * Update 'disabled' reason with more detail * Update link status name to consul.io/hcp/link * Change link version from v1 to v2 * Use feature flag/experiment to enable v2 resources with HCP	11 months ago