2203 Commits (4075629319af1117c6d35d2f9e00e0bc555e8f48)

Author	SHA1	Message	Date
Daniel Nephin	3570ce6566	Merge pull request #7948 from hashicorp/dnephin/buffer-test-logs ... testutil: NewLogBuffer - buffer logs until a test fails	4 years ago
Matt Keeler	3c09482864	Merge pull request #8311 from hashicorp/bugfix/auto-encrypt-token-update	4 years ago
Daniel Nephin	a33a7a6fe2	Merge pull request #8344 from hashicorp/dnephin/fix-flakes-in-stream ... stream: handle empty event in TestEventSnapshot	4 years ago
Daniel Nephin	51efba2c7d	testutil: NewLogBuffer - buffer logs until a test fails ... Replaces #7559 Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case. Attaching log output from background goroutines also cause data races. If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race. The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long. This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful. All of the logs are printed from the test goroutine, so they should be associated with the correct test. Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly. Related: https://github.com/golang/go/issues/38458 (may be fixed in go1.15) https://github.com/golang/go/issues/38382#issuecomment-612940030	4 years ago
Matt Keeler	12acdd7481	Disable background cache refresh for Connect Leaf Certs ... The rationale behind removing them is that all of our own code (xDS, builtin connect proxy) use the cache notification mechanism. This ensures that the blocking fetch behind the scenes is always executing. Therefore the only way you might go to get a certificate and have to wait is when 1) the request has never been made for that cert before or 2) you are using the v1/agent/connect/ca/leaf API for retrieving the cert yourself. In the first case, the refresh change doesn’t alter the behavior. In the second case, it can be mitigated by using blocking queries with that API which just like normal cache notification mechanism will cause the blocking fetch to be initiated and to get leaf certs as soon as needed. If you are not using blocking queries, or Envoy/xDS, or the builtin connect proxy but are retrieving the certs yourself then the HTTP endpoint might take a little longer to respond. This also renames the RefreshTimeout field on the register options to QueryTimeout to more accurately reflect that it is used for any type that supports blocking queries.	4 years ago
Matt Keeler	9da8c51ac5	Fix issue with changing the agent token causing failure to renew the auto-encrypt certificate ... The fallback method would still work but it would get into a state where it would let the certificate expire for 10s before getting a new one. And the new one used the less secure RPC endpoint. This is also a pretty large refactoring of the auto encrypt code. I was going to write some tests around the certificate monitoring but it was going to be impossible to get a TestAgent configured in such a way that I could write a test that ran in less than an hour or two to exercise the functionality. Moving the certificate monitoring into its own package will allow for dependency injection and in particular mocking the cache types to control how it hands back certificates and how long those certificates should live. This will allow for exercising the main loop more than would be possible with it coupled so tightly with the Agent.	4 years ago
Daniel Nephin	1910e2a246	checks: wait for goroutine to complete ... CheckAlias already had a waitGroup, but the Add() call was happening too late, which was causing a race in tests. The add must happen before the goroutine is started. CheckHTTP did not have a waitGroup, so I added it to match CheckAlias. It looks like a lot of the implementation could be shared, and may not need all of channel, waitgroup and bool, but I will leave that refactor for another time.	4 years ago
Daniel Nephin	9482961b1c	stream: handle empty event in TestEventSnapshot ... When the race detector is enabled we see this test fail occasionally. The reordering of execution seems to make it possible for the snapshot splice to happen before any events are published to the topicBuffers. We can handle this case in the test the same way it is handled by a subscription, by proceeding to the next event.	4 years ago
Daniel Nephin	57e00b0fd5	Merge pull request #8245 from hashicorp/dnephin/use-not-modified-in-cache ... agent/cache: Use AllowNotModified in CatalogListServices	4 years ago
Daniel Nephin	29571465e1	Merge pull request #8290 from hashicorp/dnephin/watch-decode ... watch: fix script watches with single arg	4 years ago
Daniel Nephin	ecccb30690	state: update calls that are no longer state methods ... In a previous commit these methods were changed to functions, so remove the Store paramter.	4 years ago
Daniel Nephin	2008884241	state: un-method funcs that don't use their receiver ... This change was mostly automated with the following First generate a list of functions with: git grep -o 'Store) \([^(]\+\)(tx \*txn' ./agent/consul/state | awk '{print $2}' | grep -o '^[^(]\+' Then the list was curated a bit with trial/error to remove and add funcs as necessary. Finally the replacement was done with: dir=agent/consul/state file=${1-funcnames} while read fn; do echo "$fn" sed -i -e "s/(s \*Store) $fn(/$fn(/" $dir/*.go sed -i -e "s/s\.$fn(/$fn(/" $dir/*.go sed -i -e "s/s\.store\.$fn(/$fn(/" $dir/*.go done < $file	4 years ago
Daniel Nephin	63b153df8c	store: convert methods that don't use their receiver to functions ... Making these functions allows them to be used without introducing an artificial dependency on the struct. Many of these will be called from streaming Event processors, which do not have a store. This change is being made ahead of the streaming work to get to reduce the size of the streaming diff.	4 years ago
André	3bc27df844	minor: fix docstring of DNSOnlyPassing (#8318) ... In runtime.go it had "duration" but it is actually a boolean.	4 years ago
Daniel Nephin	2ec3760b70	agent/cache: Use AllowNotModifiedResponse in CatalogListServices ... Co-authored-by: Pierre Souchay <pierresouchay@users.noreply.github.com>	4 years ago
Daniel Nephin	dbb8d14728	agent/cache: Update some docstrings	4 years ago
Daniel Nephin	c07acbeb6b	stream: Add forceClose and refactor subscription filtering ... Move the subscription context to Next. context.Context should generally never be stored in a struct because it makes that struct only valid while the context is valid. This is rarely obvious from the caller. Adds a forceClosed channel in place of the old context, and uses the new context as a way for the caller to stop the Subscription blocking. Remove some recursion out of bufferImte.Next. The caller is already looping so we can continue in that loop instead of recursing. This ensures currentItem is updated immediately (which probably does not matter in practice), and also removes the chance that we overflow the stack. NextNoBlock and FollowAfter do not need to handle bufferItem.Err, the caller already handles it. Moves filter to a method to simplify Next, and more explicitly separate filtering from looping. Also improve some godoc Only unwrap itemBuffer.Err when necessary	4 years ago
Daniel Nephin	f19f8e99bb	stream: Improve docstrings ... Also rename ResumeStrema to EndOfEmptySnapshot to be more consistent with other framing events Co-authored-by: Paul Banks <banks@banksco.de>	4 years ago
Daniel Nephin	fc1c2ae412	stream: change Topic to an interface ... Consumers of the package can decide on which type to use for the Topic. In the future we may use a gRPC type for the topic.	4 years ago
Daniel Nephin	6fa36e3aee	state: Move change processing out of EventPublisher ... EventPublisher was receiving TopicHandlers, which had a couple of problems: - ChangeProcessors were being grouped by Topic, but they completely ignored the topic and were performed on every change - ChangeProcessors required EventPublisher to be aware of database changes By moving ChangeProcesors out of EventPublisher, and having Publish accept events instead of changes, EventPublisher no longer needs to be aware of these things. Handlers is now only SnapshotHandlers, which are still mapped by Topic. Also allows us to remove the small 'db' package that had only two types. They can now be unexported types in state.	4 years ago
Daniel Nephin	48c766d2c6	server: Abandom state store to shutdown EventPublisher ... So that we don't leak goroutines	4 years ago
Daniel Nephin	889c57fd2d	stream: unexport identifiers ... Now that EventPublisher is part of stream a lot of the internals can be hidden	4 years ago
Daniel Nephin	4fa0fdc0e0	stream: Move EventPublisher to stream package ... The EventPublisher is the central hub of the PubSub system. It is toughly coupled with much of stream. Some stream internals were exported exclusively for EventPublisher. The two Subscribe cases (with or without index) were also awkwardly split between two packages. By moving EventPublisher into stream they are now both in the same package (although still in different files).	4 years ago
Daniel Nephin	489876c86b	state: Make handleACLUpdate async once again ... So that we keep as much as possible out of the FSM commit hot path.	4 years ago
Daniel Nephin	5f9db94956	state: Use interface for Txn ... Also store the index in Changes instead of the Txn. This change is in preparation for movinng EventPublisher to the stream package, and making handleACLUpdates async once again.	4 years ago
Daniel Nephin	a709ed1ab5	stream.Subscription unexport fields and additiona docstrings	4 years ago
Daniel Nephin	17b833b4c9	Add a context for stopping EventPublisher goroutine	4 years ago
Daniel Nephin	2c8342f115	EventPublisher: Make Unsubscribe a function on Subscription ... It is critical that Unsubscribe be called with the same pointer to a SubscriptionRequest that was used to create the Subscription. The docstring made that clear, but it sill allowed a caler to get it wrong by creating a new SubscriptionRequest. By hiding this detail from the caller, and only exposing an Unsubscribe method, it should be impossible to fail to Unsubscribe. Also update some godoc strings.	4 years ago
Daniel Nephin	1622bb3a45	EventPublisher: handleACL changes synchronously ... Use a separate lock for subscriptions.ByToken to allow it to happen synchronously in the commit flow. This removes the need to create a new txn for the goroutine, and removes the need for EventPublisher to contain a reference to DB.	4 years ago
Daniel Nephin	effab15131	stream.EventSnapshot: reduce the fields on the struct ... Many of the fields are only needed in one place, and by using a closure they can be removed from the struct. This reduces the scope of the variables making it esier to see how they are used.	4 years ago
Daniel Nephin	a5cf933fe8	stream.EventBuffer: Seed the fuzz test with time.Now() ... Otherwise the test will run with exactly the same values each time. By printing the seed we can attempt to reproduce the test by adding an env var to override the seed	4 years ago
Daniel Nephin	bbe7272d8e	state: memdb_wrapper.go -> memdb.go ... Renaming in a separate commit so that git can merge changes to the file.	4 years ago
Daniel Nephin	2a8a8f7b8d	state: publish changes from Commit ... Make topicRegistry use functions instead of unbound methods Use a regular memDB in EventPublisher to remove a reference cycle Removes the need for EventPublisher to use a store	4 years ago
Daniel Nephin	f5ecd5de5f	EventPublisher: docstrings and getTopicBuffer ... also rename commitCh -> publishCh	4 years ago
Daniel Nephin	555cfe52d9	ProcessChanges: use stream.Event ... Also remove secretHash, which was used to hash tokens. We don't expose these tokens anywhere, so we can use the string itself instead of a Hash. Fix acl_events_test.go for storing a structs type.	4 years ago
Daniel Nephin	4e0bc8013b	stream: Use local types for Event Topic SubscriptionRequest	4 years ago
Daniel Nephin	aacd514dca	Rename stream_publisher.go -> event_publisher.go	4 years ago
Daniel Nephin	c0b0109e80	Add streaming package with Subscription and Snapshot components. ... The remaining files from 7965767de0bd62ab07669b85d6879bd5f815d157 Co-authored-by: Paul Banks <banks@banksco.de>	4 years ago
Matt Keeler	8beca1cb8d	Add ability for notifications when one of the agent tokens is updated (#8301) ... Co-authored-by: Chris Piraino <cpiraino@hashicorp.com>	4 years ago
Hans Hasselberg	496fb5fc5b	add support for envoy 1.14.4, 1.13.4, 1.12.6 (#8216)	4 years ago
Chris Piraino	4d857d117f	Set enterprise metadata after resolving the token (#8302) ... The token can encode enterprise metadata information, and we must make sure we set that on the reply so that we can correct filter ACLs.	4 years ago
Daniel Nephin	7b3f26e61d	watch: Allow args from different types ... Fixes a bug where specifying a slice of args with a single item was being converted to a string when config was loaded, causing an error.	4 years ago
Freddy	e72af87918	Add api mod support for /catalog/gateway-services (#8278)	4 years ago
Daniel Nephin	653c938edc	watch: extract makeWatchPlan to facilitate testing ... There is a bug in here now that slices in opaque config are unsliced. But to test that bug fix we need a function that can be easily tested.	4 years ago
R.B. Boyer	1eef096dfe	xds: version sniff envoy and switch regular expressions from 'regex' to 'safe_regex' on newer envoy versions (#8222) ... - cut down on extra node metadata transmission - split the golden file generation to compare all envoy version	4 years ago
Daniel Nephin	f22f3d300d	Merge pull request #8231 from hashicorp/dnephin/unembed-HTTPServer-Server ... agent/http: un-embed the http.Server	4 years ago
Daniel Nephin	df4088291c	agent/http: Update TestSetupHTTPServer_HTTP2 ... To remove the need to store the http.Server. This will allow us to remove the http.Server field from the HTTPServer struct.	4 years ago
Daniel Nephin	d98a4c1317	Merge pull request #8237 from hashicorp/dnephin/remove-acls-enabled-from-delegate ... Remove ACLsEnabled from delegate interface	4 years ago
Matt Keeler	4fb535ba48	Pass the Config and TLS Configurator into the AutoConfig constructor ... This is instead of having the AutoConfigBackend interface provide functions for retrieving them. NOTE: the config is not reloadable. For now this is fine as we don’t look at any reloadable fields. If that changes then we should provide a way to make it reloadable.	4 years ago
Matt Keeler	f2f32735ce	Rename (*Server).forward to (*Server).ForwardRPC ... Also get rid of the preexisting shim in server.go that existed before to have this name just call the unexported one.	4 years ago