consul

Commit Graph

Author	SHA1	Message	Date
Dhia Ayachi	f2eed912b2	upgrade golangci-lint to v1.40.1 (#10276 ) Also: fix linter issue detected with newer version	4 years ago
R.B. Boyer	0f8cdfaeff	dev: adding a make target to regenerate envoy golden files (#10158 )	4 years ago
R.B. Boyer	85a718da63	xds: ensure that all envoyproxy/go-control-plane protobuf symbols are linked into the final binary (#10131 ) This ensures that if someone does include some extension Consul does not currently make use of, that extension is actually usable. Without linking these envoy protobufs into the main binary it can't round trip the escape hatches to send them down to envoy. Whenenver the go-control-plane library is upgraded next we just have to re-run 'make envoy-library'.	4 years ago
R.B. Boyer	39e4ae25ac	connect: connect CA Roots in the primary datacenter should use a SigningKeyID derived from their local intermediate (#9428 ) This fixes an issue where leaf certificates issued in primary datacenters using Vault as a Connect CA would be reissued very frequently (every ~20 seconds) because the logic meant to detect root rotation was errantly triggering. The hash of the rootCA was being compared against a hash of the intermediateCA and always failing. This doesn't apply to the Consul built-in CA provider because there is no intermediate in use in the primary DC. This is reminiscent of #6513	4 years ago
Kyle Havlovitz	01ce9f5b18	Update CI for leader renew CA test using Vault	4 years ago
Mike Morris	6c0907f494	test(vault): generate test coverage and upload to CodeCov (#8870 )	4 years ago
Alvin Huang	8b409529aa	ci: fix bindata autoupdate (#8801 ) * update bindata on ui-v2/ changes * Revert "Remove GIT_SHA environment variable" This reverts commit `f620f9aefd`. * Revert "Hardcode in an app version for ember-cli-app-version" This reverts commit `5ae493d79b`. * revert modtime change in #8712	4 years ago
Paul Banks	3b8125a24d	Update all the references in CI and makefile to the bindata file location	4 years ago
Paul Banks	e4db845246	Refactor uiserver to separate package, cleaner Reloading	4 years ago
Alvin Huang	87f6617eec	ci:add check for bindata_assetfs changes (#8712 ) * add check for bindata_assetfs changes * Remove GIT_SHA environment variable * Hardcode in an app version for ember-cli-app-version * change ' to " for CONSUL_UI_SETTINGS_PLACEHOLDER Co-authored-by: John Cowen <jcowen@hashicorp.com> Co-authored-by: hashicorp-ci <hashicorp-ci@users.noreply.github.com>	4 years ago
Daniel Nephin	f4791f0e82	ci: Add check-generated And remove makefile targets that can be accomplished with --always-make	4 years ago
Daniel Nephin	8836796d66	Add script to print available module updates	4 years ago
Alvin Huang	2a040342ba	remove GIT_DESCRIBE variable since we have a revision line (#8310 )	4 years ago
Daniel Nephin	e02ee13657	Make envoy integration tests a `go test` suite (#7842 ) * test/integration: only run against 1 envoy version These tests are slow enough that it seems unlikely that anyone is running multiple versions locally. If someone wants to, a for loop outside of run_test.sh should do the right thing. Remove unused vars. * Remove logic to iterate over test cases, run a single case * Add a golang runner for integration tests * Use build tags for envoy integration tests And add junit-xml report	5 years ago
Daniel Nephin	8029daa444	Add lint to makefile And support for setting build tags in CI lint job. Removed the unused 'test-ci' target.	5 years ago
Daniel Nephin	61ec7aa5c9	ci: Run all connect/ca tests from the integration suite To reduce the chance of some tests not being run because it does not match the regex passed to '-run'. Also document why some tests are allowed to be skipped on CI.	5 years ago
Daniel Nephin	3f8578f1e1	ci: Use golangci-lint for linting Using golangci-lint has a number of advantages: - adding new linters becomes much easier, its a couple lines of yaml config instead of more bash scripting - it enables whitelisting of issues using inline comments or regex - when running multiple linters less work is done. The parsed source can be reused by multiple linters - linters are run in parallel to reduce CI runtime.	5 years ago
R.B. Boyer	fdb6b80672	Switch to go 1.13.7 (#7262 ) - You can no longer cross submodule boundaries with ./... in go subcommands like `go list` or `go test`. The makefile and CI scripts were updated accordingly. - Also of note: `go mod vendor` now omits things build ignored.	5 years ago
R.B. Boyer	6404967034	add 'make go-mod-tidy' to serially run tidy on all submodules in the correct order (#7179 ) - also make go-mod-tidy a dependency of update-vendor	5 years ago
Matt Keeler	add988b665	Pass the git commits year through as an env var to the ui build (#7126 )	5 years ago
R.B. Boyer	98e0cbdebe	test: run both the TestVaultProvider and TestVaultCAProvider tests in CI (#7097 )	5 years ago
Kit Patella	0d61369fcd	Add CI test-integrations job for connect Vault CA provider (#6949 ) * vault-ca-provider: add make target and CI test-integration job for /agent/connect/ca/ Vault Provider tests * load env vars in the vault-ca-provider ci job * add checkout task, see if we need to attach workspace or not * ci: remove vault dependency from go-test job to ensure we only run the vault-provider tests in their job * ci: fix from RB and Alvin code review, add mod cache after checkout * ci: add CI context to make target and store test results * ci: fix whitespace * ci: create test results directory before we try to write to it	5 years ago
Matt Keeler	9d801d1dc2	ui: feature support templating for index.html (#6921 )	5 years ago
R.B. Boyer	7ac47d6f61	build: fix makefile and CI 'fmt' and 'vet' tasks (#6937 )	5 years ago
R.B. Boyer	734b6287b8	build: prevent 'make tools' from editing go.mod and go.sum (#6738 )	5 years ago
Matt Keeler	a31e7a5d58	Remove `gox` usage (#6596 )	5 years ago
Matt Keeler	28221f66f2	Use encoding/json instead of jsonpb even for protobuf types (#6572 ) This only works so long as we use simplistic protobuf types. Constructs such as oneof or Any types that require type annotations for decoding properly will fail hard but that is by design. If/when we want to use any of that we will probably need to consider a v2 API.	5 years ago
Matt Keeler	abed91d069	Generate JSON and Binary Marshalers for Protobuf Types (#6564 ) * Add JSON and Binary Marshaler Generators for Protobuf Types * Generate files with the correct version of gogo/protobuf I have pinned the version in the makefile so when you run make tools you get the right version. This pulls the version out of go.mod so it should remain up to date. The version at the time of this commit we are using is v1.2.1 * Fixup some shell output * Update how we determine the version of gogo This just greps the go.mod file instead of expecting the go mod cache to already be present * Fixup vendoring and remove no longer needed json encoder functions	5 years ago
Matt Keeler	51dcd126b7	Add support for implementing new requests with protobufs instea… (#6502 ) * Add build system support for protobuf generation This is done generically so that we don’t have to keep updating the makefile to add another proto generation. Note: anything not in the vendor directory and with a .proto extension will be run through protoc if the corresponding namespace.pb.go file is not up to date. If you want to rebuild just a single proto file you can do so with: make proto-rebuild PROTOFILES=<list of proto files to rebuild> Providing the PROTOFILES var will override the default behavior of finding all the .proto files. * Start adding types to the agent/proto package These will be needed for some other work and are by no means comprehensive. * Add ability to resolve/fixup the agentpb.ACLLinks structure in the state store. * Use protobuf marshalling of raft requests instead of msgpack for protoc generated types. This does not change any encoding of existing types. * Removed structs package automatically encoding with protobuf marshalling Instead the caller of raftApply that wants to opt-in to protobuf encoding will have to call `raftApplyProtobuf` * Run update-vendor to fixup modules.txt Nothing changed as far as dependencies go but the ordering of modules in that file depends on the time they are first seen and its not alphabetical. * Rename some things and implement the structs.RPCInfo interface bits agentpb.QueryOptions and agentpb.WriteRequest implement 3 of the 4 RPCInfo funcs and the new TargetDatacenter message type implements the fourth. * Use the right encoding function. * Renamed agent/proto package to agent/agentpb to prevent package name conflicts * Update modules.txt to fix ordering * Change blockingQuery to take in interfaces for the query options and meta * Add %T to error output. * Add/Update some comments	5 years ago
Alvin Huang	ac8f3c7431	change DOCKER_LOGIN env name to DOCKER_USER (#6451 )	5 years ago
Paul Banks	e87cef2bb8	Revert "connect: support AWS PCA as a CA provider" (#6251 ) This reverts commit `3497b7c00d`.	5 years ago
Todd Radel	3497b7c00d	connect: support AWS PCA as a CA provider (#6189 ) Port AWS PCA provider from consul-ent	5 years ago
Alvin Huang	3558f9cf6d	add dev docker image upload (#5879 ) * add dev docker image upload * remove the go cache since that isn't needed * add comment and image labels * get password from stdin	5 years ago
Matt Keeler	52c608df78	`make test-docker` (#6059 ) * Implement the test-docker make target Running tests within docker allows us to resource constrain them better to not take over our systems. Additionally it allows us to run the tests on linux instead of the host OS which often times is macOS. * Use GOMAXPROCS instead of -p * Add a comment about docker cpus	5 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558 ) * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
Matt Keeler	222afeae4c	Move the watch package into the api module (#5664 ) * Move the watch package into the api module It was already just a thin wrapper around the API anyways. The biggest change was to the testing. Instead of using a test agent directly from the agent package it now uses the binary on the PATH just like the other API tests. The other big changes were to fix up the connect based watch tests so that we didn’t need to pull in the connect package (and therefore all of Consul)	6 years ago
Matt Keeler	c6be3b525b	Build System Fixes for Go Modules (#5655 ) * Docker based builds can now use the module cache * Simplify building the consul-dev docker image. * Make sure to pull the latest consul image. * Allow selecting base image version for the dev image	6 years ago
Freddy	b2c12550d7	Remove old UI, option to use it, and its build processes	6 years ago
Alvin Huang	ffa123b73e	download tools without library	6 years ago
Paul Banks	11362e77be	build: use only version tags in version output now api is tagged too (#5622 ) * build: use only version tags in version output now api is tagged too Fixes #5621 Since we now have api package tags, our build tooling was picking up api tag when working out version to bake into builds. This fixes it by restricting to only tags that start with `v`. Before: ``` $ make version Version: 1.4.4 Version + release: 1.4.4-dev Version + git: api/v1.0.1-90-g3ce60db0c Version + release + git: api/v1.0.1-90-g3ce60db0c-dev (`3ce60db0c`) ``` After: ``` $ make version Version: 1.4.4 Version + release: 1.4.4-dev Version + git: v1.4.4-126-g3ce60db0c Version + release + git: v1.4.4-126-g3ce60db0c-dev (`3ce60db0c`) ``` * Update GNUmakefile	6 years ago
Alvin Huang	8e07c63c54	removing gocov packages (#5534 ) * removing gocov packages * revise cov make target	6 years ago
R.B. Boyer	27446a19e2	remove remaining references to govendor and vendorfmt (#5587 )	6 years ago
Hans Hasselberg	ac45b17482	fix remaining CI failures after Go 1.12.1 Upgrade (#5576 )	6 years ago
Mitchell Hashimoto	f76022fa63	CA Provider Plugins (#4751 ) This adds the `agent/connect/ca/plugin` library for consuming/serving Connect CA providers as [go-plugin](https://github.com/hashicorp/go-plugin) plugins. This does not wire this up in any way to Consul itself, so this will not enable using these plugins yet. ## Why? We want to enable CA providers to be pluggable without modifying Consul so that any CA or PKI system can potentially back the Connect certificates. This CA system may also be used in the future for easier bootstrapping and internal cluster security. ### go-plugin The benefit of `go-plugin` is that for the plugin consumer, the fact that the interface implementation is communicating over multi-process RPC is invisible. Internals of Consul will continue to just use `ca.Provider` interface implementations as if they're local. For plugin _authors_, they simply have to implement the interface. The network/transport/process management issues are handled by go-plugin itself. The CA provider plugins support both `net/rpc` and gRPC transports. This enables easy authoring in any language. go-plugin handles the actual protocol handshake and connection. This is just a feature of go-plugin. `go-plugin` is already in production use for years by Packer, Terraform, Nomad, Vault, and Sentinel. We've shown stability for both desktop and server-side software. It is very mature. ## Implementation Details ### `map[string]interface{}` The `Configure` method passes a `map[string]interface{}`. This map contains only Go primitives and containers of primitives (no funcs, chans, etc.). For `net/rpc` we encode as-is using Gob. For gRPC we marshal to JSON and transmit as a `bytes` type. This is the same approach we take with Vault and other software. Note that this is just the transport protocol, the end software views it fully decoded. ### `x509.Certificate` and `CertificateRequest` We transmit the raw ASN.1 bytes and decode on the other side. Unit tests are verifying we get the same cert/csrs across the wire. ### Testing `go-plugin` exposes test helpers that enable testing the full plugin RPC over real loopback network connections. We test all endpoints for success and error for both `net/rpc` and gRPC. ### Vendoring This PR doesn't introduce vendoring for two reasons: 1. @banks's `f-envoy` branch introduces a lot of these and I didn't want conflict. 2. The library isn't actually used yet so it doesn't introduce compile-time errors (it does introduce test errors). ## Next Steps With this in place, we need to figure out the proper way to actually hook these up to Consul, load them, etc. This discussion can happen elsewhere, since regardless of approach this plugin library implementation is the exact same.	6 years ago
Matt Keeler	9819683653	Single quote a directory (#4846 ) Allows building the dev docker container to work when you have spaces in your cwd.	6 years ago
Matt Keeler	18b29c45c4	New ACLs (#4791 ) This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week. Description At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers. On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though. Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though. All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management. Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are: A server running the new system must still support other clients using the legacy system. A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system. The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode. So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.	6 years ago
Freddy	524192c919	Improve resilience of api pkg tests (#4676 ) * Add function to wait for serfHealth in api tests * Disable connect when creating semaphore test clients * Wait for serfHealth when creating sessions in their tests * Add helper functions to create lock/semaphore sessions without checks * Log passing tests to prevent timeout in Travis due to lack of output	6 years ago
Freddy	b2032b05be	Add script and makefile goal to help debug flaky tests	6 years ago
Pierre Souchay	80b6653a8e	Keep same parameters on retry so results can be cached by go test (#4627 )	6 years ago
Pierre Souchay	cec5d72396	BUGFIX: Unit test relying on WaitForLeader() did not work due to wrong test (#4472 ) - Improve resilience of testrpc.WaitForLeader() - Add additionall retry to CI - Increase "go test" timeout to 8m - Add wait for cluster leader to several tests in the agent package - Add retry to some tests in the api and command packages	6 years ago

1 2 3

137 Commits (da31e0449ea6323956f8ebc942a6d76332c52e27)