117 Commits (4116a143e0d27408ac57085daaa780f8087bb0b6)

Author	SHA1	Message	Date
Kyle Havlovitz	74dc50a771	vendor: Update vault api package	4 years ago
Daniel Nephin	0c87cf468c	Update go-metrics dependencies, to use metrics.Default()	4 years ago
Hans Hasselberg	a932aafc91	add primary keys to list keyring (#8522) ... During gossip encryption key rotation it would be nice to be able to see if all nodes are using the same key. This PR adds another field to the json response from `GET v1/operator/keyring` which lists the primary keys in use per dc. That way an operator can tell when a key was successfully setup as primary key. Based on https://github.com/hashicorp/serf/pull/611 to add primary key to list keyring output: ```json [ { "WAN": true, "Datacenter": "dc2", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 6, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6 }, "NumNodes": 6 }, { "WAN": false, "Datacenter": "dc2", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 8, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "NumNodes": 8 }, { "WAN": false, "Datacenter": "dc1", "Segment": "", "Keys": { "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 3, "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "PrimaryKeys": { "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8 }, "NumNodes": 8 } ] ``` I intentionally did not change the CLI output because I didn't find a good way of displaying this information. There are a couple of options that we could implement later: * add a flag to show the primary keys * add a flag to show json output Fixes #3393.	4 years ago
s-christoff	102b7e55da	Update Go-Metrics 0.3.4 (#8478)	4 years ago
Kyle Havlovitz	f4efd53d57	vendor: Update github.com/armon/go-metrics to v0.3.3	4 years ago
Matt Keeler	a6a1a0e3d6	Update mapstructure to v1.3.3 (#8361) ... This was done in preparation for another PR where I was running into https://github.com/mitchellh/mapstructure/issues/202 and implemented a fix for the library.	4 years ago
R.B. Boyer	e853368c23	gossip: Avoid issue where two unique leave events for the same node could lead to infinite rebroadcast storms (#8343) ... bump serf to v0.9.3 to include fix for https://github.com/hashicorp/serf/pull/606	4 years ago
Pierre Souchay	20d1ea7d2d	Upgrade go-connlimit to v0.3.0 / return http 429 on too many connections (#8221) ... Fixes #7527 I want to highlight this and explain what I think the implications are and make sure we are aware: * `HTTPConnStateFunc` closes the connection when it is beyond the limit. `Close` does not block. * `HTTPConnStateFuncWithDefault429Handler(10 * time.Millisecond)` blocks until the following is done (worst case): 1) `conn.SetDeadline(10*time.Millisecond)` so that 2) `conn.Write(429error)` is guaranteed to timeout after 10ms, so that the http 429 can be written and 3) `conn.Close` can happen The implication of this change is that accepting any new connection is worst case delayed by 10ms. But only after a client reached the limit already.	4 years ago
Hans Hasselberg	95c027a3ea	Update gopsutil (#8208) ... https://github.com/shirou/gopsutil/pull/895 is merged and fixes our problem. Time to update. Since there is no new version just yet, updating to the sha.	4 years ago
Matt Keeler	e9835610f3	Add a test for go routine leaks ... This is in its own separate package so that it will be a separate test binary that runs thus isolating the go runtime from other tests and allowing accurate go routine leak checking. This test would ideally use goleak.VerifyTestMain but that will fail 100% of the time due to some architectural things (blocking queries and net/rpc uncancellability). This test is not comprehensive. We should enable/exercise more features and more cluster configurations. However its a start.	5 years ago
R.B. Boyer	c63c994b04	connect: upgrade github.com/envoyproxy/go-control-plane to v0.9.5 (#8165)	5 years ago
Paul Banks	f6ac08be04	state: track changes so that they may be used to produce change events	5 years ago
Daniel Nephin	98d271bee9	Update google.golang.org/api and stretchr/testify ... To match the versions used in enterprise, should slightly reduce the chances of getting a merge conflict when using `go.mod`.	5 years ago
Daniel Nephin	db74f09b6b	Update protobuf and golang.org/x/... vendor ... Partially extracted from #7547 Updates protobuf to the most recent in the 1.3.x series, and updates golang.org/x/sys to a7d97aace0b0 because of https://github.com/shirou/gopsutil/issues/853 prevents updating to a more recent version. This breaking change in x/sys also prevents us from getting a newer version of x/net. In the future, if gopsutil is not patched, we may want to run a fork version of gopsutil so that we can update both x/net and x/sys.	5 years ago
R.B. Boyer	1efafd7523	acl: add auth method for JWTs (#7846)	5 years ago
Mike Morris	291e6af33a	vendor: revert golang.org/x/sys bump to avoid FreeBSD regression (#7780)	5 years ago
Hans Hasselberg	c4093c87cc	agent: don't let left nodes hold onto their node-id (#7747)	5 years ago
Matt Keeler	daec810e34	Merge pull request #7714 from hashicorp/oss-sync/msp-agent-token	5 years ago
Matt Keeler	55050beedb	Update go-discover dependency (#7731)	5 years ago
Matt Keeler	8c545b5206	Update mapstructure to v1.2.3 ... This release contains a fix to prevent duplicate keys in the Metadata after decoding where the output value contains pointer fields.	5 years ago
R.B. Boyer	b989967791	cli: ensure that 'snapshot save' is fsync safe and also only writes to the requested file on success (#7698)	5 years ago
R.B. Boyer	5f1518c37c	cli: fix usage of gzip.Reader to better detect corrupt snapshots during save/restore (#7697)	5 years ago
Daniel Nephin	50d73c2674	Update github.com/joyent/triton-go to latest ... There was an RSA private key used for testing included in the old version. This commit updates it to a version that does not include the key so that the key is not detected by tools which scan the Consul binary for private keys. Commands run: go get github.com/joyent/triton-go@6801d15b779f042cfd821c8a41ef80fc33af9d47 make update-vendor	5 years ago
Daniel Nephin	5023a3b178	cli: send requested help text to stdout ... This behaviour matches the GNU CLI standard: http://www.gnu.org/prep/standards/html_node/_002d_002dhelp.html	5 years ago
R.B. Boyer	6adad71125	wan federation via mesh gateways (#6884) ... This is like a Möbius strip of code due to the fact that low-level components (serf/memberlist) are connected to high-level components (the catalog and mesh-gateways) in a twisty maze of references which make it hard to dive into. With that in mind here's a high level summary of what you'll find in the patch: There are several distinct chunks of code that are affected: * new flags and config options for the server * retry join WAN is slightly different * retry join code is shared to discover primary mesh gateways from secondary datacenters * because retry join logic runs in the *agent* and the results of that operation for primary mesh gateways are needed in the *server* there are some methods like `RefreshPrimaryGatewayFallbackAddresses` that must occur at multiple layers of abstraction just to pass the data down to the right layer. * new cache type `FederationStateListMeshGatewaysName` for use in `proxycfg/xds` layers * the function signature for RPC dialing picked up a new required field (the node name of the destination) * several new RPCs for manipulating a FederationState object: `FederationState:{Apply,Get,List,ListMeshGateways}` * 3 read-only internal APIs for debugging use to invoke those RPCs from curl * raft and fsm changes to persist these FederationStates * replication for FederationStates as they are canonically stored in the Primary and replicated to the Secondaries. * a special derivative of anti-entropy that runs in secondaries to snapshot their local mesh gateway `CheckServiceNodes` and sync them into their upstream FederationState in the primary (this works in conjunction with the replication to distribute addresses for all mesh gateways in all DCs to all other DCs) * a "gateway locator" convenience object to make use of this data to choose the addresses of gateways to use for any given RPC or gossip operation to a remote DC. This gets data from the "retry join" logic in the agent and also directly calls into the FSM. * RPC (`:8300`) on the server sniffs the first byte of a new connection to determine if it's actually doing native TLS. If so it checks the ALPN header for protocol determination (just like how the existing system uses the type-byte marker). * 2 new kinds of protocols are exclusively decoded via this native TLS mechanism: one for ferrying "packet" operations (udp-like) from the gossip layer and one for "stream" operations (tcp-like). The packet operations re-use sockets (using length-prefixing) to cut down on TLS re-negotiation overhead. * the server instances specially wrap the `memberlist.NetTransport` when running with gateway federation enabled (in a `wanfed.Transport`). The general gist is that if it tries to dial a node in the SAME datacenter (deduced by looking at the suffix of the node name) there is no change. If dialing a DIFFERENT datacenter it is wrapped up in a TLS+ALPN blob and sent through some mesh gateways to eventually end up in a server's :8300 port. * a new flag when launching a mesh gateway via `consul connect envoy` to indicate that the servers are to be exposed. This sets a special service meta when registering the gateway into the catalog. * `proxycfg/xds` notice this metadata blob to activate additional watches for the FederationState objects as well as the location of all of the consul servers in that datacenter. * `xds:` if the extra metadata is in place additional clusters are defined in a DC to bulk sink all traffic to another DC's gateways. For the current datacenter we listen on a wildcard name (`server.<dc>.consul`) that load balances all servers as well as one mini-cluster per node (`<node>.server.<dc>.consul`) * the `consul tls cert create` command got a new flag (`-node`) to help create an additional SAN in certs that can be used with this flavor of federation.	5 years ago
Matt Keeler	80ed304e04	Run make update-vendor and fixup various go.sum files ... go mod tidy removes these lines because we have a replace directive	5 years ago
Matt Keeler	3c6d9516bc	Bump `api` and `sdk` module versions ... api -> v1.4.0 sdk -> v0.4.0	5 years ago
R.B. Boyer	73ba5d9990	make the TestRPC_RPCMaxConnsPerClient test less flaky (#7255)	5 years ago
Matt Keeler	58befa754b	Update to miekg/dns v1.1.26 (#7252)	5 years ago
Hans Hasselberg	649ffcb66f	memberlist: vendor v0.1.6 to pull in new state: stateLeft (#7184)	5 years ago
Matt Keeler	d1fcf1e950	Add replace directive to prevent contacting istio.io during the… (#7194) ... They keep having TLS handshake timeouts. Its pointed at github instead.	5 years ago
Hans Hasselberg	5531678e9e	Security fixes (#7182) ... * Mitigate HTTP/RPC Services Allow Unbounded Resource Usage Fixes #7159. Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com> Co-authored-by: Paul Banks <banks@banksco.de>	5 years ago
Chris Piraino	401221de58	Allow users to configure either unstructured or JSON logging (#7130) ... * hclog Allow users to choose between unstructured and JSON logging	5 years ago
Michel Vocks	6681be918a	api: add option to set TLS options in-memory for API client (#7093) ... This PR adds the option to set in-memory certificates to the API client instead of requiring the certificate to be stored on disk in a file. This allows us to define API client TLS options per Consul secret backend in Vault. Related issue hashicorp/vault#4800	5 years ago
Hans Hasselberg	9c1361c02b	raft: update raft to v1.1.2 (#7079) ... * update raft * use hclogger for raft.	5 years ago
Wim	4f5d5020b8	dns: fix memoryleak by upgrading outdated miekg/dns (#6748) ... * Add updated github.com/miekg/dns to go modules * Add updated github.com/miekg/dns to vendor * Fix github.com/miekg/dns api breakage * Decrease size when trimming UDP packets Need more room for the header(?), if we don't decrease the size we get an "overflow unpacking uint32" from the dns library * Fix dns truncate tests with api changes * Make windows build working again. Upgrade x/sys and x/crypto and vendor This upgrade is needed because of API breakage in x/sys introduced by the minimal x/sys dependency of miekg/dns This API breakage has been fixed in commit 855e68c859	5 years ago
Worming	25da8d9cfb	accept go get command with go 1.13 (#6938)	5 years ago
Mike Morris	66b8c20990	Bump go-discover to support EC2 Metadata Service v2 (#6865) ... Refs https://github.com/hashicorp/go-discover/pull/128 * deps: add replace directive for gocheck Transitive dep, source at https://launchpad.net/gocheck indicates project moved. This also avoids a dependency on bzr when fetching modules. Refs https://github.com/hashicorp/consul/pull/6818 * deps: make update-vendor * test: update retry-join expected names from go-discover	5 years ago
Paul Banks	cd1b613352	connect: Add AWS PCA provider (#6795) ... * Update AWS SDK to use PCA features. * Add AWS PCA provider * Add plumbing for config, config validation tests, add test for inheriting existing CA resources created by user * Unparallel the tests so we don't exhaust PCA limits * Merge updates * More aggressive polling; rate limit pass through on sign; Timeout on Sign and CA create * Add AWS PCA docs * Fix Vault doc typo too * Doc typo * Apply suggestions from code review Co-Authored-By: R.B. Boyer <rb@hashicorp.com> Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com> * Doc fixes; tests for erroring if State is modified via API * More review cleanup * Uncomment tests! * Minor suggested clean ups	5 years ago
Matt Keeler	7f00fc60f1	Dependency: Update github.com/hashicorp/go-memdb to v1.0.3 (#6626)	5 years ago
Sarah Christoff	5e26971864	Prune Unhealthy Agents (#6571) ... * Add -prune flag to ForceLeave	5 years ago
R.B. Boyer	796de297c8	connect: intermediate CA certs generated with the vault provider lack URI SANs (#6491) ... This only affects vault versions >=1.1.1 because the prior code accidentally relied upon a bug that was fixed in https://github.com/hashicorp/vault/pull/6505 The existing tests should have caught this, but they were using a vendored copy of vault version 0.10.3. This fixes the tests by running an actual copy of vault instead of an in-process copy. This has the added benefit of changing the dependency on vault to just vault/api. Also update VaultProvider to use similar SetIntermediate validation code as the ConsulProvider implementation.	5 years ago
Matt Keeler	ccfdb99d00	Update memberlist to v0.1.5 (#6479)	5 years ago
Aaron Bennett	430dab8c37	update dependencies for enterprise change (#6395)	5 years ago
R.B. Boyer	bec24ed199	mod: bump main module to api v1.2.0 and sdk v0.2.0	5 years ago
Matt Keeler	a2ddaaca0a	Update to google.golang.org/grpc v1.23.0 (#6320)	5 years ago
Paul Banks	e87cef2bb8	Revert "connect: support AWS PCA as a CA provider" (#6251) ... This reverts commit 3497b7c00d.	5 years ago
Todd Radel	3497b7c00d	connect: support AWS PCA as a CA provider (#6189) ... Port AWS PCA provider from consul-ent	5 years ago
R.B. Boyer	c6c4a2251a	Merge Consul OSS branch master at commit b3541c4f34	5 years ago
Jeff Mitchell	94c73d0c92	Chunking support (#6172) ... * Initial chunk support This uses the go-raft-middleware library to allow for chunked commits to the KV	5 years ago
Matt Keeler	d7fe8befa9	Update go-bexpr (#6190) ... * Update go-bexpr to v0.1.1 This brings in: • `in`/`not in` operators to do substring matching • `matches` / `not matches` operators to perform regex string matching. * Add the capability to auto-generate the filtering selector ops tables for our docs	5 years ago
Hans Hasselberg	f13fe4b304	agent: transfer leadership when establishLeadership fails (#5247)	6 years ago
Paul Banks	ffcfdf29fc	Upgrade xDS (go-control-plane) API to support Envoy 1.10. (#5872) ... * Upgrade xDS (go-control-plane) API to support Envoy 1.10. This includes backwards compatibility shim to work around the ext_authz package rename in 1.10. It also adds integration test support in CI for 1.10.0. * Fix go vet complaints * go mod vendor * Update Envoy version info in docs * Update website/source/docs/connect/proxies/envoy.md	6 years ago
Kyle Havlovitz	1d9b8e1860	vendor: update memberlist	6 years ago
Alvin Huang	1452f4df33	bump hashicorp/yamux (#5776) ... * bump hashicorp/yamux * go mod tidy to remove older yamux	6 years ago
Matt Keeler	0ac6b6faba	Fix up the MapWalk function so that it properly handles nested map[interface{}]interface{} (#5774)	6 years ago
Paul Banks	421ecd32fc	Connect: allow configuring Envoy for L7 Observability (#5558) ... * Add support for HTTP proxy listeners * Add customizable bootstrap configuration options * Debug logging for xDS AuthZ * Add Envoy Integration test suite with basic test coverage * Add envoy command tests to cover new cases * Add tracing integration test * Add gRPC support WIP * Merged changes from master Docker. get CI integration to work with same Dockerfile now * Make docker build optional for integration * Enable integration tests again! * http2 and grpc integration tests and fixes * Fix up command config tests * Store all container logs as artifacts in circle on fail * Add retries to outer part of stats measurements as we keep missing them in CI * Only dump logs on failing cases * Fix typos from code review * Review tidying and make tests pass again * Add debug logs to exec test. * Fix legit test failure caused by upstream rename in envoy config * Attempt to reduce cases of bad TLS handshake in CI integration tests * bring up the right service * Add prometheus integration test * Add test for denied AuthZ both HTTP and TCP * Try ANSI term for Circle	6 years ago
R.B. Boyer	e47d7eeddb	acl: adding support for kubernetes auth provider login (#5600) ... * auth providers * binding rules * auth provider for kubernetes * login/logout	6 years ago
Matt Keeler	e55afa9252	Update go-msgpack version (#5683) ... Fixes #4673 Supercedes: #5677 There was an error decoding `map[string]string` values due to Go strings being immutable. This was fixes in our go-msgpack fork.	6 years ago
Matt Keeler	afa1cc98d1	Implement data filtering of some endpoints (#5579) ... Fixes: #4222 # Data Filtering This PR will implement filtering for the following endpoints: ## Supported HTTP Endpoints - `/agent/checks` - `/agent/services` - `/catalog/nodes` - `/catalog/service/:service` - `/catalog/connect/:service` - `/catalog/node/:node` - `/health/node/:node` - `/health/checks/:service` - `/health/service/:service` - `/health/connect/:service` - `/health/state/:state` - `/internal/ui/nodes` - `/internal/ui/services` More can be added going forward and any endpoint which is used to list some data is a good candidate. ## Usage When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of: ``` <selector> == <value> <selector> != <value> <value> in <selector> <value> not in <selector> <selector> contains <value> <selector> not contains <value> <selector> is empty <selector> is not empty not <other expression> <expression 1> and <expression 2> <expression 1> or <expression 2> ``` Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library ## Other changes Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.	6 years ago
Freddy	a03392c906	Add additional raft metrics (#5628) ... * Add documentation for new raft metrics * Revendor raft from master	6 years ago
Paul Banks	8ae3447447	Pull go-discover to fix Sirupsen/logrus (#5598) ... * Pull go-discover to fix Sirupsen/logrus * Actually rename Sirupsen -> sirupsen in vendor (despite macOS) * Actually _actually_ rename Sirupsen -> sirupsen in vendor (despite macOS)	6 years ago
Hans Hasselberg	ac45b17482	fix remaining CI failures after Go 1.12.1 Upgrade (#5576)	6 years ago
Jeff Mitchell	f1fe406aa8	Update SDK version (#5572)	6 years ago
Jeff Mitchell	4243c3ae42	Move internal/ to sdk/ (#5568) ... * Move internal/ to sdk/ * Add a readme to the SDK folder	6 years ago
Jeff Mitchell	3d95636dd7	Update api/internal tags (#5565)	6 years ago
Jeff Mitchell	47c390025b	Convert to Go Modules (#5517) ... * First conversion * Use serf 0.8.2 tag and associated updated deps * * Move freeport and testutil into internal/ * Make internal/ its own module * Update imports * Add replace statements so API and normal Consul code are self-referencing for ease of development * Adapt to newer goe/values * Bump to new cleanhttp * Fix ban nonprintable chars test * Update lock bad args test The error message when the duration cannot be parsed changed in Go 1.12 (ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test. * Update another test as well * Bump travis * Bump circleci * Bump go-discover and godo to get rid of launchpad dep * Bump dockerfile go version * fix tar command * Bump go-cleanhttp	6 years ago