consul/website/content/docs/architecture/coordinates.mdx

110 lines
3.9 KiB
Markdown

---
layout: docs
page_title: Network Coordinates
description: A Decentralized Network Coordinate System, with several improvements based on several follow-on papers.
---
# Network Coordinates
Consul uses a [network tomography](https://en.wikipedia.org/wiki/Network_tomography)
system to compute network coordinates for nodes in the cluster. These coordinates
allow the network round trip time to be estimated between any two nodes using a
very simple calculation. This allows for many useful applications, such as finding
the service node nearest a requesting node, or failing over to services in the next
closest datacenter.
All of this is provided through the use of the [Serf library](https://www.serf.io/).
Serf's network tomography is based on ["Vivaldi: A Decentralized Network Coordinate System"](http://www.cs.ucsb.edu/~ravenben/classes/276/papers/vivaldi-sigcomm04.pdf),
with some enhancements based on other research. There are more details about
[Serf's network coordinates here](https://www.serf.io/docs/internals/coordinates.html).
## Network Coordinates in Consul
Network coordinates manifest in several ways inside Consul:
- The [`consul rtt`](/commands/rtt) command can be used to query for the
network round trip time between any two nodes.
- The [Catalog endpoints](/api/catalog) and
[Health endpoints](/api/health) can sort the results of queries based
on the network round trip time from a given node using a "?near=" parameter.
- [Prepared queries](/api/query) can automatically fail over services
to other Consul datacenters based on network round trip times. See the
[Geo Failover](https://learn.hashicorp.com/tutorials/consul/automate-geo-failover) for some examples.
- The [Coordinate endpoint](/api/coordinate) exposes raw network
coordinates for use in other applications.
Consul uses Serf to manage two different gossip pools, one for the LAN with members
of a given datacenter, and one for the WAN which is made up of just the Consul servers
in all datacenters. It's important to note that **network coordinates are not compatible
between these two pools**. LAN coordinates only make sense in calculations with other
LAN coordinates, and WAN coordinates only make sense with other WAN coordinates.
## Working with Coordinates
Computing the estimated network round trip time between any two nodes is simple
once you have their coordinates. Here's a sample coordinate, as returned from the
[Coordinate endpoint](/api/coordinate).
<CodeBlockConfig heading="Sample coordinate from Coordinate endpoint" hideClipboard>
```json
...
"Coord": {
"Adjustment": 0.1,
"Error": 1.5,
"Height": 0.02,
"Vec": [0.34,0.68,0.003,0.01,0.05,0.1,0.34,0.06]
}
...
```
</CodeBlockConfig>
All values are floating point numbers in units of seconds, except for the error
term which isn't used for distance calculations.
Here's a complete example in Go showing how to compute the distance between two
coordinates:
<CodeBlockConfig heading="Computing distance between two coordinates with Go">
```go
import (
"math"
"time"
"github.com/hashicorp/serf/coordinate"
)
func dist(a *coordinate.Coordinate, b *coordinate.Coordinate) time.Duration {
// Coordinates will always have the same dimensionality, so this is
// just a sanity check.
if len(a.Vec) != len(b.Vec) {
panic("dimensions aren't compatible")
}
// Calculate the Euclidean distance plus the heights.
sumsq := 0.0
for i := 0; i < len(a.Vec); i++ {
diff := a.Vec[i] - b.Vec[i]
sumsq += diff * diff
}
rtt := math.Sqrt(sumsq) + a.Height + b.Height
// Apply the adjustment components, guarding against negatives.
adjusted := rtt + a.Adjustment + b.Adjustment
if adjusted > 0.0 {
rtt = adjusted
}
// Go's times are natively nanoseconds, so we convert from seconds.
const secondsToNanoseconds = 1.0e9
return time.Duration(rtt * secondsToNanoseconds)
}
```
</CodeBlockConfig>