mirror of https://github.com/hashicorp/consul
119 lines
4.8 KiB
Markdown
119 lines
4.8 KiB
Markdown
---
|
|
layout: docs
|
|
page_title: Consistency Verification | Jepsen Testing Results
|
|
description: >-
|
|
Jepsen is a tool to measure the reliability and consistency of distributed systems across network partitions. Learn about the Jepsen testing performed on Consul to ensure it gracefully recovers from partitions and maintains consistent state.
|
|
---
|
|
|
|
# Jepsen Testing Results
|
|
|
|
[Jepsen](http://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions)
|
|
is a tool, written by Kyle Kingsbury, designed to test the partition
|
|
tolerance of distributed systems. It creates network partitions while fuzzing
|
|
the system with random operations. The results are analyzed to see if the system
|
|
violates any of the consistency properties it claims to have.
|
|
|
|
As part of our Consul testing, we ran a Jepsen test to determine if
|
|
any consistency issues could be uncovered. In our testing, Consul
|
|
gracefully recovered from partitions without introducing any consistency
|
|
issues.
|
|
|
|
## Running the tests
|
|
|
|
At the moment, testing with Jepsen is rather complex as it requires
|
|
setting up multiple virtual machines, SSH keys, DNS configuration,
|
|
and a working Clojure environment. We hope to contribute our Consul
|
|
testing code upstream and to provide a Vagrant environment for Jepsen
|
|
testing soon.
|
|
|
|
## Output
|
|
|
|
Below is the output captured from Jepsen. We ran Jepsen multiple times,
|
|
and it passed each time. This output is only representative of a single
|
|
run and has been edited for length. Please reach out on [Consul's Discuss](https://discuss.hashicorp.com/c/consul)
|
|
if you would like to reproduce the Jepsen results.
|
|
|
|
<!--googleoff: all-->
|
|
|
|
```shell-session
|
|
$ lein test :only jepsen.system.consul-test
|
|
|
|
lein test jepsen.system.consul-test
|
|
INFO jepsen.os.debian - :n5 setting up debian
|
|
INFO jepsen.os.debian - :n3 setting up debian
|
|
INFO jepsen.os.debian - :n4 setting up debian
|
|
INFO jepsen.os.debian - :n1 setting up debian
|
|
INFO jepsen.os.debian - :n2 setting up debian
|
|
INFO jepsen.os.debian - :n4 debian set up
|
|
INFO jepsen.os.debian - :n5 debian set up
|
|
INFO jepsen.os.debian - :n3 debian set up
|
|
INFO jepsen.os.debian - :n1 debian set up
|
|
INFO jepsen.os.debian - :n2 debian set up
|
|
INFO jepsen.system.consul - :n1 consul nuked
|
|
INFO jepsen.system.consul - :n4 consul nuked
|
|
INFO jepsen.system.consul - :n5 consul nuked
|
|
INFO jepsen.system.consul - :n3 consul nuked
|
|
INFO jepsen.system.consul - :n2 consul nuked
|
|
INFO jepsen.system.consul - Running nodes: {:n1 false, :n2 false, :n3 false, :n4 false, :n5 false}
|
|
INFO jepsen.system.consul - :n2 consul nuked
|
|
INFO jepsen.system.consul - :n3 consul nuked
|
|
INFO jepsen.system.consul - :n4 consul nuked
|
|
INFO jepsen.system.consul - :n5 consul nuked
|
|
INFO jepsen.system.consul - :n1 consul nuked
|
|
INFO jepsen.system.consul - :n1 starting consul
|
|
INFO jepsen.system.consul - :n2 starting consul
|
|
INFO jepsen.system.consul - :n4 starting consul
|
|
INFO jepsen.system.consul - :n5 starting consul
|
|
INFO jepsen.system.consul - :n3 starting consul
|
|
INFO jepsen.system.consul - :n3 consul ready
|
|
INFO jepsen.system.consul - :n2 consul ready
|
|
INFO jepsen.system.consul - Running nodes: {:n1 true, :n2 true, :n3 true, :n4 true, :n5 true}
|
|
INFO jepsen.system.consul - :n5 consul ready
|
|
INFO jepsen.system.consul - :n1 consul ready
|
|
INFO jepsen.system.consul - :n4 consul ready
|
|
INFO jepsen.core - Worker 0 starting
|
|
INFO jepsen.core - Worker 2 starting
|
|
INFO jepsen.core - Worker 1 starting
|
|
INFO jepsen.core - Worker 3 starting
|
|
INFO jepsen.core - Worker 4 starting
|
|
INFO jepsen.util - 2 :invoke :read nil
|
|
INFO jepsen.util - 3 :invoke :cas [4 4]
|
|
INFO jepsen.util - 0 :invoke :write 4
|
|
INFO jepsen.util - 1 :invoke :write 1
|
|
INFO jepsen.util - 4 :invoke :cas [4 0]
|
|
INFO jepsen.util - 2 :ok :read nil
|
|
INFO jepsen.util - 4 :fail :cas [4 0]
|
|
(Log Truncated...)
|
|
INFO jepsen.util - 4 :invoke :cas [3 3]
|
|
INFO jepsen.util - 4 :fail :cas [3 3]
|
|
INFO jepsen.util - :nemesis :info :stop nil
|
|
INFO jepsen.util - :nemesis :info :stop "fully connected"
|
|
INFO jepsen.util - 0 :fail :read nil
|
|
INFO jepsen.util - 1 :fail :write 0
|
|
INFO jepsen.util - :nemesis :info :stop nil
|
|
INFO jepsen.util - :nemesis :info :stop "fully connected"
|
|
INFO jepsen.core - nemesis done
|
|
INFO jepsen.core - Worker 3 done
|
|
INFO jepsen.util - 1 :invoke :read nil
|
|
INFO jepsen.core - Worker 2 done
|
|
INFO jepsen.core - Worker 4 done
|
|
INFO jepsen.core - Worker 0 done
|
|
INFO jepsen.util - 1 :ok :read 3
|
|
INFO jepsen.core - Worker 1 done
|
|
INFO jepsen.core - Run complete, writing
|
|
INFO jepsen.core - Analyzing
|
|
(Log Truncated...)
|
|
INFO jepsen.core - Analysis complete
|
|
INFO jepsen.system.consul - :n3 consul nuked
|
|
INFO jepsen.system.consul - :n2 consul nuked
|
|
INFO jepsen.system.consul - :n4 consul nuked
|
|
INFO jepsen.system.consul - :n1 consul nuked
|
|
INFO jepsen.system.consul - :n5 consul nuked
|
|
1964 element history linearizable. :D
|
|
|
|
Ran 1 tests containing 1 assertions.
|
|
0 failures, 0 errors.
|
|
```
|
|
|
|
<!--googleon: all-->
|