| 1 |
[[ch-benchmark]] |
| 2 |
== Measuring block device performance |
| 3 |
|
| 4 |
|
| 5 |
[[s-measure-throughput]] |
| 6 |
=== Measuring throughput |
| 7 |
|
| 8 |
When measuring the impact of using DRBD on a system's I/O throughput, |
| 9 |
the _absolute_ throughput the system is capable of is of little |
| 10 |
relevance. What is much more interesting is the _relative_ impact DRBD |
| 11 |
has on I/O performance. Thus it is always necessary to measure I/O |
| 12 |
throughput both with and without DRBD. |
| 13 |
|
| 14 |
CAUTION: The tests described in this section are intrusive; they |
| 15 |
overwrite data and bring DRBD devices out of sync. It is thus vital |
| 16 |
that you perform them only on scratch volumes which can be discarded |
| 17 |
after testing has completed. |
| 18 |
|
| 19 |
I/O throughput estimation works by writing significantly large chunks |
| 20 |
of data to a block device, and measuring the amount of time the system |
| 21 |
took to complete the write operation. This can be easily done using a |
| 22 |
fairly ubiquitous utility, `dd`, whose reasonably recent versions |
| 23 |
include a built-in throughput estimation. |
| 24 |
|
| 25 |
A simple ``dd``-based throughput benchmark, assuming you have a scratch |
| 26 |
resource named `test`, which is currently connected and in the |
| 27 |
secondary role on both nodes, is one like the following: |
| 28 |
|
| 29 |
[source,drbd] |
| 30 |
---------------------------- |
| 31 |
# TEST_RESOURCE=test |
| 32 |
# TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE | head -1) |
| 33 |
# TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE | head -1) |
| 34 |
# drbdadm primary $TEST_RESOURCE |
| 35 |
# for i in $(seq 5); do |
| 36 |
dd if=/dev/zero of=$TEST_DEVICE bs=1M count=512 oflag=direct |
| 37 |
done |
| 38 |
# drbdadm down $TEST_RESOURCE |
| 39 |
# for i in $(seq 5); do |
| 40 |
dd if=/dev/zero of=$TEST_LL_DEVICE bs=1M count=512 oflag=direct |
| 41 |
done |
| 42 |
---------------------------- |
| 43 |
|
| 44 |
This test simply writes 512MiB of data to your DRBD device, and |
| 45 |
then to its backing device for comparison. Both tests are repeated 5 |
| 46 |
times each to allow for some statistical averaging. The relevant |
| 47 |
result is the throughput measurements generated by `dd`. |
| 48 |
|
| 49 |
NOTE: For freshly enabled DRBD devices, it is normal to see |
| 50 |
slightly reduced performance on the first `dd` run. This is due |
| 51 |
to the Activity Log being "cold", and is no cause for concern. |
| 52 |
|
| 53 |
See our <<ch-throughput>> chapter for some performance numbers. |
| 54 |
|
| 55 |
|
| 56 |
[[s-measure-latency]] |
| 57 |
=== Measuring latency |
| 58 |
|
| 59 |
Latency measurements have objectives completely different from |
| 60 |
throughput benchmarks: in I/O latency tests, one writes a very small |
| 61 |
chunk of data (ideally the smallest chunk of data that the system can |
| 62 |
deal with), and observes the time it takes to complete that write. The |
| 63 |
process is usually repeated several times to account for normal |
| 64 |
statistical fluctuations. |
| 65 |
|
| 66 |
Just as throughput measurements, I/O latency measurements may be |
| 67 |
performed using the ubiquitous `dd` utility, albeit with different |
| 68 |
settings and an entirely different focus of observation. |
| 69 |
|
| 70 |
Provided below is a simple ``dd``-based latency micro-benchmark, |
| 71 |
assuming you have a scratch resource named `test` which is currently |
| 72 |
connected and in the secondary role on both nodes: |
| 73 |
|
| 74 |
[source,drbd] |
| 75 |
---------------------------- |
| 76 |
# TEST_RESOURCE=test |
| 77 |
# TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE | head -1) |
| 78 |
# TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE | head -1) |
| 79 |
# drbdadm primary $TEST_RESOURCE |
| 80 |
# dd if=/dev/zero of=$TEST_DEVICE bs=4k count=1000 oflag=direct |
| 81 |
# drbdadm down $TEST_RESOURCE |
| 82 |
# dd if=/dev/zero of=$TEST_LL_DEVICE bs=4k count=1000 oflag=direct |
| 83 |
---------------------------- |
| 84 |
|
| 85 |
This test writes 1,000 chunks with 4kiB each to your DRBD device, |
| 86 |
and then to its backing device for comparison. 4096 bytes is the |
| 87 |
smallest block size that a Linux system (on all architectures except s390), |
| 88 |
modern hard disks, and SSDs, are expected to handle. |
| 89 |
|
| 90 |
It is important to understand that throughput measurements generated |
| 91 |
by `dd` are completely irrelevant for this test; what is important is |
| 92 |
the _time_ elapsed during the completion of said 1,000 writes. Dividing |
| 93 |
this time by 1,000 gives the average latency of a single block write. |
| 94 |
|
| 95 |
NOTE: This is the _worst-case_, in that it is single-threaded and does one |
| 96 |
write strictly after the one before, i.e. runs with an I/O-depth of 1. Please |
| 97 |
take a look at <<s-latency-iops>>. |
| 98 |
|
| 99 |
Furthermore, see our <<ch-latency>> chapter for some typical performance |
| 100 |
values. |