[[ch-benchmark]] == Measuring block device performance [[s-measure-throughput]] === Measuring throughput When measuring the impact of using DRBD on a system's I/O throughput, the _absolute_ throughput the system is capable of is of little relevance. What is much more interesting is the _relative_ impact DRBD has on I/O performance. Thus it is always necessary to measure I/O throughput both with and without DRBD. CAUTION: The tests described in this section are intrusive; they overwrite data and bring DRBD devices out of sync. It is thus vital that you perform them only on scratch volumes which can be discarded after testing has completed. I/O throughput estimation works by writing significantly large chunks of data to a block device, and measuring the amount of time the system took to complete the write operation. This can be easily done using a fairly ubiquitous utility, `dd`, whose reasonably recent versions include a built-in throughput estimation. A simple ``dd``-based throughput benchmark, assuming you have a scratch resource named `test`, which is currently connected and in the secondary role on both nodes, is one like the following: [source,drbd] ---------------------------- # TEST_RESOURCE=test # TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE | head -1) # TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE | head -1) # drbdadm primary $TEST_RESOURCE # for i in $(seq 5); do dd if=/dev/zero of=$TEST_DEVICE bs=1M count=512 oflag=direct done # drbdadm down $TEST_RESOURCE # for i in $(seq 5); do dd if=/dev/zero of=$TEST_LL_DEVICE bs=1M count=512 oflag=direct done ---------------------------- This test simply writes 512MiB of data to your DRBD device, and then to its backing device for comparison. Both tests are repeated 5 times each to allow for some statistical averaging. The relevant result is the throughput measurements generated by `dd`. NOTE: For freshly enabled DRBD devices, it is normal to see slightly reduced performance on the first `dd` run. This is due to the Activity Log being "cold", and is no cause for concern. See our <> chapter for some performance numbers. [[s-measure-latency]] === Measuring latency Latency measurements have objectives completely different from throughput benchmarks: in I/O latency tests, one writes a very small chunk of data (ideally the smallest chunk of data that the system can deal with), and observes the time it takes to complete that write. The process is usually repeated several times to account for normal statistical fluctuations. Just as throughput measurements, I/O latency measurements may be performed using the ubiquitous `dd` utility, albeit with different settings and an entirely different focus of observation. Provided below is a simple ``dd``-based latency micro-benchmark, assuming you have a scratch resource named `test` which is currently connected and in the secondary role on both nodes: [source,drbd] ---------------------------- # TEST_RESOURCE=test # TEST_DEVICE=$(drbdadm sh-dev $TEST_RESOURCE | head -1) # TEST_LL_DEVICE=$(drbdadm sh-ll-dev $TEST_RESOURCE | head -1) # drbdadm primary $TEST_RESOURCE # dd if=/dev/zero of=$TEST_DEVICE bs=4k count=1000 oflag=direct # drbdadm down $TEST_RESOURCE # dd if=/dev/zero of=$TEST_LL_DEVICE bs=4k count=1000 oflag=direct ---------------------------- This test writes 1,000 chunks with 4kiB each to your DRBD device, and then to its backing device for comparison. 4096 bytes is the smallest block size that a Linux system (on all architectures except s390), modern hard disks, and SSDs, are expected to handle. It is important to understand that throughput measurements generated by `dd` are completely irrelevant for this test; what is important is the _time_ elapsed during the completion of said 1,000 writes. Dividing this time by 1,000 gives the average latency of a single block write. NOTE: This is the _worst-case_, in that it is single-threaded and does one write strictly after the one before, i.e. runs with an I/O-depth of 1. Please take a look at <>. Furthermore, see our <> chapter for some typical performance values.