Debian Codegraph @

UG9/en/proxy.adoc

raw
1 [[s-using-drbd-proxy]]
2 == Using DRBD Proxy
3
4 [[s-drbd-proxy-deployment-considerations]]
5 === DRBD Proxy deployment considerations
6
7 The <<s-drbd-proxy,DRBD Proxy>> processes can either be located
8 directly on the machines where DRBD is set up, or they can be placed
9 on distinct dedicated servers. A DRBD Proxy instance can serve as a
10 proxy for multiple DRBD devices distributed across multiple nodes.
11
12 DRBD Proxy is completely transparent to DRBD. Typically you will
13 expect a high number of data packets in flight, therefore the activity
14 log should be reasonably large. Since this may cause longer re-sync
15 runs after the crash of a primary node, it is recommended to enable
16 DRBD's `csums-alg` setting.
17
18 For more information about the rationale for the DRBD Proxy, please
19 see the feature explanation <<s-drbd-proxy>>.
20
21 The DRBD Proxy 3 uses several kernel features that are only available since
22 2.6.26, so running it on older systems (e.g. RHEL 5) is not possible; here
23 we can still provide DRBD Proxy 1 packages, thoughfootnote:[The v1 uses
24 a different scheduling model and will therefore not reach the same performance
25 as v3; so even if your production setup is still RHEL 5, perhaps you can run
26 one RHEL 6/7 VM in each data center?].
27
28
29 [[s-drbd-proxy-installation]]
30 === Installation
31
32 To obtain DRBD Proxy, please contact your Linbit sales
33 representative. Unless instructed otherwise, please always use the
34 most recent DRBD Proxy release.
35
36 To install DRBD Proxy on Debian and Debian-based systems, use the dpkg
37 tool as follows (replace version with your DRBD Proxy version, and
38 architecture with your target architecture):
39
40 ----------------------------
41 # dpkg -i drbd-proxy_3.2.2_amd64.deb
42 ----------------------------
43
44 To install DRBD Proxy on RPM based systems (like SLES or RHEL) use
45 the rpm tool as follows (replace version with your DRBD Proxy version,
46 and architecture with your target architecture):
47
48 ----------------------------
49 # rpm -i drbd-proxy-3.2.2-1.x86_64.rpm
50 ----------------------------
51
52 Also install the DRBD administration program drbdadm since it is
53 required to configure DRBD Proxy.
54
55 This will install the DRBD proxy binaries as well as an init script
56 which usually goes into `/etc/init.d`. Please always use the init
57 script to start/stop DRBD proxy since it also configures DRBD Proxy
58 using the `drbdadm` tool.
59
60 [[s-drbd-proxy-license]]
61 === License file
62
63 When obtaining a license from Linbit, you will be sent a DRBD Proxy
64 license file which is required to run DRBD Proxy. The file is called
65 `drbd-proxy.license`, it must be copied into the `/etc` directory of the
66 target machines, and be owned by the user/group `drbdpxy`.
67
68 ----------------------------
69 # cp drbd-proxy.license /etc/
70 ----------------------------
71
72 [[s-drbd-proxy-configuration-linstor]]
73 === Configuration using LINSTOR
74
75 DRBD Proxy can be configured using LINSTOR as described in the LINSTOR
76 USER'S GUIDE.
77
78 [[s-drbd-proxy-configuration]]
79 === Configuration using resource files
80
81 DRBD Proxy can also be configured by editing resource files.
82 It is configured by an additional section called `proxy` and
83 additional `proxy on` sections within the host sections.
84
85 Below is a DRBD configuration example for proxies running directly on
86 the DRBD nodes:
87
88 [source,drbd]
89 ----------------------------
90 resource r0 {
91 protocol A;
92 device /dev/drbd15;
93 disk /dev/VG/r0;
94 meta-disk internal;
95
96 proxy {
97 memlimit 512M;
98 plugin {
99 zlib level 9;
100 }
101 }
102
103 on alice {
104 address 127.0.0.1:7915;
105 proxy on alice {
106 inside 127.0.0.1:7815;
107 outside 192.168.23.1:7715;
108 }
109 }
110
111 on bob {
112 address 127.0.0.1:7915;
113 proxy on bob {
114 inside 127.0.0.1:7815;
115 outside 192.168.23.2:7715;
116 }
117 }
118 }
119 ----------------------------
120
121 The `inside` IP address is used for communication between DRBD and the
122 DRBD Proxy, whereas the `outside` IP address is used for communication
123 between the proxies. The latter channel might have to be allowed in your
124 firewall setup.
125
126 [[s-drbd-proxy-controlling]]
127 === Controlling DRBD Proxy
128
129 `drbdadm` offers the `proxy-up` and `proxy-down` subcommands to
130 configure or delete the connection to the local DRBD Proxy process of
131 the named DRBD resource(s). These commands are used by the `start` and
132 `stop` actions which `/etc/init.d/drbdproxy` implements.
133
134 The DRBD Proxy has a low level configuration tool, called
135 `drbd-proxy-ctl`. When called without any option it operates in
136 interactive mode.
137
138 To pass a command directly, avoiding interactive mode, use
139 the `-c` parameter followed by the command.
140
141 To display the available commands use:
142 ----------------------------
143 # drbd-proxy-ctl -c "help"
144 ----------------------------
145
146 Note the double quotes around the command being passed.
147
148
149 Here is a list of commands; while the first few ones are typically only used
150 indirectly (via `drbdadm proxy-up` resp. `drbdadm proxy-down`), the latter ones
151 give various status information.
152
153 .`add connection _<name>_ _lots of arguments_`
154 Creates a communication path. As this is run via `drbdadm proxy-up` the long list of arguments is omitted here.
155
156 .`del connection _<name>_`
157 Removes a communication path.
158
159 .`set memlimit _<name>_ _<memlimit-in-bytes>_`
160 Sets the memory limit for a connection; this can only be done when setting it
161 up afresh, changing it during runtime is not possible. +
162 This command understands the usual units `k`, `M`, and `G`.
163
164 .`show`
165 Shows currently configured communication paths.
166
167 .`show memusage`
168 Shows memory usage of each connection. +
169 --
170 As an example,
171
172 --------
173 # watch -n 1 'drbd-proxy-ctl -c "show memusage"'
174 --------
175
176 monitors memory usage. Please note that the quotes are required as listed above.
177 --
178
179 .`show [h]subconnections`
180 Shows currently established individual connections
181 together with some stats. With `h` outputs bytes in human
182 readable format.
183
184 .`show [h]connections`
185 Shows currently configured connections and their states
186 With `h` outputs bytes in human readable format. +
187 --
188 The column `Status` will show one of these states:
189
190 * _Off_: No communication to the remote DRBD Proxy process.
191 * _Half-up_: The connection to the remote DRBD Proxy could be established; the Proxy => DRBD paths are not up yet.
192 * _DRBD-conn_: The first few packets are being pushed across the connection; but still e.g. a Split-Brain situation might serve it again.
193 * _Up_: The DRBD connection is fully established.
194 --
195
196 .`shutdown`
197 Shuts down the `drbd-proxy` program. Attention: this
198 unconditionally terminates any DRBD connections using the DRBD proxy.
199
200 .`quit`
201 Exits the client program (closes the control connection), but
202 leaves the DRBD proxy running.
203
204
205 .`print statistics`
206 This prints detailed statistics for the currently active connections, in an
207 easily parseable format. Use this for integration to your monitoring solution! +
208
209 NOTE: While the commands above are only accepted from UID 0 (i.e., the `root` user),
210 this one can be used by any user
211 (provided that unix permissions allow access on the proxy socket at
212 `/var/run/drbd-proxy/drbd-proxy-ctl.socket`); see the init script at
213 `/etc/init.d/drbdproxy` about setting the rights.
214
215
216
217 [[s-drbd-proxy-plugins]]
218 === About DRBD Proxy plugins
219
220 Since DRBD Proxy version 3 the proxy allows to enable a few specific
221 plugins for the WAN connection. +
222 The currently available plugins are `zstd`, `lz4`, `zlib` and
223 `lzma` (all software compression), and `aha` (hardware compression support, see http://www.aha.com/data-compression/).
224
225 `zstd` (Zstandard) is a real-time compression algorithm, providing high compression ratios. It offers a very
226 wide range of compression / speed trade-off, while being backed by a very fast decoder. Compression rates are
227 dependent on "level" parameter which can be arranged between 1 to 22. Over level 20, Drbd Proxy will require
228 more memory.
229
230 `lz4` is a very fast compression algorithm; the data typically gets compressed
231 down by 1:2 to 1:4, half- to two-thirds of the bandwidth can be saved.
232
233 The `zlib` plugin uses the GZIP algorithm for compression; it uses a bit more CPU than `lz4`, but gives a ratio of 1:3 to 1:5.
234
235 The `lzma` plugin uses the `liblzma2` library. It can
236 use dictionaries of several hundred MiB; these allow for very
237 efficient delta-compression of repeated data, even for small changes.
238 `lzma` needs much more CPU and memory, but results in much better
239 compression than `zlib` -- real-world tests with a VM sitting on top of DRBD
240 gave ratios of 1:10 to 1:40. The `lzma` plugin has to be enabled in your
241 license.
242
243 `aha` uses hardware compression cards, like the AHA367PCIe (10Gbit/sec) or
244 AHA372 (20GBit/sec); this is the fastest compression for contemporary hardware. +
245 You will need a special flag in your license file to enable this plugin.
246
247
248 Please contact LINBIT to find the best settings for your environment - it
249 depends on the CPU (speed, number of threads), available memory, input and
250 available output bandwidth, and expected IO spikes. Having a week of `sysstat`
251 data already available helps in determining the configuration, too.
252
253
254 Please note that the older `compression on` in the
255 `proxy` section is deprecated, and will be removed in
256 a future release. +
257 Currently it is treated as `zlib level 9`.
258
259
260 [[s-drbd-proxy-bwlimit]]
261 ==== Using a WAN Side Bandwidth Limit
262
263 The experimental `bwlimit` option of DRBD Proxy is broken. Do not use
264 it, as it may cause applications on DRBD to block on IO. It will
265 be removed.
266
267 Instead use the Linux kernel's traffic control framework to
268 limit bandwidth consumed by proxy on the WAN side.
269
270 In the following example you would need to replace the interface
271 name, the source port and the ip address of the peer.
272
273 ----------------------------
274 # tc qdisc add dev eth0 root handle 1: htb default 1
275 # tc class add dev eth0 parent 1: classid 1:1 htb rate 1gbit
276 # tc class add dev eth0 parent 1:1 classid 1:10 htb rate 500kbit
277 # tc filter add dev eth0 parent 1: protocol ip prio 16 u32 \
278 match ip sport 7000 0xffff \
279 match ip dst 192.168.47.11 flowid 1:10
280 # tc filter add dev eth0 parent 1: protocol ip prio 16 u32 \
281 match ip dport 7000 0xffff \
282 match ip dst 192.168.47.11 flowid 1:10
283 ----------------------------
284
285 You can remove this bandwidth limitation with
286
287 ----------------------------
288 # tc qdisc del dev eth0 root handle 1
289 ----------------------------
290
291 [[s-drbd-proxy-troubleshoot]]
292 === Troubleshooting
293
294 DRBD proxy logs via syslog using the `LOG_DAEMON` facility. Usually
295 you will find DRBD Proxy messages in `/var/log/daemon.log`.
296
297 Enabling debug mode in DRBD Proxy can be done with the following command.
298
299 --------------------------
300 # drbd-proxy-ctl -c 'set loglevel debug'
301 --------------------------
302
303 For example, if proxy fails to connect it will log something like
304 `Rejecting connection because I can't connect on the other side`. In
305 that case, please check if DRBD is running (not in StandAlone mode) on
306 both nodes and if both proxies are running. Also double-check your
307 configuration.