Debian Codegraph @

doc/html/_sources/imap/reference/admin/murder/murder-installation.rst.txt

raw
1 .. _murder-installation:
2
3 =============================================
4 Cyrus Murder: Installation and Administration
5 =============================================
6
7 Architecture
8 ============
9
10 :ref:`Overall structure of a Cyrus Murder <architecture_murder>`.
11
12 The Cyrus Murder provides the ability to horizontally scale your Cyrus
13 IMAP environment across multiple servers. It is similar in nature to
14 various proxy solutions such as nginx or perdition with the difference
15 being that Murder offers a uniform namespace. Those not currently using
16 shared mailboxes and who don't intend to use shared mailboxes in the
17 future, should probably just consider using a simple proxy solution.
18
19 Before beginning Cyrus Murder configuration, we strongly recommended a
20 thorough review of the :ref:`Cyrus Murder Concepts <murder_concepts>`
21 guide.
22
23 Terminology
24 ===========
25
26 Master
27 In this document, Master always means the MUPDATE master server.
28
29 Installation
30 ============
31
32 Cyrus IMAPd must be built with the ``--enable-murder`` configure
33 option. This builds the proxyds and the associated utilities.
34
35 .. Note::
36 Those using their distribution's packages may need to install a
37 separate package for aggregation support. For example, on Debian
38 and derived distros, install the ``cyrus-murder`` package.
39
40 Requirements
41 ------------
42
43 * At least one Cyrus IMAP server instance. If there are more than
44 one, their namespaces **must not** conflict. That is, all the
45 mailbox names must be unique (or in different namespaces)
46
47 * At least one machine that will become the first Frontend Server.
48
49 * One machine to become the MUPDATE Master server. This can be the
50 same as one of the frontend servers.
51
52 Configuring the MUPDATE Master
53 ------------------------------
54
55 The MUPDATE Master server needs to be running the mupdate service in
56 master mode. The MUPDATE master may be one of the cluster's frontend
57 instances, in which case no slave mupdate process should be run on this
58 instance.
59
60 On the mupdate master :cyrusman:`cyrus.conf(5)` must include a line
61 similar to the following in the SERVICES section::
62
63 mupdate cmd="/usr/cyrus/bin/mupdate -m" listen=3905 prefork=1
64
65 Note the ``-m`` option to tell mupdate that it should start in master
66 mode.
67
68 The MUPDATE Master will also need at least a skeleton
69 :cyrusman:`imapd.conf(5)` that defines the config directory, a bogus
70 partition-default and the ``admin``\ s who can authenticate to the server.
71 Slave mupdate servers as well as the back end servers will need to be
72 able to authenticate as admins on the master.
73
74 Here is a very simple :cyrusman:`imapd.conf(5)` for a master server::
75
76 configdirectory: /imap/conf
77 partition-default: /tmp
78
79 admins: mupdateslave1 backend1
80
81 SASL must also be configured as needed to properly allow authentication.
82
83 Setting up the backends to push changes to the MUPDATE Master
84 -------------------------------------------------------------
85
86 On the backends, configuration to be a part of a murder is easy. Simply
87 set the ``mupdate_server`` option in :cyrusman:`imapd.conf(5)` and add
88 an entry to :cyrusman:`cyrus.conf(5)` to push the mailboxes list to
89 the MUPDATE Master.
90
91 Depending on the authentication mechanisms used, some or all of the
92 following settings in :cyrusman:`imapd.conf(5)` may be required:
93
94 * mupdate_username
95 * mupdate_authname
96 * mupdate_realm
97 * mupdate_password
98 * servername
99
100 Once these settings are made, any mailbox operation on the backend will
101 be sent to the mupdate master for confirmation and entry into the
102 mupdate database.
103
104 At least one user/group must be configured using the ``proxyservers``
105 :cyrusman:`imapd.conf(5)` option. This user should not be an
106 administrator, as that would give anyone compromising this credential
107 full administrative control on all back ends.
108
109 .. Note::
110 For lmtp to work in a murder, the proxyservers entries must also
111 appear in the lmtp_admins entry.
112
113 Example of the :cyrusman:`imapd.conf(5)` settings discussed thus far::
114
115 # How this server identifies itself within the murder
116 servername: mailbox.example.com
117 # Who's permitted to authenticate for which purposes
118 admins: cyrus
119 proxyservers: mailproxy
120 lmtp_admins: mailproxy
121 # Auth credentials for MUPDATE Master
122 mupdate_server: postman.example.com
123 mupdate_username: postman
124 mupdate_authname: postman
125 mupdate_password: <secret>
126
127 All proxy user(s) must exist within the authentication domain of both
128 the MUPDATE Master and the back end, as well.
129
130 .. Note::
131
132 Do not set ``proxyservers`` on frontends.
133
134 Exporting the database from the backend
135 ---------------------------------------
136
137 The existing mailboxes database must be exported to the MUPDATE Master.
138 Use the :cyrusman:`ctl_mboxlist(8)` command to do so. For the first
139 synchronization, change to the cyrus user, and run ``ctl_mboxlist -m``.
140
141 .. Important::
142 One should use ``ctl_mboxlist -mw`` (dry run) first to be sure of
143 understanding all the operations that this command will perform,
144 since it does require that all mailboxes are unique in the murder
145 namespace and could lead to deletions of conflicting mailboxes on
146 other back ends already in the murder.
147
148 If everything is configured properly, the mailbox database of the
149 current host will upload to the mupdate master. If there are problems,
150 the most likely cause is a misconfiguration of the authentication
151 settings, or :cyrusman:`mupdate(8)` might not be running on the master.
152 Using :cyrusman:`mupdatetest(1)` may be helpful in this case (it
153 establishes an authenticated connection to the mupdate server, if it
154 can).
155
156 It is also useful to have the backends automatically resync the state
157 of their local mailboxes database with the master on start up. This is
158 configured by adding the following to the START section of
159 :cyrusman:`cyrus.conf(5)` on the backends::
160
161 mupdatepush cmd="ctl_mboxlist -m"
162
163 This will perform synchronization with the mupdate master each time the
164 backend restarts, bringing the mupdate database up to date with the
165 contents of the backend (and performing ACTIVATE and DELETES as needed
166 to do so).
167
168 .. Warning::
169
170 If somehow a mailbox exists on two (or more) backend servers, each
171 time one of them synchronizes its database that backend server will
172 become authoritative. Though this should not happen during normal
173 operation of the murder (because of the consistency guarantees of
174 the MUPDATE protocol, and the fact that mailbox operations are
175 denied if the mupdate master is down), it is possible when first
176 creating the mupdate database or when bringing a new backend server
177 into the murder.
178
179 Configuring the front ends
180 --------------------------
181
182 Configuring the front ends is a two step process. First, define
183 mupdate_server (and friends) as done for the backends above. However,
184 as the frontends only talk to the mupdate master via a slave running on
185 the local machine, also set up a slave on the same machine, in the
186 SERVICES section of :cyrusman:`cyrus.conf(5)`, like so::
187
188 # mupdate database service - must prefork at least 1
189 mupdate cmd="mupdate" listen=3905 prefork=1
190
191 As this is a threaded service, prefork at least 1 so that the database
192 synchronizes at startup. Otherwise, the service will not start running
193 until after receiving a mupdate client connection to the slave (which
194 is not a recommended configuration at this point).
195
196 The front end SERVICES section should now look like this::
197
198 mupdate cmd="mupdate" listen=3905 prefork=1
199
200 imap cmd="imap" listen="imap" prefork=5
201 imaps cmd="imap -s" listen="imaps" prefork=1
202 pop3 cmd="pop3d" listen="pop3" prefork=0
203 pop3s cmd="pop3d -s" listen="pop3s" prefork=0
204 nntp cmd="nntpd" listen="nntp" prefork=0
205 nntps cmd="nntpd -s" listen="nntps" prefork=0
206 http cmd="httpd" listen="http" prefork=0
207 https cmd="httpd -s" listen="https" prefork=0
208 sieve cmd="timsieved" listen="sieve" prefork=0
209 lmtp cmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0
210
211 Note that timsieved does not need a proxy daemon, the managesieve
212 protocol deals with the murder with referrals to the backends
213 internally.
214
215 Additionally, entries in :cyrusman:`imapd.conf(5)` are required to
216 indicate the proxy auth name and passwords (if using a SASL mechanism
217 that requires them) to the backends.
218
219 For example, if the backends are ``mail1.andrew.cmu.edu`` and
220 ``mail2.andrew.cmu.edu`` with passwords of ``foo`` and ``bar``, and an
221 auth name of ``mailproxy``::
222
223 mail1_password: foo
224 mail2_password: bar
225 proxy_authname: mailproxy
226
227 For SASL mechanisms not using authnames or passwords (e.g.
228 GSSAPI), the password options are not required. Note the use of
229 the same authname as configured in the proxyservers line of the
230 backend's :cyrusman:`imapd.conf(5)` above.
231
232 Upon starting :cyrusman:`master(8)` on the frontend, the local
233 mailboxes database should automatically synchronize with the contents
234 of the MUPDATE master, and it's ready to go. Clients should connect to
235 the frontends, and the frontends will proxy or refer as applicable to
236 the backend servers.
237
238 Additional backend configuration
239 --------------------------------
240
241 Authentication system expecting usernames, passwords, etc, to
242 authenticate, will also need to specify proxy_authname (and friends) in
243 the backend imapd.confs. This is so the backends can authenticate to
244 each other to facilitate mailbox moves. (Backend machines will need to
245 be full admins).
246
247 Delivering mail
248 ---------------
249
250 To deliver mail to a Murder, configure MTAs just as before, but instead
251 of connecting directly to lmtpd on a back end, they should connect to
252 lmtpproxyd on any front end. Remote MTAs may connect to the lmtpproxyd
253 running on any front end machine (listening on a TCP socket), or
254 install master and lmtpproxyd on your SMTP servers to connect via Unix
255 domain socket.
256
257 Administration
258 ==============
259
260 Keeping the database synced
261 ---------------------------
262
263 Consistency in the database is maintained by pushing the current status
264 of the backends to the master, and having the frontends stay up to date
265 with the master's database. Since the frontends resync themselves
266 entirely when they startup, downtime should not be a problem.
267 (While they are up they should be continuously receiving database
268 updates, as well when they lose connection to the master, they will
269 try to reconnect and resync their database upon reconnection)
270
271 Provided that the namespace of the backend servers is kept discrete
272 (with no mailboxes existing on the same server), it is not a big deal
273 to resync the mupdate master using ``ctl_mboxlist -m``. If two servers
274 do have the same mailbox, this will need to be resolved before database
275 consistency can be guaranteed.
276
277 Moving Mailboxes between backends
278 ---------------------------------
279
280 There is currently no 100% foolproof way to do this, however, if you
281 issue a rename command to a frontend (as you would to move a mailbox
282 between partitions), and replace the partition name with the name of
283 the new backend, it will move the mailbox to the indicated backend. You
284 can also use the format ``backend.domain.com!partition`` to move to a
285 specific partition (otherwise the default partition will be used).
286
287 In cyradm, this looks like::
288
289 cyrus.andrew.cmu.edu> rename user.bcyrus user.bcyrus mail2.andrew.cmu.edu!u2
290
291 Note that since seen state is stored per-user, it is possible that when
292 moving a shared mailbox users will have strange effects. The general
293 rule is that moving an INBOX will move the entire user (including all
294 sub-mailboxes to the INBOX, and seen state, and subscriptions, and
295 sieve scripts, etc). The seen state is merged with the seen state on
296 the new backend, so that no data is lost (seen state is also the only
297 part left behind on the source backend). In the case of any other
298 mailbox, however, only that individual mailbox is moved. If it is a
299 quota root, the new quota root is instantiated on the new server, but
300 otherwise quotas can appear to be violated, since each backend only
301 takes care of its own quota.
302
303 In general, it's better to leave trees of mailboxes on the same server,
304 and not move submailboxes of inboxes between servers.
305
306 Adding additional backend servers
307 ---------------------------------
308
309 This is very easy to do, simply configure an empty backend server and
310 set its ``mupdate_server`` parameter to point at the mupdate master.
311 Then, issue mailbox creates to it as you would any other backend
312 server.
313
314 Distributing Mailboxes between Back Ends
315 ----------------------------------------
316
317 Several options exist within :cyrusman:`imapd.conf(5)` to aid in the
318 distribution of new users and mailboxes within a murder; across servers
319 and partitions. We recommend exploring these:
320
321 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
322 :start-after: startblob partition_select_mode
323 :end-before: endblob partition_select_mode
324
325 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
326 :start-after: startblob partition_select_exclude
327 :end-before: endblob partition_select_exclude
328
329 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
330 :start-after: startblob partition_select_usage_reinit
331 :end-before: endblob partition_select_usage_reinit
332
333 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
334 :start-after: startblob partition_select_soft_usage_limit
335 :end-before: endblob partition_select_soft_usage_limit
336
337 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
338 :start-after: startblob serverlist
339 :end-before: endblob serverlist
340
341 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
342 :start-after: startblob serverlist_select_mode
343 :end-before: endblob serverlist_select_mode
344
345 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
346 :start-after: startblob serverlist_select_usage_reinit
347 :end-before: endblob serverlist_select_usage_reinit
348
349 .. include:: /imap/reference/manpages/configs/imapd.conf.rst
350 :start-after: startblob serverlist_select_soft_usage_limit
351 :end-before: endblob serverlist_select_soft_usage_limit
352
353
354 Backups
355 -------
356
357 .. :todo:
358 xxx, need to write stuff. You don't need to really backup the data
359 on the mupdate master or slaves, since this data can all be
360 generated directly from the backends quite easily.
361
362 Gotchas
363 =======
364
365 **Clients dealing with a pool of frontend servers**
366 Some clients may not be terribly efficient caching connections to a
367 pool of imap servers, this isn't a problem, as such, but it may
368 mean that you will see many more authentications than you are used
369 to.
370
371 **Kerberos issues**
372 If you are using kerberos authentication, you will want to ensure
373 that all your machines are keyed properly, as we have seen problems
374 with different clients trying to authenticate to different services
375 (e.g. imap.imap-pool instead of imap.pool-frontend-1), so test the
376 clients in use in your environment and be sure that they work with
377 whatever keying scheme you use.
378
379 **Clients dealing with referrals**
380 Some clients (we've had particular trouble with pine, though most
381 of these issues have now been resolved and new versions should be
382 OK (that is, pine > 4.44), but as referrals have not been
383 extensively used by any IMAP server until now, referrals are very
384 likely to not work correctly or have surprising effects.
385
386 **Clients dealing with getting a NO on LSUB commands**
387 Some clients (Outlook, for example) may behave poorly if an LSUB
388 command returns a NO, which may be the case if the backend server
389 with the user's inbox is down. We have, for example, seen this
390 result in the deletion of the disconnected message cache.
391
392 **Behavior of cyradm / some mailbox operations**
393 The behaviour of some administrative commands might be slightly
394 unexpected. For example, you can only issue a SETQUOTA to a
395 frontend server if the entire mailbox tree underneath where you are
396 setting the quota exists on the same backend server, otherwise you
397 will need to connect directly to the backend servers to perform the
398 needed changes. Similarly, mailboxes will be created on the same
399 backend server that their parent is in. In order to create them on
400 a different server (or to create a new top level mailbox) you will
401 need to connect directly to the desired backend server.
402
403 **Subscriptions**
404 If users want subscribe to a mailbox other than on their backend
405 home server, they won't be able to, unless you set
406 ``allowallsubscribe: t`` in the backend imapd.confs. This
407 essentially lets any string be subscribed to successfully.
408
409 **Restarting the mupdate master**
410 Because ``ctl_cyrusdb -r`` clears reservations on mailbox, if you
411 restart the mupdate master (and run recovery), then this could (we
412 suspect, very rarely) lead to inconsistencies in the mupdate
413 database.
414
415 Troubleshooting
416 ===============
417
418 **Mailbox operations are being denied**
419 This is an indication that the mupdate master may be down. Restart
420 it.
421
422 **Mailbox operations are not being seen by one or more frontends**
423 This indicates that the mupdate process on a slave may have died,
424 you may need to restart master. Alternatively, mupdate will retry
425 connections every 20 seconds or so for about 20 attempts if the
426 master does go down.
427
428 **A frontend's mailboxes.db is corrupt or out of sync**
429 Restart master on the frontend, and have the mupdate process
430 resynch the local database. You may need to remove the local
431 mailboxes database if the corruption is extreme.
432
433 **A mailbox's location keeps switching between two (or more) backend hosts.**
434 It probably actually exists on both hosts. Delete the mailbox from
435 all but one of the hosts, and run a ``ctl_mboxlist -m`` on the one
436 where you want it to actually live.
437
438 **Databases are never created on the frontends/slaves**
439 Check to ensure that the mupdate slave process is started, (is
440 prefork=1)
441
442 **mupdate crashes with SIGSERV when using STARTTLS**
443 The OpenSSL code in Cyrus Imap is for single-threaded applications and
444 mupdate is a multi-threaded application. Do not encrypt the communication
445 with mupdate. See also the discussion “SIGSEGV in cyrus-imapd 3.0.7 mupdate”
446 on `cyrus-devel from July 2018 <https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2018-July/>`_
447 and https://github.com/cyrusimap/cyrus-imapd/issues/2774 .