Debian Codegraph @

doc/html/_sources/imap/concepts/overview_and_concepts.rst.txt

raw
1 ########
2 Concepts
3 ########
4
5 This section gives an overview of several aspects of the Cyrus IMAP
6 server, as they relate to deployment. In an effort to reduce
7 duplication of information, we will often direct you to documentation
8 in other areas. Please do follow such referrals.
9
10 Access Control Lists
11 ********************
12
13 Access to each mailbox is controlled by access control
14 lists. Access Control Lists (ACLs) provide a powerful mechanism for
15 specifying the users, or groups of users, who have permission to access
16 the mailboxes, and the degree of that access.
17
18 An ACL is a list of zero or more entries. Each entry contains a
19 mailbox, an Access Control Identifier (ACI) and a set of rights. The
20 ACI specifies the user or group of users for which the entry
21 applies. The set of rights is one or more letters or digits, each
22 letter or digit conferring a particular privilege.
23
24 Working with ACLs
25 =================
26
27 ACLs are manipulated via these subcommands within the
28 :cyrusman:`cyradm(8)` program:
29
30 * :ref:`imap-reference-manpages-systemcommands-cyradm-setaclmailbox`
31 * :ref:`imap-reference-manpages-systemcommands-cyradm-listaclmailbox`
32 * :ref:`imap-reference-manpages-systemcommands-cyradm-deleteaclmailbox`
33
34 Sample ACL
35 ==========
36
37 A typical ACL is expressed like this:
38
39 .. parsed-literal::
40
41 **setaclmailbox** *mailbox* *id* *rights* [*id* *rights* ...]
42
43 where *mailbox* is the name of the mailbox to which the ACL is applied,
44 *id* is the ACI for the user or group for which the ACL applies, and
45 *rights* is a concatenated list of Access Rights from the list below.
46
47 A real world example may look like this:
48
49 ::
50
51 setaclmailbox user/bovik/public bovik all group:users lrsp anyone lrs
52
53 Here are samples illustrated via output from the ``listaclmailbox``
54 command in :cyrusman:`cyradm(8)`:
55
56 .. parsed-literal::
57
58 localhost> **listaclmailbox tech/%**
59 tech/Commits:
60 group:tech lrswipkxtea
61 anyone lrs
62 tech/abuse:
63 group:tech lrswipkxtecda
64 anyone lrsp
65 tech/security:
66 anyone lrsp
67 group:tech lrswipkxtecda
68 tech/support:
69 group:tech lrswipkxtecda
70 anyone lrsp
71
72 localhost> **listaclmailbox user/bovik/%**
73 user/bovik/Drafts:
74 bovik lrswipkxtecda
75 user/bovik/Sent:
76 bovik lrswipkxtecda
77 user/bovik/Sent Items:
78 bovik lrswipkxtecda
79 user/bovik/Spam:
80 anyone p
81 bovik lrswipkxtecda
82 user/bovik/Trash:
83 bovik lrswipkxtecda
84
85 Access Rights
86 =============
87
88 The following lists Access Rights that can be used in an Access Control
89 List entry.
90
91 l
92 The user may see that the mailbox exists (**lookup**).
93
94 r
95 The user may read the mailbox (**read**).
96
97 s
98 Keep per-user seen state (i.e. modify the "Seen" flag)
99 (**setseen**).
100
101 w
102 The user may modify flags and keywords other than "Seen" and
103 "Deleted". (**write**)
104
105 i
106 The user may insert (append) new messages into the mailbox
107 (**insert**).
108
109 p
110 The user may send email to the submission address for the mailbox
111 (**post**).
112
113 c
114 [**deprecated**: see ``k`` right, below.]
115
116 k
117 The user may create new mailboxes in this mailbox, delete the
118 current mailbox, or rename the mailbox (**create**).
119
120 x
121 The user may delete the mailbox itself. (**deletembox**)
122
123 t
124 The user may store the "Deleted" flag. In other words, delete
125 messages.
126
127 e
128 The user may Expunge messages which have the "Deleted" flag already
129 set (**expunge**).
130
131 d
132 This "legacy" right is treated by the software as a macro for ``te``
133 (**deletemsg** && **expunge**).
134
135 n
136 The user may store annotations for a message (**annotatemsg**)
137
138 a
139 The user may change the *Access Control Information* (ACI) on the
140 mailbox (**administer**).
141
142 For a complete reference to Access Rights, please see
143 :ref:`imap-admin-access-control-lists-rights-reference`
144
145 Rights are combined through concatenation. Please see
146 :ref:`imap-admin-access-control-combining-rights`
147
148 .. include:: /imap/reference/admin/access-control/defaults.rst
149 :start-after: _imap-admin-access-control-defaults:
150
151 .. include:: /imap/reference/admin/access-control/identifiers.rst
152 :start-after: _imap-admin-access-control-identifiers:
153 :end-before: _imap-admin-access-control-authorization-mechanisms:
154
155 Negative Rights
156 ===============
157
158 Any of the above defined identifiers may be prefixed with a ``-``
159 character. The associated rights are then removed from that identifier.
160 These are referred to as *negative rights*.
161
162 Calculating a Users' Rights
163 ===========================
164
165 To calculate the set of rights granted to a user, the server first
166 calculates the union of all of the rights granted to the user and to
167 all groups the user is a member of. The server then calculates and
168 removes the union of all the negative rights granted to the user and to
169 all groups the user is a member of.
170
171 ::
172
173 anyone lrsp
174 fred lwi
175 -anonymous s
176
177 The user ``fred`` will be granted the rights ``lrswip`` and the
178 anonymous user will be granted the rights ``lrp``.
179
180
181 .. _imap-concepts-login-authentication:
182
183 Login Authentication
184 ********************
185
186 This section discusses different types of authentication (ways of logging in) that can be used with Cyrus IMAP.
187
188 The Cyrus IMAP server uses the Cyrus SASL library for authentication. This section describes how to configure SASL with use with Cyrus imapd. Please consult the :ref:`Cyrus SASL System Administrator's Guide <cyrussasl:sasl-index>` for more detailed, up-to-date information.
189
190 Anonymous Login
191 ===============
192
193 Regardless of the SASL mechanism used by an individual connection, the
194 server may support anonymous login. If the ``allowanonymouslogin``
195 option in :cyrusman:`imapd.conf(5)` is turned on, then the server will
196 permit plaintext password logins using the user ``anonymous`` and any
197 password.
198
199 Additionally, the server will enable any SASL mechanisms that allow anonymous logins.
200
201 Plaintext Authentication
202 ========================
203
204 The SASL library has several ways of verifying plaintext passwords. Plaintext passwords are passed either by the IMAP ``LOGIN`` command or by the SASL ``PLAIN`` mechanism (under a TLS layer).
205
206 * PAM
207 * GSSAPI: Plaintext passwords are verified by obtaining a ticket for the server's Kerberos identity, to protect against Kerberos server spoofing attacks.
208
209 * ``/etc/passwd``
210 * ``/etc/shadow``: ``sasl_auto_transition`` automatically creates secrets for shared secret authentication when given a password.
211
212 The method of plaintext password verification is always through the SASL library, even in the case of the internal LOGIN command. This is to allow the SASL library to be the only source of authentication information. You'll want to look at the ``sasl_pwcheck_method`` option in the SASL documentation to understand how to configure a plaintext password verifier for your system.
213
214 To disallow the use of plaintext passwords for authentication, you can set ``allowplaintext: no`` in imapd.conf. This will still allow PLAIN under TLS, but IMAP LOGIN commands will now fail.
215
216 Kerberos Logins
217 ===============
218
219 The Kerberos SASL mechanism supports the ``GSSAPI`` authentication mechanism. The mechanism requires that a ``keytab`` file exist in the location given in the ``sasl_keytab`` configuration option. The ``keytab`` file must be readable by the Cyrus server and must contain a ``$service/$host@$realm`` service key. ``$host`` is the first component of the server's host name, the FQDN or IP address (whatever the client uses). ``$realm`` is the server's Kerberos realm. ``$service`` is name of the service: ``imap``, ``HTTP``, ``sieve``.
220
221 The server will permit logins by identities in the local realm and identities in the realms listed in the ``loginrealms`` option in :cyrusman:`imapd.conf(5)`.
222
223 If the ``loginuseacl`` configuration option is turned on, than any Kerberos identity that is granted the ``a`` right on the user's ``INBOX`` is permitted to log in as that user.
224
225 Shared Secrets Logins
226 =====================
227
228 The SCRAM mechanisms require the user and the server to share a secret (generally a password) that can be used for comparison without actually passing the password in the clear across the network. For these mechanisms, you will need to supply a source of passwords, such as the sasldb (which is described more fully in the :ref:`Cyrus SASL distribution <cyrussasl:sasl-index>`).
229
230 Quotas
231 ******
232
233 Quotas allow server administrators to limit resources used by hierarchies of mailboxes on the server.
234
235 Working with Quotas
236 ===================
237
238 Quotas are manipulated via these subcommands within the
239 :cyrusman:`cyradm(8)` program:
240
241 * :ref:`imap-reference-manpages-systemcommands-cyradm-setquota`
242 * :ref:`imap-reference-manpages-systemcommands-cyradm-listquota`
243 * :ref:`imap-reference-manpages-systemcommands-cyradm-listquotaroot`
244
245 .. include:: /imap/reference/admin/quotas.rst
246 :start-after: _imap-admin-quotas-repair:
247 :end-before: _imap-admin-quotas-config:
248
249 .. include:: /imap/reference/admin/quotas/quotatypes.rst
250 :start-after: _imap-admin-quotas-types:
251
252 .. include:: /imap/reference/admin/quotas/quotaroots.rst
253 :start-after: _imap-admin-quotas-roots:
254
255 Controlling Quota Behavior
256 ==========================
257
258 How restrictive quotas will be may be tailored to the needs of different
259 sites, via the use of several settings in :cyrusman:`imapd.conf(5)`.
260
261 Please consult the :ref:`imap-admin-quotas-config` section of the Cyrus
262 IMAP Administrator Guide for complete details.
263
264 Mail Delivery Behavior
265 ======================
266
267 Mailboxes Near Quota
268 --------------------
269
270 Normally, in order for a message to be *appended* into a mailbox, the
271 quota root for the mailbox must have enough unused storage that
272 appending the message will not cause the quota to go over limit.
273
274 Mail delivery (posting) is a special case. In order for a message to be
275 delivered to a mailbox, the quota root for the mailbox merely need not
276 already be over the limit *in the default configuration*.
277
278 As long as usage is not over the limit, new messages may be delivered
279 regardless of size, unless ``lmtp_strict_quota: on`` is set in
280 :cyrusman:`imapd.conf(5)`. In that case, delivery of messages will be
281 rejected would such delivery exceed quota.
282
283 If a delivery puts the mailbox's usage over the quota, the server will
284 issue an alert notifying the user that usage is close to or over the
285 limit, permitting them to correct it. If delivery were not permitted in
286 this case, the user would have no practical way of knowing that there
287 was mail that could not be delivered.
288
289 .. note::
290
291 While the Cyrus IMAP server may from time to time issue alerts,
292 there is great variability in how IMAP clients handle these.
293 Further, such alerts are only visible to users *while they are
294 connected*.
295
296 Therefore, many sites find it preferable to install cron jobs which
297 use the :cyrusman:`quota(8)` command to produce periodic reports of
298 users at or near quota, so administrators may nag them or so that
299 warnings may be issued to users via some other mechanism.
300
301 Mailboxes Over Quota
302 --------------------
303
304 If the usage is over the limit, mail delivery will fail with a temporary
305 error (LMTP error 452), unless ``lmtp_over_quota_perm_failure: on``
306 is set in :cyrusman:`imapd.conf(5)` in which case a permanent error
307 (LMTP error 552) will be returned.
308
309 A temporary error will *typically* cause the delivery system to requeue
310 the message and re-attempt delivery for a few days (permitting the user
311 time to notice and correct the problem) before returning the mail to
312 the sender.
313
314 .. Note::
315
316 Such requeuing behaviour is controlled by the MTA (i.e. Sendmail,
317 EXIM or Postfix) and as such is outside the purview of this
318 document.
319
320 Quota Warnings Upon Select When User Has ``d`` Rights
321 =====================================================
322
323 When a user selects a mailbox whose quota root has usage that is close to or
324 over the limit and the user has ``d`` rights on the mailbox, the server will
325 issue an alert notifying the user that usage is close to or over the limit.
326 The threshold of usage at which the server will issue quota warnings is set
327 by the ``quotawarnpercent`` configuration option.
328
329 The server only issues warnings when the user has ``d`` rights because only
330 users with ``d`` rights are capable of correcting the problem.
331
332 Quotas and Partitions
333 =====================
334
335 Quota roots are independent of partitions. A single quota root can apply to mailboxes in different partitions.
336
337 .. include: /imap/reference/admin/quotas.rst
338 :start-after:`_imap-admin-quotas-database:`
339 :end-before:`_imap-admin-quotas-convert-db:`
340
341
342 New Mail Notification
343 *********************
344
345 The Cyrus IMAP server comes with a notification daemon which supports
346 multiple mechanisms for notifying users of new mail. Notifications can
347 be configured to be sent upon normal delivery (``MAIL`` class) and/or
348 sent as requested by a Sieve script (``SIEVE`` class).
349
350 By default, both types of notifications are disabled. Notifications are
351 enabled by using one or both of the following configuration options:
352
353 * the ``mailnotifier`` option selects the :cyrusman:`notifyd(8)` method
354 to use for ``MAIL`` class notifications
355
356 * the ``sievenotifier`` option selects the :cyrusman:`notifyd(8)`
357 method to use for ``SIEVE`` class notifications (when no method is
358 specified by the Sieve action)
359
360
361 Partitions
362 **********
363
364 Partitions allow administrators to store different mailboxes in different parts of the Unix filesystem. This is intended to be used to allow hierarchies of mailboxes to be spread across multiple disks.
365
366 Specifying Partitions with "create"
367 ===================================
368
369 When an administrator creates a new mailbox, the name of the partition for the mailbox may be specified using an optional second argument to the "create" command. Non-administrators are not permitted to specify the partition of a mailbox. If the partition is not specified, then the mailbox inherits the partition of its most immediate parent mailbox. If the mailbox has no parent, it gets the partition specified in the "defaultpartition" configuration option.
370
371 The optional second argument to the "create" command can usually be given only when using a specialized Cyrus-aware administrative client such as ``cyradm``.
372
373 Changing Partitions with "rename"
374 =================================
375
376 An administrator may change the partition of a mailbox by using the
377 rename command with an optional third argument. When a third argument
378 to rename is given, the first and second arguments can be the
379 same; this changes the partition of a mailbox without changing its
380 name. If a third argument to rename is not given and the first
381 argument is not ``INBOX``, the partition of a mailbox does not change.
382 If a third argument to rename is not given and the first argument is
383 ``INBOX``, the newly created mailbox gets the same partition it would
384 get from the ``create`` command.
385
386 News
387 *****
388
389 Cyrus has the ability to export Usenet via IMAP and/or export shared
390 IMAP mailboxes via an NNTP server which is included with Cyrus.
391
392 POP3 Server
393 ***********
394
395 The Cyrus IMAP server software comes with a compatibility POP3 server.
396 Due to limitations in the POP3 protocol, the server can only access a
397 user's ``INBOX`` and only one instance of a POP3 server may exist for any
398 one user at any time. While a POP3 server has a user's ``INBOX`` open,
399 expunge operations from any concurrent IMAP session will fail.
400
401 When Kerberos login authentication is being used, the POP3 server
402 uses the server identity
403 ``pop.host@realm`` instead of
404 ``imap.host@realm``, where
405 ``host`` is the first component of the server's host
406 name and ``realm`` is the server's Kerberos realm.
407
408 The syslog facility
409 *******************
410
411 The Cyrus IMAP server software sends log messages to the ``local6``
412 syslog facility. The severity levels used are:
413
414 * **CRIT** - Critical errors which probably require prompt administrator action
415 * **ERR** - I/O errors, including failure to update quota usage. The syslog message includes the specific file and Unix error.
416 * **WARNING** - Protection mechanism failures, client inactivity timeouts
417 * **NOTICE** - Authentications, both successful and unsuccessful
418 * **INFO** - Mailbox openings, duplicate delivery suppression
419
420 Mail Directory Recovery
421 ***********************
422
423 This section describes the various databases used by the Cyrus IMAP
424 server software and what can be done to recover from various
425 inconsistencies in these databases.
426
427 Reconstructing Mailbox Directories
428 ==================================
429
430 The largest database is the mailbox directories. Each
431 mailbox directory contains the following files:
432
433 message files
434 There is one file per message, containing the message in :rfc:`822` format. Lines in the message are separated by CRLF, not just LF. The file name of each message is the message's UID followed by a dot (.).
435
436 In netnews newsgroups, the message files instead follow the format and naming conventions imposed by the netnews software.
437
438 ``cyrus.header``
439 This file contains a magic number and variable-length information about the mailbox itself.
440
441 ``cyrus.index``
442 This file contains fixed-length information about the mailbox itself and each message in the mailbox.
443
444 ``cyrus.cache``
445 This file contains variable-length information about each message in the mailbox.
446
447 ``cyrus.seen``
448 This file contains variable-length state information about each reader of the mailbox who has ``s`` permissions.
449
450 The ``reconstruct`` program can be used to recover from
451 corruption in mailbox directories. If ``reconstruct`` can find
452 existing header and index files, it attempts to preserve any data in
453 them that is not derivable from the message files themselves. The
454 state ``reconstruct`` attempts to preserve includes the flag
455 names, flag state, and internal date. ``Reconstruct``
456 derives all other information from the message files.
457
458 An administrator may recover from a damaged disk by restoring message
459 files from a backup and then running reconstruct to regenerate what it
460 can of the other files.
461
462 The ``reconstruct`` program does not adjust the quota usage
463 recorded in any quota root files. After running reconstruct, it is
464 advisable to run ``quota -f`` (described below) in order to fix
465 the quota root files.
466
467 Reconstructing the Mailboxes File
468 =================================
469
470 .. note::
471
472 CURRENTLY UNAVAILABLE
473
474 The mailboxes file in the configuration directory is the most critical
475 file in the entire Cyrus IMAP system. It contains a sorted list of
476 each mailbox on the server, along with the mailboxes quota root and
477 ACL.
478
479 To reconstruct a corrupted mailboxes file, run the ``reconstruct
480 -m`` command. The ``reconstruct`` program, when invoked
481 with the ``-m`` switch, scavenges and corrects whatever data it
482 can find in the existing mailboxes file. It then scans all partitions
483 listed in the imapd.conf file for additional mailbox directories to
484 put in the mailboxes file.
485
486 The ``cyrus.header`` file in each mailbox directory stores a
487 redundant copy of the mailbox ACL, to be used as a backup when
488 rebuilding the mailboxes file.
489
490 Reconstructing Quota Roots
491 ==========================
492
493 .. note::
494
495 The following instructions are valid where ``quota_db: quotalegacy``
496 is set in :cyrusman:`imapd.conf(5)`. If your site uses a different
497 quota DB type, then these steps do not apply.
498
499 The subdirectory ``quota`` of the configuration directory (specified in
500 the ``configdirectory`` configuration option) contains one file per
501 quota root, with the file name being the name of the quota root. These
502 files store the quota usage and limits of each of the quota roots.
503
504 The ``quota`` program, when invoked with the ``-f``
505 switch, recalculates the quota root of each mailbox and the quota
506 usage of each quota root.
507
508 Removing Quota Roots
509 ====================
510
511 To remove a quota root, remove the quota root's file. Then run
512 ``quota -f`` to make the quota files consistent again.
513
514 Subscriptions
515 =============
516
517 The subdirectory ``user`` of the configuration directory contains user
518 subscriptions. There is one file per user, with a filename of the
519 userid followed by ``.sub``. Each file contains a sorted list of
520 subscribed mailboxes.
521
522 There is no program to recover from damaged subscription files. A
523 site may recover from lost subscription files by restoring from backups.
524
525 Configuration Directory
526 ***********************
527
528 Many objects in the configuration directory are discussed in
529 the Database Recovery section. This section documents two
530 other directories that reside in the configuration directory.
531
532 Log Directory
533 =============
534
535 The subdirectory ``log`` under the configuration directory permits
536 administrators to keep protocol telemetry logs on a per-user basis.
537
538 If a subdirectory of ``log`` exists with the same name as a user, the
539 IMAP and POP3 servers will keep a telemetry log of protocol sessions
540 authenticating as that user. The telemetry log is stored in the
541 subdirectory with a filename of the server process-id and starts with
542 the first command following authentication.
543
544 Proc Directory
545 ==============
546
547 The subdirectory ``proc`` under the configuration directory
548 contains one file per active server process. The file name is the ASCII
549 representation of the process id and the file contains the following
550 tab-separated fields:
551
552 * hostname of the client
553 * login name of the user, if logged in
554 * selected mailbox, if a mailbox is selected
555
556 The file may contain arbitrary characters after the first newline
557 character.
558
559 The ``proc`` subdirectory is normally be cleaned out on
560 server reboot.
561
562 Message Delivery
563 ****************
564
565 Mail transport agents such as Sendmail, Postfix, or Exim communicate
566 with the Cyrus server via LMTP (the Local Mail Transport Protocol)
567 implemented by the LMTP daemon. This can be done either directly by the
568 MTA (prefered, for performance reasons) or via the ``deliver`` LMTP
569 client.
570
571 Local Mail Transfer Protocol (lmtp)
572 ===================================
573
574 LMTP, the Local Mail Transfer Protocol, is a variant of SMTP design for
575 transferring mail to the final message store. LMTP allows MTAs to deliver
576 "local" mail over a network. This is an easy optimization so that the
577 IMAP server doesn't need to maintain a queue of messages or run an
578 MTA.
579
580 The Cyrus server implements LMTP via the ``lmtpd`` daemon. LMTP
581 can either be used over a network via TCP or local via a UNIX domain
582 socket. There are security differences between these two alternatives; read
583 more below.
584
585 For final delivery via LMTP over a TCP socket, it is necessary to use
586 LMTP AUTH. This is accomplished using SASL to authenticate the delivering
587 user. If your mail server is performing delivery via LMTP AUTH (that is,
588 using a SASL mechanism), you will want their authentication id to be an
589 LMTP admins (either via the ``admins`` imapd.conf option or via the
590 ``<service>_admins`` option, typically ``lmtp_admins``).
591
592 Alternatively you may deliver via LMTP to a unix domain socket, and the
593 connection will be preauthenticated as an administrative user (and access
594 control is accomplished by controlling access to the socket).
595
596 Note that if a user has a sieve script, the sieve script runs authorized
597 as *that* user, and the rights of the posting user are ignored for the purposes
598 of determining the outcome of the sieve script.
599
600 Single Instance Store
601 =====================
602
603 If a delivery attempt mentions several recipients (only possible if
604 the MTA is speaking LMTP to ``lmtpd``), the server attempts to
605 store as few copies of a message as possible. It will store one copy
606 of the message per partition, and create hard links for all other
607 recipients of the message.
608
609 Single instance store can be turned off by using the
610 "singleinstancestore" flag in the configuration file.
611
612 Duplicate Delivery Suppression
613 ==============================
614
615 A message is considered a duplicate if two copies of a message with
616 the same message-id and the same envelope recipient are received.
617 Cyrus uses the duplicate delivery database to hold this information,
618 and it looks approximately 3 days back in the default install.
619
620 Duplicate delivery suppression can be turned off by using the
621 "duplicatesuppression" flag in the configuration file.
622
623 Sieve, a Mail Filtering Language
624 ********************************
625
626 Sieve is a mail filtering language that can filter mail into an appropriate
627 IMAP mailbox as it is delivered via lmtp.
628
629 Cyrus Murder, the IMAP Aggregator
630 *********************************
631
632 Cyrus now supports the distribution of mailboxes across a number of IMAP
633 servers to allow for horizontal scalability.