Debian Codegraph @

doc/html/_sources/imap/download/release-notes/2.5/x/2.5.0.rst.txt

raw
1 :tocdepth: 3
2
3 ==============================
4 Cyrus IMAP 2.5.0 Release Notes
5 ==============================
6
7 .. IMPORTANT::
8
9 Make sure to read the :ref:`imap-relnotes-2.5.0-upgrading` notes
10 (all of them).
11
12 * :ref:`relnotes-2.5.0-new-features`
13 * :ref:`relnotes-2.5.0-config-option-changes`
14 * :ref:`relnotes-2.5.0-development-autoconf`
15 * :ref:`relnotes-2.5.0-development-phabricator`
16 * :ref:`relnotes-2.5.0-development-documentation`
17
18 Download from GitHub:
19
20 * https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.0/cyrus-imapd-2.5.0.tar.gz
21 * https://github.com/cyrusimap/cyrus-imapd/releases/download/cyrus-imapd-2.5.0/cyrus-imapd-2.5.0.tar.gz.sig
22
23 .. _imap-relnotes-2.5.0-upgrading:
24
25 Upgrading to Cyrus IMAP 2.5.0
26 =============================
27
28 It is strongly recommended to shut down the Cyrus IMAP services before
29 performing the upgrade, as the newer binaries will end up writing to
30 ``mailboxes.db`` in a way that is not compatible with the older binaries
31 (that would otherwise still be running).
32
33 You can start the server immediately after upgrading, however.
34
35 Underscores in ``cmd`` Names in :cyrusman:`cyrus.conf(5)`
36 ---------------------------------------------------------
37
38 Underscores (the ``_`` character) are no longer allowed in the
39 ``START``, ``SERVICES`` and ``EVENTS`` sections of
40 :cyrusman:`cyrus.conf(5)`, as they interfere with configuration options
41 in :cyrusman:`imapd.conf(5)` being prefixed by service names and an
42 underscore (``_``) character.
43
44 Database Format Upgrade for Mailboxes
45 -------------------------------------
46
47 Upgrading to Cyrus IMAP 2.5.0 recommends the upgrade of database
48 formats, for which the reconstruct utility has been modified allowing
49 administrators to run:
50
51 .. parsed-literal::
52
53 # :command:`/usr/lib/cyrus-imapd/reconstruct -V max` [options]
54
55 This command can be run at the administrator's convenience, and while
56 running the database format upgrade process is not strictly required, it
57 is certainly strongly recommended.
58
59 Releases prior to 2.5.0, namely the 2.4 series, upgraded ``cyrus.index``
60 database formats in-line (i.e. as the mailbox in question as being
61 used). This may have caused an I/O storm in some situations, which 2.5.0
62 aims to address.
63
64 The minor version of the ``cyrus.index`` file in 2.5.0 is **13**. While
65 mailboxes with an older format will continue to work "forever", newer
66 features such as annotation quotas will fail, as there is no space in
67 the database formats prior to version 13 to store the new feature's
68 information.
69
70 Until you upgrade the database format, you may experience slightly
71 inconsistent search results, due to the ``cyrus.cache`` format in Cyrus
72 IMAP versions prior to 2.4.0 running words together and not obeying word
73 boundaries (as well as versions in the 2.4 and 2.5 series).
74
75 If you were running Cyrus 2.3.x with ``expunge_mode: delayed``, then the
76 contents of the ``cyrus.expunge`` file will be removed the first time
77 the mailbox is opened, and the mailbox will act as if ``expunge_mode``
78 was set to ``immediate`` -- until it is upgraded.
79
80 This is because version 2.3 mailboxes used the ``cyrus.expunge`` file,
81 and it's not worth the complexity to try to recreate that file.
82
83 This upgrade path supports upgrades from Cyrus IMAP version 2.2.12 and
84 later versions.
85
86 .. IMPORTANT::
87
88 The above command can run while the Cyrus IMAP services are in
89 operation, and does **not** interfere with the service's operations.
90
91 .. NOTE::
92
93 Additional options can be specified to :cyrusman:`reconstruct(8)` to
94 more granularly specify which mailboxes' :file:`cyrus.index` is to
95 be upgraded.
96
97 Quota Fixes and Enhancements
98 ----------------------------
99
100 After all mailboxes are upgraded, you should run ``quota -f`` to make
101 all message and folder counts are correct. The new quota types that
102 Cyrus IMAP 2.5.0 supports require the counts to need to be accurate.
103
104 Cyrus IMAP Murder Topologies
105 ----------------------------
106
107 Environments that run a Cyrus IMAP Murder topology will want to upgrade
108 their backends before they upgrade their frontends.
109
110 Greater Memory Footprint
111 ------------------------
112
113 .. include:: /assets/cyrus-more-memory-post23.rst
114
115 .. _relnotes-2.5.0-new-features:
116
117 New Features
118 ============
119
120 .. _relnotes-2.5.0-index-namelock-release:
121
122 Index Namelock Release
123 ----------------------
124
125 Long-running (idling) connections may have previously intervened with
126 mailbox deletions and general cleanup work, resulting in one connection
127 blocking another from, for example, deleting and recreating a mailbox:
128
129 .. code-block:: bash
130 :linenos:
131
132 C1: SELECT "Foo"
133 C2: DELETE "Foo"
134 C2: CREATE "Foo"
135
136 The ``CREATE`` command on line 3 (by client 2, while client 1 still has
137 the ``Foo`` mailbox selected) would have failed before, but does now
138 succeed.
139
140 Extended Quota Types
141 --------------------
142
143 New ways to restrict resource usage:
144
145 * Number of Folders,
146 * Number of Messages,
147 * Number of Annotations
148
149 CalDAV and CardDAV Support
150 --------------------------
151
152 CalDAV and CardDAV support no longer live out-of-tree in the same GIT
153 repository, but are now mainstream and included in Cyrus IMAP 2.5.0.
154
155 This has been a major effort by Carnegie Mellon University and FastMail,
156 and continues to be, enriching the experience of Cyrus IMAP users
157 globally.
158
159 If upgrading to Cyrus IMAP 2.5 from one of the 2.4.17-caldav-beta
160 releases, you MUST run the ``dav_reconstruct`` utility for each of
161 your CalDAV users.
162
163 Support for RFC 5464: IMAP METADATA
164 -----------------------------------
165
166 Cyrus IMAP now fully supports :rfc:`5464`, *The IMAP METADATA
167 Extension*.
168
169 This also means the support for the ANNOTATEMORE draft for IMAP will
170 ultimately be dropped.
171
172 Aside from the trusted folder metadata, this also introduces message
173 annotations. Users will need to be given the ``n`` right to allow them
174 to set message annotations.
175
176 Event Notifications
177 -------------------
178
179 Various events occuring in Cyrus IMAP, among which mailbox, message and
180 access events, can now be pushed out through a side-channel and notify
181 client applications or provide other infrastructure with detailed
182 information.
183
184 Mailbox Distribution Enhancements: Backend and Partition Selection
185 ------------------------------------------------------------------
186
187 Thanks to the work of Julien Coloos and colleagues, a new mode is
188 available for server and partition selection upon mailbox creation.
189
190 Prior to Cyrus IMAP 2.5.0, the server and/or partition on which to
191 create a new mailbox was selected by detecting the largest amount of
192 absolute free disk space on all servers and partitions. The mailbox
193 distribution feature allows for more intelligent and flexible routines
194 to be used in the selection. Please see our Administrator Guide for
195 more details.
196
197 New Database Format for ``mailboxes.db``
198 ----------------------------------------
199
200 The database format for mailboxes.db has been upgraded, adding;
201
202 * A new mailbox type for deleted mailboxes.
203
204 In versions of Cyrus IMAP prior to 2.5.0, mailboxes that were
205 deleted may have become unavailable for actual cleanup expecting
206 another session on the same mailbox to clean up the directories and
207 files. See also :ref:`relnotes-2.5.0-index-namelock-release`.
208
209 * A key-value storage format is used, allowing for faster and better
210 parsing of :file:`mailboxes.db`, more granular updates to runtime
211 environments, and more sustainable future upgrades.
212
213 New Database Format ``twoskip``
214 -------------------------------
215
216 A new database format has been added, called ``twoskip`` [#]_.
217
218 This new database format is reputedly better, faster, safer and 64-bit
219 capable, as opposed to our former favorite ``skiplist``.
220
221 ``twoskip`` can be used for the following databases:
222
223 * ``annotation_db``
224 * ``duplicate_db``
225 * ``mboxkey_db``
226 * ``mboxlist_db``
227 * ``ptscache_db``
228 * ``quota_db``
229 * ``seenstate_db``
230 * ``subscription_db``
231 * ``statuscache_db``
232 * ``tls_sessions_db``
233 * ``user_deny_db``
234
235 Miscellaneous
236 -------------
237
238 Allowing Undefined Annotations
239 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
240
241 Cyrus IMAP 2.5.0 allows administrators to configure that undefined
242 annotations should be allowed, using a new
243 ``annotation_allow_undefined`` setting in :cyrusman:`imapd.conf(5)`.
244
245 Catchall Mailbox for LMTP
246 ^^^^^^^^^^^^^^^^^^^^^^^^^
247
248 Thanks to the work by Carsten Hoeger and Ralf Haferkamp, this new
249 feature enables administrators to configure a target mailbox for mail
250 delivered through LMTP to targetted mailboxes that do not exist.
251
252 For example, a mail that LMTP would deliver to ``user/bovik``, which
253 for the sake of argument does not exist in this example, setting
254 ``lmtp_catchall_mailbox`` to ``admin`` will instead deliver the mail
255 to ``user/admin``.
256
257 .. NOTE::
258 **Mailbox name, not Email Address**
259
260 Note that **lmtp_catchall_mailbox** must be a user mailbox name,
261 not an email address. Also note that the **user/** namespace
262 indicator as well as the hierarchy separator are to be omitted.
263
264 Does this impact lmtp_fuzzy_mailbox_match?
265 ++++++++++++++++++++++++++++++++++++++++++
266
267 Environments that have ``lmtp_fuzzy_mailbox_match`` enabled, in order
268 to have LMTP seek from the targetted, non-existent mailbox sub-folder
269 (example: ``user/bovik/spam/probably``) all the way to the toplevel
270 mailbox folder (i.e. ``user/bovik``) until it finds a mailbox
271 (sub-)folder that does exist (example: ``user/bovik/spam``), are not
272 impacted by this setting.
273
274 Can the lmtp_catchall_mailbox include the path to a sub-folder of a target mailbox?
275 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
276
277 UNCONFIRMED
278
279 Can the lmtp_catchall_mailbox be a shared folder?
280 +++++++++++++++++++++++++++++++++++++++++++++++++
281
282 UNCONFIRMED
283
284 Callout for SETMETADATA
285 ^^^^^^^^^^^^^^^^^^^^^^^
286
287 A callout program can be called when annotations are set, configured
288 through ``annotation_callout``.
289
290 Host & User Login Restrictions
291 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
292
293 Logins can now be restricted on a per host (source IP address) or per
294 user basis, using the settings ``maxlogins_per_host`` and
295 ``maxlogins_per_user``.
296
297 .. _relnotes-2.5.0-config-option-changes:
298
299 Configuration Option Changes and Enhancements
300 =============================================
301
302 .. IMPORTANT::
303
304 While it is not mandatory to update your configuration file with
305 these new settings, not doing so may have undesired side-effects,
306 including but not limited to deprecation warnings in log messages.
307
308 Option Name Changes for ``autocreate``
309 --------------------------------------
310
311 The options related to automatic creation of user mailboxes and
312 sub-folders (aka. *autocreate*) have been changed to hold a prefix of
313 ``autocreate_``.
314
315 The following *autocreate* options are now available:
316
317 **autocreate_inbox_folders** (was: ``autocreateinboxfolders``)
318
319 ``autocreate_inbox_folders`` controls which folders to create in
320 addition to the INBOX folder.
321
322 Separate the folder names by ``|``.
323
324 **autocreate_post** (was: ``createonpost``)
325
326 Controls whether or not to create a folder when a message is first
327 posted to it (by LTMP).
328
329 **autocreate_quota** (was: ``autocreatequota``)
330
331 When creating a user mailbox, set the quota for that mailbox to the
332 value of this configuration option.
333
334 **autocreate_quota_messages** (not available)
335
336 When creating a user mailbox, set the message quota (maximum number
337 of messages allowed in the folder hierarchy) to the value of this
338 configuration option.
339
340 **autocreate_sieve_folders** (was: ``autosievefolders``)
341
342 Limit the folders that can be created automatically by a Sieve
343 script performing a "fileinto" action, to the folders listed in
344 this configuration option.
345
346 Separate the folder names by ``|``.
347
348 **autocreate_sieve_script** (unchanged)
349
350 When creating a user mailbox, associate the Sieve script configured
351 here.
352
353 **autocreate_sieve_script_compile** (was: ``generate_compiled_sieve_script``)
354
355 Whether or not to compile the Sieve script configured by
356 ``autocreate_sieve_script``.
357
358 **autocreate_sieve_script_compiled** (was: ``autocreate_sieve_compiled_script``)
359
360 When creating a user mailbox, associate the already compiled Sieve
361 script configured here.
362
363 **autocreate_subscribe_folders** (was: ``autosubscribeinboxfolders``)
364
365 List the folder names to which the user for which a mailbox is
366 being created should be subscribed.
367
368 .. NOTE::
369
370 All folders listed here are considered to reside in the
371 personal namespace.
372
373 Separate the folder names by ``|``.
374
375 **autocreate_subscribe_sharedfolders** (was: ``autosubscribesharedfolders``)
376
377 List the folder names of shared folders to which the user for which
378 a mailbox is being automatically created should be subscribed.
379
380 Separate the folder names by ``|``.
381
382 **autocreate_subscribe_sharedfolders_all** (was: ``autosubscribe_all_sharedfolders``)
383
384 Rather than subscribe the user for which a mailbox is being
385 automatically created to some shared folders, simply subscribe the
386 user to all shared folders.
387
388 **autocreate_users** (unchanged)
389
390 Limit the users for which mailboxes may be created to the list
391 configured here.
392
393 Default Change: ``delete_mode``
394 -------------------------------
395
396 The default for the :cyrusman:`imapd.conf(5)` configuration option
397 ``delete_mode`` has changed from ``immediate`` to ``delayed``.
398
399 This causes mail folders that are deleted by a client to not
400 immediately dissappear from the filesystem. Instead, they are renamed
401 to a deleted namespace that is visible only to administrators.
402
403 A separate job ``cyr_expire -D $x`` is to be included in the master
404 service configuration file :cyrusman:`cyrus.conf(5)`, specifically in
405 the EVENTS section. ``$x`` is a number of days to keep already deleted
406 folders.
407
408 **Example section of :cyrusman:`cyrus.conf(5)`**
409
410 .. parsed-literal::
411
412 EVENTS {
413 deleteprune cmd="cyr_expire -D 69" at=0430
414 }
415
416 In the aforementioned example, folders are purged from the filesystem
417 only after 2 times 31 plus 7 days, corresponding with 2 cycles of a
418 monthly (full, virtual) backup of which one might fail.
419
420 Default Change: ``expunge_mode``
421 --------------------------------
422
423 The default for the :cyrusman:`imapd.conf(5)` configuration option
424 ``expunge_mode`` has changed from ``default`` to ``delayed``.
425
426 This causes the mail message files associated with messages that are
427 flagged as \Deleted in a folder that is subsequently expunged, or
428 individual messages that are expunged, to not be removed from the
429 filesystem directly.
430
431 A separate job ``cyr_expire -X $x`` is to be included in the master
432 service configuration file :cyrusman:`cyrus.conf(5)`, specifically in
433 the EVENTS section. ``$x`` is a number of days to keep the message
434 files on the filesystem.
435
436 **Example section of :cyrusman:`cyrus.conf(5)`**
437
438 .. parsed-literal::
439
440 EVENTS {
441 expungeprune cmd="cyr_expire -X 69" at=0430
442 }
443
444 In the aforementioned example, message files are purged from the
445 filesystem only after 2 times 31 plus 7 days, corresponding with 2
446 cycles of a monthly (full, virtual) backup of which one might fail.
447
448 Option Name Changes for ``ldap_tls_*``
449 --------------------------------------
450
451 Configuration option names for LDAP SSL/TLS configuration in
452 :cyrusman:`imapd.conf(5)` have been changed:
453
454 **ldap_ca_dir** (was: ``ldap_tls_cacert_dir``)
455
456 **ldap_ca_file** (was: ``ldap_tls_cacert_file``)
457
458 **ldap_client_cert** (was: ``ldap_tls_cert``)
459
460 **ldap_verify_peer** (was: ``ldap_tls_check_peer``)
461
462 **ldap_ciphers** (was: ``ldap_tls_ciphers``)
463
464 **ldap_client_key** (was: ``ldap_tls_key``)
465
466 Option Name Changes for ``tls_*``
467 ---------------------------------
468
469 Configuration option names for SSL/TLS configuration in
470 :cyrusman:`imapd.conf(5)` have been changed to better reflect how
471 they are used, as enhancements would otherwise create great confusion.
472
473 **tls_client_ca_dir** (was: ``tls_ca_path``)
474
475 **tls_client_ca_file** (was: ``tls_ca_file``)
476
477 The former ``tls_ca_*`` configuration options specified one or more
478 SSL Certificate Authority certificates against which SSL
479 certificates offered by clients could be verified.
480
481 In a Cyrus IMAP Murder topology however, Cyrus IMAP servers
482 themselves become clients of other Cyrus IMAP servers, but may not
483 have been issued certificates under the same verification chain.
484
485 With the (too) generic names for ``tls_ca_*`` configuration options out
486 of the way, Cyrus IMAP 2.5.0 adds the following configuration options:
487
488 **tls_server_cert** (was: ``tls_cert_file``)
489
490 **tls_server_key** (was: ``tls_key_file``)
491
492 Server SSL certificate and key to use for connections from
493 clients.
494
495 **tls_sessions_db** (was: ``tlscache_db``)
496
497 **tls_sessions_db_path** (was: ``tlscache_db_path``)
498
499 New Options for ``tls_*``
500 -------------------------
501
502 **tls_client_cert** (<none>)
503
504 **tls_client_key** (<none>)
505
506 Client SSL certificate and key to use when cyrus-imapd behaves as
507 a client (to other cyrus-imapd server (instances)).
508
509 **tls_client_ca_file** (<none>)
510
511 **tls_client_ca_dir** (<none>)
512
513 Certificate Authority file or directory used to verify client SSL
514 certificates.
515
516 **tls_client_certs** (``optional``)
517
518 Disable (``off``), allow (``optional``) or require (``require``)
519 clients authenticate with an SSL certificate.
520
521 **tls_server_ca_file** (<none>)
522
523 **tls_server_ca_dir** (<none>)
524
525 Certificate Authority file or directory used to verify SSL
526 certificates offered by other servers.
527
528 **tls_compression** (``0``)
529
530 Enable TLS compression. Disabled by default.
531
532 **tls_eccurve** (``prime256v1``)
533
534 Select the elliptic curve used for ECDHE. See
535 :command:`openssl ecparams -list_curves` for supported values on
536 your platform.
537
538 **tls_prefer_server_ciphers** (``0``)
539
540 Prefer the cipher order configured on the server-side.
541
542 **tls_versions** (``ssl2 ssl3 tls1_0 tls1_1 tls1_2``)
543
544 Disable SSL/TLS protocols not in this list.
545
546 .. _relnotes-2.5.0-development-autoconf:
547
548 Development: Switch to ``autoconf`` and ``libtool``
549 ===================================================
550
551 With the release of Cyrus IMAP 2.5.0, the Cyrus IMAP project has
552 switched to using autoconf and libtool.
553
554 .. _relnotes-2.5.0-development-phabricator:
555
556 Development: Switch to Phabricator
557 ==================================
558
559 An instance of Phabricator is going to be replacing our old Bugzilla.
560
561 We believe this better facilitates our processes, and will make it
562 easier to contribute code and collaborate.
563
564 Please see https://git.cyrus.foundation/.
565
566 .. NOTE::
567
568 The Phabricator installation suffered a disk failure. We now use
569 GitHub for code and bug management. https://github.com/cyrusimap/cyrus-imapd
570
571 .. _relnotes-2.5.0-development-documentation:
572
573 Development: Sphinx for Documentation
574 =====================================
575
576 While a work in progress still, you're looking at the new and improved
577 documentation effort for the Cyrus project as a whole.
578
579 This documentation is written in reStructuredText, and rendered by
580 Sphinx.
581
582 The GIT repository for the documentation is at
583
584
585 .. rubric:: Footnotes
586
587 .. [#]
588
589 http://opera.brong.fastmail.fm.user.fm/talks/twoskip/twoskip-yapc12.pdf