Debian Codegraph @

doc/html/_sources/imap/reference/admin/tweaking.rst.txt

raw
1 ===================
2 Tweaking Cyrus IMAP
3 ===================
4
5 .. _admin-tweaking-cyrus-imapd-storage-tiering:
6
7 Storage Tiering
8 ===============
9
10 Various opportunities exist to tier storage. With storage
11 tiering, we mean to distribute payload over different levels of storage,
12 where levels mean to refer to storage cost, performance and capacity, in
13 a way that makes optimal use of the storage solution(s) -- with an eye
14 on excellent performance yet remain cost-effective.
15
16 Imagine that you have the following levels of storage:
17
18 * A **low-cost** solution with large capacity,
19
20 such as many consumer-grade 4 TB HDDs,
21
22 * A **high-cost** solution with little capacity,
23
24 such as a few business-grade 512 GB SSDs.
25
26 The following options are available to split up the payload for a Cyrus
27 IMAP server:
28
29 #. Partitioning the mail spool;
30
31 where you divide a single backend server's mail spool in to multiple
32 :term:`partitions`, presumably at least two, and make sure that
33 mail folders end up on the correct partition.
34
35 You would have one partition mounted off of the fast storage contain
36 the *day-to-day* payload, such as user's INBOX folders, while the
37 partition mounted off of the slower, cheap storage holds less
38 frequently used data, such as archive folders.
39
40 .. seealso::
41
42 * :ref:`admin-tweaking-cyrus-imap-storage-tiering-partitioning`
43
44 #. Distributing the mail spool;
45
46 where you would have one server hold *day-to-day* payload, and other
47 servers hold less frequently used data, such as archive folders.
48
49 Note that distribution of the mail spool requires a Cyrus IMAP
50 Murder topology, so that access to the mailboxes remains
51 transparent.
52
53 Also note that such archive folders may be hosted using not only
54 a low-cost storage tier, but perhaps also using a less resourceful
55 compute node.
56
57 .. seealso::
58
59 * :ref:`admin-tweaking-cyrus-imap-storage-tiering-distributing`
60
61 #. Caching I/O using `dm-cache`_;
62
63 This mechanism is a feature of Linux distributions that ship a
64 kernel version of 3.9 or later, called `dm-cache`_, and allows
65 multiple disk volumes to be used as tiered caching storage.
66
67 Using this technology, frequently accessed data can automatically be
68 promoted to the fast storage, while less frequently accessed data
69 can automatically be demoted to the cheaper storage.
70
71 #. Using metadata partitions;
72
73 While the access patterns for data and metadata are different, so
74 could their storage locations be.
75
76 In this scenario, you would distinguish between a mail folder's
77 message content and metadata -- the metadata consists of mail
78 headers, indexes created for the purpose of searching, and such,
79 while mail messages are stored in separate files.
80
81 When a mail folder is opened, and the messages in the folder are
82 listed, no mail message content is touched -- the results come
83 entirely from metadata.
84
85 It is only when the individual mail message is being fetched, that
86 the message file is opened.
87
88 .. _admin-tweaking-cyrus-imap-storage-tiering-partitioning:
89
90 Partitioning the Mail Spool
91 ---------------------------
92
93 .. _admin-tweaking-cyrus-imap-storage-tiering-distributing:
94
95 Distributing the Mail Spool
96 ---------------------------
97
98 .. _admin-tweaking-cyrus-imap-storage-tiering-metadata:
99
100 Using Metadata Partitions
101 -------------------------
102
103 Synchronous File Operations
104 ===========================
105
106 A default Kolab Groupware installation comes with a set of settings
107 suitable for the vast majority of our users -- mostly home users and
108 other small(er) deployments.
109
110 It is typical for these installations to **not** have battery-backed I/O
111 controllers, and/or some other form of enterprise-level storage.
112
113 To prevent data from being corrupted during a power outage, the default
114 for our Cyrus IMAP packages is to ensure the mail spool
115 (:file:`/var/spool/cyrus/`) and configuration directory
116 (:file:`/var/lib/imap/`) and all files contained therein have the
117 *synchronous* filesystem flag set.
118
119 To gain performance, execute the following:
120
121 #. Remove the synchronous flag from the directories and files:
122
123 .. parsed-literal::
124
125 :command:`chattr -RV -S /var/lib/imap/ /var/spool/cyrus/`
126
127 #. In :file:`/etc/sysconfig/cyrus-imapd` (or
128 :file:`/etc/default/cyrus-imapd`), change the following:
129
130 .. parsed-literal::
131
132 CHATTRSYNC=1
133
134 to:
135
136 .. parsed-literal::
137
138 CHATTRSYNC=0
139
140 .. _dm-cache: http://en.wikipedia.org/wiki/Dm-cache
141
142
143 Mailbox locking
144 ---------------
145 Cyrus IMAP uses fcntl(2) based file locking for mailboxes, for example during
146 SELECT commands. To mitigate race conditions it locks mailbox names even
147 for non-existing mailboxes. For example, if user foo issued the following
148 command
149
150 .. parsed-literal::
151
152 SELECT INBOX.x
153
154 for non-existing mailbox x, it creates a lock file
155
156 .. parsed-literal::
157
158 $CYRUS_CONFDIR/lock/user/foo/x.lock
159
160 which might be left on the filesystem after completion of the command.
161
162 This has in practice not shown to be an issue. If this is a concern however,
163 keeping the lock file directory in a tmpfs allows for both fast locking and
164 to purge stale locks during controlled Cyrus downtimes.