Debian Codegraph @

doc/html/_sources/imap/reference/faqs/o-deleted-expired-expunged-purged.rst.txt

raw
1 .. _imap-admin-deleted-expired-expunged-purged:
2
3 ======================================================
4 When is What ... Deleted, Expired, Expunged or Purged?
5 ======================================================
6
7 The terminology sometimes plays administrators for fools -- which they
8 are obviously not -- but an article clarifying what it is that is meant
9 by either term of Deleted, Expired, Expunged or Purged goes a long way.
10
11 **Deleted**
12
13 A message has been flagged as ``\Deleted``.
14
15 In the context of folders, *Deleted* only really applies to a folder
16 having been removed (from the user's (IMAP client) view), as opposed
17 to having been renamed to a hierarchy in a trash folder.
18
19 **Expired**
20
21 The index records of a message have been expired, and usually this
22 means the message file has been purged as well. However the message
23 file could be purged prior to index records being expired.
24
25 **Expunged**
26
27 The message (which has been flagged as ``\Deleted``) is also
28 expunged, meaning that the user can in no way retrieve the message
29 autonomously.
30
31 **Purged**
32
33 The message's index record may still exist (until they are expired),
34 but the message file is removed from the filesystem, or in the
35 context of folders, the mail folder is removed from the filesystem.
36
37 Users and IMAP Clients Deleting Messages
38 ========================================
39
40 When a **message** is deleted by a user, this means that the user's IMAP
41 client has in fact *flagged* the message with ``\Deleted``, or
42 alternatively, the IMAP client has moved the message to a trash folder
43 (and has at least flagged the original copy as ``\Deleted``).
44
45 What is the exact behavior depends on the IMAP client software, and if
46 so allowed, the user's preferences specified within the IMAP client
47 software, and the Cyrus IMAP server configuration.
48
49 Flagged As ``\Deleted``
50 -----------------------
51
52 When a message is merely flagged with ``\Deleted``, the message itself
53 as such remains available to the IMAP client, but the IMAP client used
54 may not make it possible for the user to view a list of messages flagged
55 with ``\Deleted``. As such, the user may interpret the message as
56 removed and unavailable -- if the removal was accidental, a support call
57 may be on its way.
58
59 The message in fact still resides in the Cyrus IMAP mail spool, still
60 resides in the same IMAP folder, and still resides on the filesystem.
61
62 Only when the user (or the user's IMAP client as is often the case)
63 issues an ``EXPUNGE`` against the folder, or a ``UID EXPUNGE`` against
64 the message [#]_, will the message be actually removed -- at least from
65 the user's perspective. It then becomes irretrievable even if the IMAP
66 client allows the listing of messages flagged with ``\Deleted``.
67
68 Issuing an ``EXPUNGE`` may come in the form of a button to "compact" the
69 folder, or an IMAP client routine that is executed periodically or at
70 the end of a session (e.g. as the client application is closed), such as
71 an "Empty Trash folder" kind of option.
72
73 It is here that the Cyrus IMAP server settings come in to play, most
74 prominently the ``expunge_mode`` setting, which has three possible
75 values:
76
77 **delayed** (the default *since version 2.5.0*)
78
79 The message files as well as the index records are retained for an
80 undetermined period of time -- possibly indefinitely.
81
82 A separate job (using :ref:`imap-reference-manpages-systemcommands-cyr_expire`) is
83 responsible for actually removing index records and message files.
84
85 **default**
86
87 The message files are removed at the first opportunity, while the
88 index records remain available to facilitate ``QRESYNC``.
89
90 In this context also, when we say "message files are removed", we
91 mean "purged from the filesystem".
92
93 **immediate**
94
95 The message files as well as the index records are removed at the
96 earliest opportunity.
97
98 In this context, when we say "message files are removed", we mean
99 "purged from the filesystem".
100
101 Exceptional circumstances aside, when ``immediate`` or ``default`` is
102 the configured ``expunge_mode``, message files are often purged from the
103 filesystem too quickly for anyone to recover.
104
105 .. NOTE::
106
107 One such exceptional circumstance is a mailbox with multiple
108 sessions keeping the mailbox open. Cyrus IMAP ensures no mailbox
109 records disappear from underneath an existing open mailbox session.
110
111 Moved to Trash Folder
112 ---------------------
113
114 Should the IMAP client normally, or allow the user to specify through
115 preferences, that messages being deleted should be moved in to a trash
116 folder, then the user will usually be able to recover from accidental
117 deletion autonomously, for as long as a copy of the deleted message(s)
118 resides in such trash folder.
119
120 However, the trash folder would typically continue to grow and grow, and
121 usually counts towards the user's resource usage (a.k.a.
122 :ref:`imap-features-quota`); many IMAP clients therefore allow the user
123 to specify a preference to empty the trash folder at the end of a
124 session, or otherwise periodically.
125
126 If the IMAP client does not support :rfc:`6851` (for ``UID MOVE``), the
127 client may choose to ``COPY`` the message then flag the original with
128 ``\Deleted``, then ``EXPUNGE`` the folder or ``UID EXPUNGE``
129 (:rfc:`4315`) the message.
130
131 This does not fare well in situations where the user is over quota,
132 though, and (other) messages will need to be flagged as ``\Deleted`` and
133 expunged, and/or folders within the quota root hierarchy will need to be
134 deleted.
135
136 Expunged Messages
137 -----------------
138
139 Messages in expunged folders, or messages that have been expunged
140 individually, can not autonomously be restored by a user, and are gone
141 permanently unless ``expunge_mode: delayed`` is used.
142
143 Recovering expunged messages requires administrator assistance, who can
144 use command-line tools such as :ref:`imap-reference-manpages-systemcommands-unexpunge` to
145 list and restore messages expunged. See the documentation on
146 :ref:`imap-reference-manpages-systemcommands-unexpunge` for a walk-through on how that
147 works.
148
149 With the use of ``expunge_mode: delayed``, a regular ``EVENT`` (see
150 :cyrusman:`cyrus.conf(5)`) is responsible for triggering
151 :ref:`imap-reference-manpages-systemcommands-cyr_expire`. This utility takes a parameter
152 ``-X <days>`` to delete from the filesystem any messages that had been
153 expunged (by the user or the IMAP client) more than ``<days>`` days ago.
154
155 In other words, using ``expunge_mode: delayed`` and
156 :ref:`imap-reference-manpages-systemcommands-cyr_expire` allows an administrator to recover
157 messages that have been deleted by the user less than ``<days>`` ago.
158
159 .. NOTE::
160
161 This also offers a backup program the chance to obtain all message
162 files. For a monthly full cycle, for example, one could choose to
163 purge message files from the filesystem only after 69 days: two
164 months plus the maximum margin for a first Saturday to Sunday night
165 of the week.
166
167 Deleting Folders
168 ================
169
170 When folders are deleted the IMAP client tends to either delete the
171 folder, or rename the folder to a hierarchy in a trash folder.
172
173 .. NOTE::
174
175 Note that deleting a folder ``A/B`` in a hierarchy ``A/B/C`` also
176 deletes the folder ``A/B/C``.
177
178 If the folder is not renamed to a hierarchy in a trash folder but
179 instead removed directly, then the user has no way to autonomously
180 recover from such event.
181
182 This is where the Cyrus IMAP server settings come in to play, most
183 prominently ``delete_mode``.
184
185 The setting holds two values:
186
187 **delayed** (the default *since version 2.5.0*)
188
189 Mailboxes that are being deleted are not deleted from the
190 filesystem, but instead renamed to a special mailbox hierarchy under
191 the deleted prefix, to be removed later by
192 :ref:`imap-reference-manpages-systemcommands-cyr_expire`.
193
194 **immediate**
195
196 In immediate mode, the mailbox is removed from the filesystem
197 immediately. Note that for large folders, this can be a
198 comparatively expensive operation.
199
200 Where are the Messages?
201 =======================
202
203 This part of the documentation assumes that you have run with the
204 default settings of ``delete_mode: delayed`` and
205 ``expunge_mode: delayed``.
206
207 The result of a message having deleted in either of the former ways, or
208 an entire folder having been deleted, is one of the following stages;
209
210 * The message has only been flagged as ``\Deleted`` and the message
211 nor the folder has been expunged.
212
213 Result: The message resides in the original folder.
214
215 * The message has only been flagged as ``\Deleted`` and either the
216 message individually or the entire folder as a whole has been
217 expunged.
218
219 Result: The message resides in the original folder and can be
220 retrieved using :ref:`imap-reference-manpages-systemcommands-unexpunge`.
221
222 * The message has been copied to the trash folder and at least flagged
223 ``\Deleted`` in the source folder, and the original message or the
224 entire folder in which the original message resided may or may not
225 have been expunged.
226
227 Similarly, the trash folder may or may not have been "emptied".
228
229 Result: A copy of the message still exists in the original folder
230 and can be retrieved using :ref:`imap-reference-manpages-systemcommands-unexpunge`.
231
232 * The message was moved in to the trash folder, implying the original
233 message is expunged from the source folder -- through ``UID MOVE``
234 or :rfc:`6851` support *since version 2.5.0*.
235
236 The trash folder may or may not have been "emptied".
237
238 Result: A copy of the message still exists in the original folder
239 and can be retrieved using :ref:`imap-reference-manpages-systemcommands-unexpunge`.
240
241 * The folder was moved to a hierarchy in the trash folder, and the
242 trash folder has not yet been "emptied".
243
244 Result: A copy of the message exists in the trash folder's
245 hierarchy.
246
247 * The folder was moved to a hierarchy in the trash folder, and the
248 trash folder as subsequently been emptied.
249
250 Result: The folder hierarchy has been renamed to the deleted
251 namespace.
252
253 .. rubric:: Footnotes
254
255 .. [#]
256
257 Only if the IMAP client supports :rfc:`4315`, the IMAP UIDPLUS
258 Extension.