Debian Codegraph @

doc/html/_sources/imap/developer/API/cyrusdb2.rst.txt

raw
1 .. _imap-developer-api-cyrusdb2:
2
3 .. Note: This document was converted from the original by Nic Bernstein
4 (Onlight). Any formatting mistakes are my fault and not the
5 original author's. Converted via the pandoc tool from HTML.
6
7 cyrusdb API
8 ===========
9
10 Intro
11 -----
12
13 The ``cyrusdb`` API is a common interface to a key-value store, used
14 throughout the Cyrus code. It allows a choice of different backends for
15 different access patterns, while ensuring a consistent interface.
16
17 This document will describe the interface, and how to use the cyrusdb
18 interface from within parts of Cyrus code, as well as how to implement
19 your own backend
20
21 If you pass incorrect values to these APIs, you will get an assertion
22 failure in most cases. That's generally considered safer than silently
23 breaking things. Exceptions are noted below.
24
25 Code Layout
26 -----------
27
28 The implementation of each interface is in ``lib/cyrusdb_NAME.c``, for
29 example lib/cyrusdb\_flat.c. General functions are in ``lib/cyrusdb.c``
30 and the interface in ``lib/cyrusdb.h``.
31
32 Configuration
33 -------------
34
35 The name of the backend for each of the main internal databases can be
36 configured in imapd.conf, for example: ``annotation_db: skiplist``. This
37 is then read in imap/global.h and imap/global.c during startup, so that
38 the global variable ``config_annotation_db`` gets set to the configured
39 backend name:
40
41 ::
42
43 config_annotation_db = config_getstring(IMAPOPT_ANNOTATION_DB);
44
45 (Beware the misleading naming here: ``config_annotation_db`` is a string
46 describing the *backend* used by the database, not, say, its location on
47 disk.)
48
49 Internally, the main module for each database sets up struct of pointers
50 to the cyrusdb functions it implements, which is registered in
51 ``lib/cyrusdb.c``
52
53 ``lib/cyrusdb.c`` provides backend-agnostic wrapper functions for
54 interacting with cyrusdb databases.
55
56 A full example
57 --------------
58
59 ::
60
61 struct db *db = NULL;
62 struct txn *tid = NULL;
63 const char *filename = NULL;
64 int flags = CYRUSDB_CREATE;
65
66 cyrus_init(alt_config, "toolname", 0);
67
68 filename = config_getstring(IMAPOPT_ANNOTATION_DB_PATH);
69
70 r = cyrusdb_open(config_annotation_db, filename, flags, &db);
71
72 r = cyrusdb_fetch(db, key, keylen, &data, &datalen, &tid);
73
74 r = cyrusdb_commit(db, tid);
75
76 r = cyrusdb_close(db);
77
78 cyrus_done();
79
80 Note that you always open a database first, and close it at the end. You
81 must always call cyrus\_init() and cyrus\_done() to properly initialize
82 and clean up the ``cyrusdb`` environments.
83
84 This example also uses a transaction, meaning that the database is
85 locked in exclusive mode between the 'fetch' (the first use of the
86 transaction) and the commit.
87
88 About Transactions
89 ------------------
90
91 The cyrusdb interface works in two modes - transactional and
92 non-transactional. The value of the 'tid' parameter decides which mode
93 is used. There are three possible values:
94
95 - NULL - non-transactional. Will create a temporary lock for the
96 duration of the current action - either a write lock for "store" or a
97 read lock for "fetch". If you call "foreach", the lock will be
98 dropped between each record fetched
99 - Pointer to NULL - transactional, transaction not yet started. Will
100 always take a write lock on the database, and update the pointer to
101 point to the new transaction.
102 - Pointer to a valid transaction. Will keep using this transaction
103
104 If you are currently in a transaction, you MUST pass the same
105 transaction to every database call. It is not possible to mix or nest
106 transactions. There is one exception in the skiplist backend: *If you
107 pass NULL to a fetch or foreach while the database is in a transaction,
108 it will silently do the read in the current transaction rather than
109 returning an error*
110
111 API Reference
112 -------------
113
114 All functions follow the normal C API of returning '0' on success, and
115 an error code on failure
116
117 cyrusdb\_init(void)
118 ~~~~~~~~~~~~~~~~~~~
119
120 Is called once per process. Don't call this yourself, use
121 ``cyrus_init()``. No other calls will be made until this is called.
122
123 cyrusdb\_done(void)
124 ~~~~~~~~~~~~~~~~~~~
125
126 The opposite of ``cyrusdb_init()`` - called once per process to do any
127 cleaning up after all database usage is finished. Don't call this
128 yourself, use ``cyrus_done()``.
129
130 cyrusdb\_sync(const char \*backend)
131 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
132
133 Perform a checkpoint of the database environment. Used by berkeley
134 backend. Is called by ``ctl_cyrusdb -c`` on a regular basis
135
136 cyrusdb\_open(const char \*backend, const char \*fname, int flags, struct db \*\*retdb)
137 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
138
139 Opens the database with the specified 'file name' (or other descriptor,
140 for example the sql backend is not a filename), and if successful
141 returns an opaque database structure
142
143 Flags:
144
145 - CYRUSDB\_CREATE - create the database if it doesn't exist
146
147 Errors:
148
149 - CYRUSDB\_IOERROR - if there is any error reading the file, or any
150 corruption detected while loading the file
151
152 cyrusdb\_close(struct db \*db)
153 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
154
155 Close the named database. Will release any locks if they are still held,
156 but it's bad practice to close without committing or aborting, so the
157 backend should log an error
158
159 Errors:
160
161 - CYRUSDB\_IOERROR - if there are any errors during close
162
163 cyrusdb\_fetch(struct db \*db, const char \*key, size\_t keylen, const char \*\*data, size\_t \*datalen, struct txn \*\*tidptr)
164 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165
166 cyrusdb\_fetchlock(struct db \*db, const char \*key, size\_t keylen, const char \*\*data, size\_t \*datalen, struct txn \*\*tidptr)
167 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
168
169 Fetch the value for the exact key given by key and keylen. If data is
170 not NULL, set datalen and return a valid pointer to the start of the
171 value.
172
173 Fetchlock is identical to fetch, but gives a hint to the database that
174 the record is likely to be modified soon.
175
176 NOTE: it is possible to store a key with a zero length data record, in
177 which case \*datalen will be set to zero, and \*data will be set to a
178 non-NULL value
179
180 It is an error to call fetch with a NULL key or a zero keylen
181
182 It is an error to call fetch with a NULL datalen and a non-NULL data,
183 however it is acceptable to call with a NULL data and a non-NULL datalen
184 if you are only interested in the length
185
186 Errors:
187
188 - CYRUSDB\_IOERROR - if any error occurs reading from the database
189 - CYRUSDB\_LOCKED - if tidptr is incorrect
190 - CYRUSDB\_NOTFOUND - if there is no record that matches the key
191
192 cyrusdb\_foreach(struct db \*db, const char \*prefix, size\_t prefixlen, foreach\_p \*goodp, foreach\_p \*procp, void \*rock, struct txn \*\*tidptr)
193 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
194
195 cyrusdb\_forone(struct db \*db, const char \*key, size\_t keylen, foreach\_p \*goodp, foreach\_p \*procp, void \*rock, struct txn \*\*tidptr)
196 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
197
198 ``cyrusdb_foreach()`` iterates over all records matching the given
199 prefix, in database order (which may be MBOXLIST sort, depending on the
200 parameters given to open
201
202 It is legal to give a NULL pointer as prefix if prefixlen is zero, in
203 which case it will return all records in the database. It is an error to
204 give a non-zero prefixlen with a NULL prefix.
205
206 ``cyrusdb_forone()`` "iterates" over the single record matched by the
207 given key. If you've already built callbacks for processing each record
208 from a foreach, this lets you use the same interface to process a single
209 record.
210
211 ``goodp`` - this function is only used for deciding if the record needs
212 to be further processed. It can be used for basic filtering, and returns
213 true (non-zero) to process, or zero to skip and move straight to the
214 next record. Because goodp can't make any database changes, it doesn't
215 break the lock, so it's faster to use goodp to filter records if you
216 don't need to process all of them. NULL is a legal value for goodp, and
217 means that all records will be processed.
218
219 ``procp`` - procp is the main callback function. If you use foreach in
220 non-transactional mode, the database is unlocked before calling procp,
221 and locked again afterwards. You are allowed to add, delete or modify
222 values in the same database from within procp. If procp returns
223 non-zero, the foreach loop breaks at this point, and the return value of
224 the foreach becomes the return value of procp. If procp returns zero,
225 the foreach loop will continue at the NEXT record by sort order,
226 regardless of whether the current record has changed or been removed.
227 procp MUST NOT be NULL.
228
229 Errors:
230
231 - procp\_result - whatever your callback returns
232 - CYRUSDB\_IOERROR - if any error occurs while reading
233 - CYRUSDB\_LOCKED - if tidptr is incorrect
234
235 cyrusdb\_create(struct db \*db, const char \*key, size\_t keylen, const char \*data, size\_t datalen, struct txn \*\*tidptr)
236 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
237
238 cyrusdb\_store(struct db \*db, const char \*key, size\_t keylen, const char \*data, size\_t datalen, struct txn \*\*tidptr)
239 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
240
241 Create a new record or replace an existing one. The only difference
242 between these two is that ``cyrusdb_create`` will return an error if the
243 record already exists, while ``cyrusdb_store`` will replace it
244
245 If tidptr is NULL, create/store will take a write lock for the duration
246 of the action.
247
248 Any failure during create/store will abort the current transaction as
249 well as returning an error
250
251 It is legal to pass NULL for the data field ONLY if datalen is zero. It
252 is not legal to pass NULL for key or zero for keylen
253
254 Errors:
255
256 - CYRUSDB\_IOERROR - any error to write to the database
257 - CYRUSDB\_LOCKED - if tidptr is incorrect
258 - CYRUSDB\_EXISTS - if ``cyrusdb_create`` is called on an existing key
259 - CYRUSDB\_AGAIN - if a deadlock is created. The current transaction
260 has been aborted, but a retry may succeed
261
262 cyrusdb\_delete(struct db \*db, const char \*key, size\_t keylen, struct txn \*\*tidptr, int force)
263 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
264
265 Delete the given record from the database. If force is true, then
266 succeed even if the record doesn't currently exist.
267
268 It is not legal to pass NULL for key or zero for keylen
269
270 Errors:
271
272 - CYRUSDB\_IOERROR - any error to write to the database
273 - CYRUSDB\_LOCKED - if tidptr is incorrect
274 - CYRUSDB\_NOTFOUND - if force is not set and the key doesn't exist
275 - CYRUSDB\_AGAIN - if a deadlock is created. The current transaction
276 has been aborted, but a retry may succeed
277
278 cyrusdb\_commit(struct db \*db, struct txn \*tid)
279 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
280
281 Commit the current transaction. tid will not be valid after this call,
282 regardless of success
283
284 If the commit fails, it will attempt to abort the transaction
285
286 Errors:
287
288 - CYRUSDB\_IOERROR - any error to write to the database
289 - CYRUSDB\_LOCKED - if tidptr is incorrect
290 - CYRUSDB\_AGAIN - if a deadlock is created. The current transaction
291 has been aborted, but a retry may succeed
292
293 cyrusdb\_abort(struct db \*db, struct txn \*tid)
294 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
295
296 Abort the current transaction. tid will not be valid after this call,
297 regardless of success
298
299 Attempt to roll back all changes made in the current transaction.
300
301 Errors:
302
303 - CYRUSDB\_IOERROR - any error to write to the database
304 - CYRUSDB\_LOCKED - if tidptr is incorrect
305
306 cyrusdb\_dump(struct db \*db, int detail)
307 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
308
309 Optional function to dump the internal structure of the database to
310 stdout for debugging purposes. Don't use.
311
312 cyrusdb\_consistent(struct db \*db)
313 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314
315 Check if the DB is internally consistent. Looks pretty bogus, and isn't
316 used anywhere. Don't use.