| 1 |
.. _imap-developer-api-cyrusdb: |
| 2 |
|
| 3 |
.. Note: This document was converted from the original by Nic Bernstein |
| 4 |
(Onlight). Any formatting mistakes are my fault and not the |
| 5 |
original author's. Converted via the pandoc tool from Markdown. |
| 6 |
|
| 7 |
CyrusDB API |
| 8 |
=========== |
| 9 |
|
| 10 |
The interface |
| 11 |
------------- |
| 12 |
|
| 13 |
The bulk, especially the beginning, of this document is from an |
| 14 |
`email <https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2018-February/004217.html>`__ |
| 15 |
Bron sent to the |
| 16 |
`cyrus-devel <https://lists.andrew.cmu.edu/pipermail/cyrus-devel/>`__ |
| 17 |
mailing list on 27 February 2018 05:50:39GMT. |
| 18 |
|
| 19 |
The cyrusdb interface is a common API for accessing key-value |
| 20 |
datastores from within cyrus code. It's part of the libcyrus shared |
| 21 |
object (but not libcyrus_min), used by all cyrus binaries. The use of |
| 22 |
the cyrusdb API abstracts the details of the underlying data store, |
| 23 |
allowing sites to use different database types depending on their |
| 24 |
needs, and allowing Cyrus developers to use a common API for data |
| 25 |
storage needs. |
| 26 |
|
| 27 |
The entire cyrusdb source lives in the lib/ directory of the cyrus-imapd repository, in the following files: |
| 28 |
|
| 29 |
:: |
| 30 |
|
| 31 |
lib/cyrusdb.h |
| 32 |
lib/cyrusdb.c |
| 33 |
lib/cyrusdb_flat.c |
| 34 |
lib/cyrusdb_quotalegacy.c |
| 35 |
lib/cyrusdb_skiplist.c |
| 36 |
lib/cyrusdb_sql.c |
| 37 |
lib/cyrusdb_twoskip.c |
| 38 |
|
| 39 |
cyrusdb.h |
| 40 |
+++++++++ |
| 41 |
|
| 42 |
* interface definitions (all access to cyrusdb databases happens |
| 43 |
through the functions defined here) |
| 44 |
* the ``struct cyrusdb_backend`` data structure which defines the |
| 45 |
interface implemented by each backend. |
| 46 |
* constants for flags to the cyrusdb_open call, and return codes. |
| 47 |
Cyrusdb functions all return their own CYRUSDB_* error codes, which |
| 48 |
are not compatible with the r = IMAP_* return codes used throughout |
| 49 |
much of the rest of the codebase. |
| 50 |
|
| 51 |
cyrusdb.c |
| 52 |
+++++++++ |
| 53 |
|
| 54 |
* implementations of the wrapper functions around the backends, |
| 55 |
including default implementations of some functions which are |
| 56 |
common to many backends but overridden by some. |
| 57 |
* a wrapper to initialise and cleanup the state of each backend |
| 58 |
(if needed) during Cyrus set up / tear down. |
| 59 |
|
| 60 |
cyrusdb_*.c |
| 61 |
+++++++++++ |
| 62 |
|
| 63 |
* the actual implementations of each backend! We'll look at some |
| 64 |
in a sec. |
| 65 |
|
| 66 |
Configuration |
| 67 |
------------- |
| 68 |
|
| 69 |
The name of the backend for each of the main internal databases can be |
| 70 |
configured in imapd.conf, for example: ``annotation_db: skiplist``. This |
| 71 |
is then read in imap/global.h and imap/global.c during startup, so that |
| 72 |
the global variable ``config_annotation_db`` gets set to the configured |
| 73 |
backend name: |
| 74 |
|
| 75 |
:: |
| 76 |
|
| 77 |
config_annotation_db = config_getstring(IMAPOPT_ANNOTATION_DB); |
| 78 |
|
| 79 |
(Beware the misleading naming here: ``config_annotation_db`` is a string |
| 80 |
describing the *backend* used by the database, not, say, its location on |
| 81 |
disk.) |
| 82 |
|
| 83 |
Internally, the main module for each database sets up struct of pointers |
| 84 |
to the cyrusdb functions it implements, which is registered in |
| 85 |
``lib/cyrusdb.c`` |
| 86 |
|
| 87 |
``lib/cyrusdb.c`` provides backend-agnostic wrapper functions for |
| 88 |
interacting with cyrusdb databases. |
| 89 |
|
| 90 |
A full example |
| 91 |
-------------- |
| 92 |
|
| 93 |
:: |
| 94 |
|
| 95 |
struct db *db = NULL; |
| 96 |
struct txn *tid = NULL; |
| 97 |
const char *filename = NULL; |
| 98 |
int flags = CYRUSDB_CREATE; |
| 99 |
|
| 100 |
cyrus_init(alt_config, "toolname", 0); |
| 101 |
|
| 102 |
filename = config_getstring(IMAPOPT_ANNOTATION_DB_PATH); |
| 103 |
|
| 104 |
r = cyrusdb_open(config_annotation_db, filename, flags, &db); |
| 105 |
|
| 106 |
r = cyrusdb_fetch(db, key, keylen, &data, &datalen, &tid); |
| 107 |
|
| 108 |
r = cyrusdb_commit(db, tid); |
| 109 |
|
| 110 |
r = cyrusdb_close(db); |
| 111 |
|
| 112 |
cyrus_done(); |
| 113 |
|
| 114 |
Note that you always open a database first, and close it at the end. You |
| 115 |
must always call cyrus\_init() and cyrus\_done() to properly initialize |
| 116 |
and clean up the ``cyrusdb`` environments. |
| 117 |
|
| 118 |
This example also uses a transaction, meaning that the database is |
| 119 |
locked in exclusive mode between the 'fetch' (the first use of the |
| 120 |
transaction) and the commit. |
| 121 |
|
| 122 |
Tools |
| 123 |
----- |
| 124 |
|
| 125 |
There are also some tools to work with and support cyrus databases: |
| 126 |
|
| 127 |
imap/ctl_cyrusdb |
| 128 |
++++++++++++++++ |
| 129 |
|
| 130 |
:cyrusman:`ctl_cyrusdb(8)` |
| 131 |
|
| 132 |
* performs maintenance on the cyrusdb subsystem. This is called in two places: |
| 133 |
|
| 134 |
* START: "ctl_cyrusdb -r" (recovery). |
| 135 |
This is the ONLY PLACE that code is guaranteed to be run at |
| 136 |
startup on every cyrus installation, so you'll find quite a lot |
| 137 |
of detritus has built up in this codepath over the years. |
| 138 |
* EVENTS: "ctl_cyrusdb -c" (checkpoint). |
| 139 |
This is run regularly (period=180 at Fastmail, examples in the |
| 140 |
codebase have period=5 or period=30). Both this codepath and |
| 141 |
cyr_expire tend to run periodically on cyrus systems, and |
| 142 |
cleanup code is spread between those two locations. |
| 143 |
|
| 144 |
imap/cvt_cyrusdb |
| 145 |
++++++++++++++++ |
| 146 |
|
| 147 |
:cyrusman:`cvt_cyrusdb(8)` |
| 148 |
|
| 149 |
* used for converting a database between versions. |
| 150 |
This is often used to prepare for upgrade, particularly in the past |
| 151 |
when Cyrus supported berkeley DB which didn't upgrade cleanly |
| 152 |
across OS versions, it was common to use cvt_cyrusdb to turn |
| 153 |
databases into a very portable format (flat or skiplist) before |
| 154 |
upgrading, upgrade the OS, convert back to the fast format |
| 155 |
(berkeley) and then restart. |
| 156 |
|
| 157 |
imap/cyr_dbtool |
| 158 |
+++++++++++++++ |
| 159 |
|
| 160 |
:cyrusman:`cyr_dbtool(8)` |
| 161 |
|
| 162 |
* once known as brontool, this is the first piece of Cyrus code |
| 163 |
I ever wrote! It's a fairly dumb wrapper around the CyrusDB |
| 164 |
interface, and able to be used to read, write, or iterate any parts |
| 165 |
of a database. Its interactive mode is not special-character clean, |
| 166 |
but it can also be used in batch mode, which uses IMAP atom-string |
| 167 |
literal8 for input/output, and hence can roundtrip data reliably. |
| 168 |
|
| 169 |
There are also tools like: :cyrusman:`ctl_conversationsdb(8)`, |
| 170 |
:cyrusman:`dav_reconstruct(1)` and :cyrusman:`ctl_mboxlist(8)` which |
| 171 |
can be used to manage individual databases through a more specific |
| 172 |
interface which understands the context as well as just the raw |
| 173 |
key/value. |
| 174 |
|
| 175 |
How to use CyrusDB |
| 176 |
------------------ |
| 177 |
|
| 178 |
Assuming that ``cyrus_init()`` has been called, which calls |
| 179 |
``cyrusdb_init()``, you can assume that databases will work in any |
| 180 |
Cyrus code. |
| 181 |
|
| 182 |
The first step is to open a database. Databases have a filename - this |
| 183 |
might be a literal filename on the backend, a directly containing data, |
| 184 |
or an opaque token used by the backend to locate a datasource. |
| 185 |
|
| 186 |
:: |
| 187 |
|
| 188 |
int flags = 0; |
| 189 |
struct db *mydb = NULL; |
| 190 |
int r = cyrusdb_open("skiplist", "/tmp/database.db", flags, &mydb); |
| 191 |
if (!r) return mydb; // if (r == CYRUSDB_OK) { ... } |
| 192 |
/* XXX: error handling */ |
| 193 |
|
| 194 |
Accepted flags: |
| 195 |
|
| 196 |
CYRUSDB_CREATE - if the named database doesn't exist, create a blank |
| 197 |
database. |
| 198 |
|
| 199 |
CYRUSDB_CONVERT - if set and the database fails to open, attempt a |
| 200 |
magic detection on the file content and try to convert the database to |
| 201 |
the requested backend type before opening it. In-place upgrades! If |
| 202 |
this is NOT set, then instead the magic detection will still be |
| 203 |
performed, but the open database will be returned using the correct |
| 204 |
engine for that database rather than converted. Magic detection only |
| 205 |
currently works for single-file database formats. |
| 206 |
|
| 207 |
CYRUSDB_NOCOMPACT - if the database format supports automatic |
| 208 |
compacting, don't use it. Handy for when you want to read without |
| 209 |
causing any possible issues (e.g. read-only filesystem during recovery) |
| 210 |
or when performance is critical and you don't want to risk waiting |
| 211 |
while a recompact happens. |
| 212 |
|
| 213 |
All the remaining functions take that "struct db" pointer. |
| 214 |
|
| 215 |
There's also a ``cyrusdb_lockopen()`` interface which takes a |
| 216 |
transaction pointer and returns with the transaction already active. |
| 217 |
This isn't actually being used yet, but is intended to allow slightly |
| 218 |
more efficient single-operation database use. Right now, open returns |
| 219 |
an unlocked database, but may need to lock as part of the setup, so |
| 220 |
keeping that lock would avoid one extra unlock/lock cycle. |
| 221 |
|
| 222 |
Reading, Writing, & Transactions |
| 223 |
-------------------------------- |
| 224 |
|
| 225 |
CyrusDB supports both transactional and non-transactional access. |
| 226 |
Transactions are always exclusive. This is arguably a deficiency in the |
| 227 |
interface, particularly since many engines implement a non-exclusive |
| 228 |
(read) lock internally anyway. |
| 229 |
|
| 230 |
Reading |
| 231 |
+++++++ |
| 232 |
|
| 233 |
There are now 4 interfaces to read data. Two of which are original |
| 234 |
cyrusdb and two of which are more recently added. |
| 235 |
|
| 236 |
original: |
| 237 |
|
| 238 |
``cyrusdb_fetch()`` |
| 239 |
fetch a single value by exact key. |
| 240 |
``cyrusdb_foreach()`` |
| 241 |
given a prefix, iterate over all the keys with that prefix |
| 242 |
(including exactly that key) in order. |
| 243 |
|
| 244 |
newer: |
| 245 |
|
| 246 |
``cyrusdb_fetchnext()`` |
| 247 |
given an exact key, fetch the following key and value |
| 248 |
(regardless of whether the key exists), e.g given keys "f" and |
| 249 |
"g", fetchnext "foo" would return "g", as would fetchnext "f". |
| 250 |
This can be used to implement foreach (indeed, the skips do |
| 251 |
exactly that). |
| 252 |
|
| 253 |
``cyrusdb_forone()`` |
| 254 |
given an exact key, act like ``cyrusdb_foreach`` but only for |
| 255 |
that one key. |
| 256 |
|
| 257 |
This is a convenience wrapper around fetch to |
| 258 |
allow doing things like:: |
| 259 |
|
| 260 |
r = cyrusdb_forone(mydb, "folder", 6, p, cb, rock, &tid); |
| 261 |
if (!r) r = cyrusdb_foreach(mydb, "folder.", 7, p, cb, rock, &tid); |
| 262 |
|
| 263 |
Which does precisely "folder" and its children without visiting |
| 264 |
any other keys that have "folder" as a prefix. |
| 265 |
|
| 266 |
Since the cyrusdb interface always takes both a pointer and a length, |
| 267 |
it's also possible to use:: |
| 268 |
|
| 269 |
char *key = "folder."; |
| 270 |
r = cyrusdb_forone(mydb, key, 6, p, cb, rock, &tid); |
| 271 |
if (!r) r = cyrusdb_foreach(mydb, key, 7, p, cb, rock, &tid); |
| 272 |
|
| 273 |
OK, foreach. Foreach is very tricky, because it takes TWO callbacks. |
| 274 |
The callbacks have an identical signature, but different return codes! |
| 275 |
|
| 276 |
:: |
| 277 |
|
| 278 |
typedef int foreach_p(void *rock, |
| 279 |
const char *key, size_t keylen, |
| 280 |
const char *data, size_t datalen); |
| 281 |
|
| 282 |
typedef int foreach_cb(void *rock, |
| 283 |
const char *key, size_t keylen, |
| 284 |
const char *data, size_t datalen); |
| 285 |
|
| 286 |
The difference is this: ``foreach_p`` is called with the database |
| 287 |
locked, always - even if called without a transaction. ``foreach_p`` |
| 288 |
returns 1 or 0. 0 means "skip this record", 1 means "process this |
| 289 |
record". This is useful to pre-filter records when called without a |
| 290 |
transaction, because otherwise you lock and unlock all the time. |
| 291 |
|
| 292 |
NULL for ``foreach_p`` is treated like a test which always returns '1', |
| 293 |
so you can pass NULL if you don't need filtering. |
| 294 |
|
| 295 |
If ``foreach_p`` returns 1, then with an unlocked transaction, the |
| 296 |
database is now unlocked BEFORE calling ``foreach_cb``, the callback. |
| 297 |
``foreach_cb`` returns a CYRUSDB\_ response. If zero, the foreach will |
| 298 |
continue. If non-zero, the foreach will abort and return the non-zero |
| 299 |
response. This is both useful for error cases, and useful for |
| 300 |
short-circuiting, if you only care that a key exists, you can do |
| 301 |
something like:: |
| 302 |
|
| 303 |
static int exists_cb(void rock attribute((unused)), [...]) |
| 304 |
{ |
| 305 |
return CYRUSDB_DONE; /* one is enough */ |
| 306 |
} |
| 307 |
|
| 308 |
and then use ``exists_cb`` as your ``foreach_cb`` and check if the |
| 309 |
return code is CYRUSDB_DONE to know if the foreach found a key. |
| 310 |
|
| 311 |
If foreach is called with a transaction pointer, then it is your |
| 312 |
responsibility as the caller to also pass that pointer (and a pointer |
| 313 |
to the database) in that rock, so that callees can make further |
| 314 |
operations within the same transaction. A foreach with a transaction |
| 315 |
does NOT unlock before calling its callback. |
| 316 |
|
| 317 |
About Transactions |
| 318 |
++++++++++++++++++ |
| 319 |
|
| 320 |
You may have noticed that 'tid' at the end. Every function for acting |
| 321 |
on the database takes as its last argument a ``struct txn **``. |
| 322 |
|
| 323 |
The cyrusdb interface works in two modes - transactional and |
| 324 |
non-transactional. The value of the 'tid' parameter decides which mode |
| 325 |
is used. There are three possible values: |
| 326 |
|
| 327 |
* NULL - non-transactional request. |
| 328 |
Do whatever you need for internal locking, but starts with an |
| 329 |
unlocked database and ends with an unlocked database. |
| 330 |
|
| 331 |
.. Note:: |
| 332 |
|
| 333 |
At least skiplist and twoskip implement a hack where if the |
| 334 |
database IS locked for a non-transactional read request, they |
| 335 |
will act as if you'd passed the current transaction in for the |
| 336 |
NULL case. This is a hack around layering violations and kind |
| 337 |
of sucks. |
| 338 |
|
| 339 |
* &NULL - e.g:: |
| 340 |
|
| 341 |
struct txn *tid = NULL; |
| 342 |
const char *data = NULL; |
| 343 |
size_t datalen = 0; |
| 344 |
int r = cyrusdb_fetch(mydb, key, keylen, &data, &datalen, &tid); |
| 345 |
|
| 346 |
After calling this, tid will have an opaque value allocated by the |
| 347 |
database backend, which must be passed to all further cyrusdb |
| 348 |
operations on that database until either ``cyrusdb_commit()`` or |
| 349 |
``cyrusdb_abort()`` are called. |
| 350 |
|
| 351 |
* &tid - e.g:: |
| 352 |
|
| 353 |
if (r == CYRUSDB_NOTFOUND) { |
| 354 |
r = cyrusdb_store(mydb, key, keylen, "DEFAULT", 7, &tid); // set a default value |
| 355 |
} |
| 356 |
|
| 357 |
Given an existing transaction, perform this call in the context of the transaction. |
| 358 |
|
| 359 |
If you are currently in a transaction, you MUST pass the same |
| 360 |
transaction to every database call. It is not possible to mix or nest |
| 361 |
transactions. |
| 362 |
|
| 363 |
There is one exception in the skiplist backend: |
| 364 |
|
| 365 |
If you pass NULL to a fetch or foreach while the database is in a |
| 366 |
transaction, it will silently do the read in the current |
| 367 |
transaction rather than returning an error. |
| 368 |
|
| 369 |
Writing |
| 370 |
+++++++ |
| 371 |
|
| 372 |
There are three write operations: |
| 373 |
|
| 374 |
:: |
| 375 |
|
| 376 |
cyrusdb_create |
| 377 |
cyrusdb_store |
| 378 |
cyrusdb_delete |
| 379 |
|
| 380 |
``cyrusdb_store`` will either create or overwrite an existing key. |
| 381 |
``cyrusdb_create`` will abort if the key already exists. |
| 382 |
``cyrusdb_delete`` takes a flag 'force' which just makes it return |
| 383 |
CYRUSDB_OK (0) rather than CYRUSDB_NOTFOUND if the key doesn't exist. |
| 384 |
Strangely, 'force' is after &tid, making it the only cyrusdb API that |
| 385 |
does that, but hey - keeps you on your toes. |
| 386 |
|
| 387 |
&tid behaves exactly the same for the write APIs. If not passed, then |
| 388 |
the database engine will behave as it if creates a writable |
| 389 |
transaction, does the operation, then commits all within the |
| 390 |
``cyrusdb_*`` call. |
| 391 |
|
| 392 |
Gotchas! |
| 393 |
-------- |
| 394 |
|
| 395 |
* ``\0`` is permitted in both keys and values, though 'flat' and |
| 396 |
'quotalegacy' have 8-bit cleanliness issues. |
| 397 |
|
| 398 |
* zero-length keys are not supported |
| 399 |
|
| 400 |
* zero-length values are theoretically supported, but a little |
| 401 |
interesting. Certainly, pass "" rather than NULL as the value when |
| 402 |
writing or things will get weird. I'm pretty sure at least the |
| 403 |
\*skip databases assert on these kinds of weirdness. |
| 404 |
|
| 405 |
* unlocked foreach: this is the land of the gotcha! They key and data |
| 406 |
pointers (const char \*) passed to your ``foreach_cb`` are only |
| 407 |
valid UNTIL YOU TOUCH THE DATABASE. A common cause of rare and hard |
| 408 |
to diagnose bugs is writing something to the same database in the |
| 409 |
same process (OR EVEN READING FROM IT AGAIN). I cannot emphasise |
| 410 |
this enough. If you want to zero-copy access that data, you need to |
| 411 |
access it first, before touching that DB again. Otherwise the map |
| 412 |
in which the data was a pointer may have been replaced as the next |
| 413 |
read found a new file and mapped it in! |
| 414 |
|
| 415 |
also: if you're implementing a backend. Unlocked foreach must find |
| 416 |
future records created by the current callback. Consider a database |
| 417 |
containing 4 keys:: |
| 418 |
|
| 419 |
A B C D |
| 420 |
|
| 421 |
If you are at key B and insert a key BB, then it must be iterated over. |
| 422 |
If you insert AA while at B, it must NOT be iterated over. |
| 423 |
|
| 424 |
* Opening the same database multiple times. |
| 425 |
In the bad old days, opening the same database multiple times in |
| 426 |
the same process led to locking bugs (fcntl is braindead). Each |
| 427 |
database engine is responsible for making sure this doesn't happen. |
| 428 |
Most engines keep a linked lists of open databases. If you try to |
| 429 |
open the same database again, they will just return the existing |
| 430 |
opened copy and bump a refcount. Beware. If a database is locked |
| 431 |
and you try to lock again - thinking you were opening it brand new, |
| 432 |
it will assertion fail and/or error. |
| 433 |
|
| 434 |
I think that covers about everything! Cyrusdb is used just about |
| 435 |
everywhere that sorted key-value databases give what's needed, |
| 436 |
including mailboxes.db, annotations.db (global and per mailbox |
| 437 |
databases), seen state (non-owner), subscriptions, cyrus.indexed.db for |
| 438 |
Xapian, and the rather massive (and increasingly inaccurately named) |
| 439 |
user.conversations. |
| 440 |
|
| 441 |
Future plans are to increase the usage of cyrusdb databases, possibly |
| 442 |
by building an indexing layer on top and using that instead of the |
| 443 |
sqldb interface used for sqlite databases by DAV code, and possibly |
| 444 |
also moving other custom file formats into a cyrusdb to allow easier |
| 445 |
stateless server builds on a distributed backend. |
| 446 |
|
| 447 |
|
| 448 |
API Reference |
| 449 |
------------- |
| 450 |
|
| 451 |
All functions follow the normal C API of returning '0' on success, and |
| 452 |
an error code on failure |
| 453 |
|
| 454 |
cyrusdb\_init(void) |
| 455 |
+++++++++++++++++++ |
| 456 |
|
| 457 |
Is called once per process. Don't call this yourself, use |
| 458 |
``cyrus_init()``. No other calls will be made until this is called. |
| 459 |
|
| 460 |
cyrusdb\_done(void) |
| 461 |
+++++++++++++++++++ |
| 462 |
|
| 463 |
The opposite of ``cyrusdb_init()`` - called once per process to do any |
| 464 |
cleaning up after all database usage is finished. Don't call this |
| 465 |
yourself, use ``cyrus_done()``. |
| 466 |
|
| 467 |
cyrusdb\_sync(const char \*backend) |
| 468 |
+++++++++++++++++++++++++++++++++++ |
| 469 |
|
| 470 |
Perform a checkpoint of the database environment. Used by berkeley |
| 471 |
backend. Is called by ``ctl_cyrusdb -c`` on a regular basis |
| 472 |
|
| 473 |
cyrusdb\_open(const char \*backend, const char \*fname, int flags, struct db \*\*retdb) |
| 474 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 475 |
|
| 476 |
Opens the database with the specified 'file name' (or other descriptor, |
| 477 |
for example the sql backend is not a filename), and if successful |
| 478 |
returns an opaque database structure |
| 479 |
|
| 480 |
Flags: |
| 481 |
|
| 482 |
- CYRUSDB\_CREATE - create the database if it doesn't exist |
| 483 |
|
| 484 |
Errors: |
| 485 |
|
| 486 |
- CYRUSDB\_IOERROR - if there is any error reading the file, or any |
| 487 |
corruption detected while loading the file |
| 488 |
|
| 489 |
cyrusdb\_close(struct db \*db) |
| 490 |
++++++++++++++++++++++++++++++ |
| 491 |
|
| 492 |
Close the named database. Will release any locks if they are still held, |
| 493 |
but it's bad practice to close without committing or aborting, so the |
| 494 |
backend should log an error |
| 495 |
|
| 496 |
Errors: |
| 497 |
|
| 498 |
- CYRUSDB\_IOERROR - if there are any errors during close |
| 499 |
|
| 500 |
cyrusdb\_fetch(struct db \*db, const char \*key, size\_t keylen, const char \*\*data, size\_t \*datalen, struct txn \*\*tidptr) |
| 501 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 502 |
|
| 503 |
cyrusdb\_fetchlock(struct db \*db, const char \*key, size\_t keylen, const char \*\*data, size\_t \*datalen, struct txn \*\*tidptr) |
| 504 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 505 |
|
| 506 |
Fetch the value for the exact key given by key and keylen. If data is |
| 507 |
not NULL, set datalen and return a valid pointer to the start of the |
| 508 |
value. |
| 509 |
|
| 510 |
Fetchlock is identical to fetch, but gives a hint to the database that |
| 511 |
the record is likely to be modified soon. |
| 512 |
|
| 513 |
NOTE: it is possible to store a key with a zero length data record, in |
| 514 |
which case \*datalen will be set to zero, and \*data will be set to a |
| 515 |
non-NULL value |
| 516 |
|
| 517 |
It is an error to call fetch with a NULL key or a zero keylen |
| 518 |
|
| 519 |
It is an error to call fetch with a NULL datalen and a non-NULL data, |
| 520 |
however it is acceptable to call with a NULL data and a non-NULL datalen |
| 521 |
if you are only interested in the length |
| 522 |
|
| 523 |
Errors: |
| 524 |
|
| 525 |
- CYRUSDB\_IOERROR - if any error occurs reading from the database |
| 526 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 527 |
- CYRUSDB\_NOTFOUND - if there is no record that matches the key |
| 528 |
|
| 529 |
cyrusdb\_foreach(struct db \*db, const char \*prefix, size\_t prefixlen, foreach\_p \*goodp, foreach\_p \*procp, void \*rock, struct txn \*\*tidptr) |
| 530 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 531 |
|
| 532 |
cyrusdb\_forone(struct db \*db, const char \*key, size\_t keylen, foreach\_p \*goodp, foreach\_p \*procp, void \*rock, struct txn \*\*tidptr) |
| 533 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 534 |
|
| 535 |
``cyrusdb_foreach()`` iterates over all records matching the given |
| 536 |
prefix, in database order (which may be MBOXLIST sort, depending on the |
| 537 |
parameters given to open |
| 538 |
|
| 539 |
It is legal to give a NULL pointer as prefix if prefixlen is zero, in |
| 540 |
which case it will return all records in the database. It is an error to |
| 541 |
give a non-zero prefixlen with a NULL prefix. |
| 542 |
|
| 543 |
``cyrusdb_forone()`` "iterates" over the single record matched by the |
| 544 |
given key. If you've already built callbacks for processing each record |
| 545 |
from a foreach, this lets you use the same interface to process a single |
| 546 |
record. |
| 547 |
|
| 548 |
``goodp`` - this function is only used for deciding if the record needs |
| 549 |
to be further processed. It can be used for basic filtering, and returns |
| 550 |
true (non-zero) to process, or zero to skip and move straight to the |
| 551 |
next record. Because goodp can't make any database changes, it doesn't |
| 552 |
break the lock, so it's faster to use goodp to filter records if you |
| 553 |
don't need to process all of them. NULL is a legal value for goodp, and |
| 554 |
means that all records will be processed. |
| 555 |
|
| 556 |
``procp`` - procp is the main callback function. If you use foreach in |
| 557 |
non-transactional mode, the database is unlocked before calling procp, |
| 558 |
and locked again afterwards. You are allowed to add, delete or modify |
| 559 |
values in the same database from within procp. If procp returns |
| 560 |
non-zero, the foreach loop breaks at this point, and the return value of |
| 561 |
the foreach becomes the return value of procp. If procp returns zero, |
| 562 |
the foreach loop will continue at the NEXT record by sort order, |
| 563 |
regardless of whether the current record has changed or been removed. |
| 564 |
procp MUST NOT be NULL. |
| 565 |
|
| 566 |
Errors: |
| 567 |
|
| 568 |
- procp\_result - whatever your callback returns |
| 569 |
- CYRUSDB\_IOERROR - if any error occurs while reading |
| 570 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 571 |
|
| 572 |
cyrusdb\_create(struct db \*db, const char \*key, size\_t keylen, const char \*data, size\_t datalen, struct txn \*\*tidptr) |
| 573 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 574 |
|
| 575 |
cyrusdb\_store(struct db \*db, const char \*key, size\_t keylen, const char \*data, size\_t datalen, struct txn \*\*tidptr) |
| 576 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 577 |
|
| 578 |
Create a new record or replace an existing one. The only difference |
| 579 |
between these two is that ``cyrusdb_create`` will return an error if the |
| 580 |
record already exists, while ``cyrusdb_store`` will replace it |
| 581 |
|
| 582 |
If tidptr is NULL, create/store will take a write lock for the duration |
| 583 |
of the action. |
| 584 |
|
| 585 |
Any failure during create/store will abort the current transaction as |
| 586 |
well as returning an error |
| 587 |
|
| 588 |
It is legal to pass NULL for the data field ONLY if datalen is zero. It |
| 589 |
is not legal to pass NULL for key or zero for keylen |
| 590 |
|
| 591 |
Errors: |
| 592 |
|
| 593 |
- CYRUSDB\_IOERROR - any error to write to the database |
| 594 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 595 |
- CYRUSDB\_EXISTS - if ``cyrusdb_create`` is called on an existing key |
| 596 |
- CYRUSDB\_AGAIN - if a deadlock is created. The current transaction |
| 597 |
has been aborted, but a retry may succeed |
| 598 |
|
| 599 |
cyrusdb\_delete(struct db \*db, const char \*key, size\_t keylen, struct txn \*\*tidptr, int force) |
| 600 |
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| 601 |
|
| 602 |
Delete the given record from the database. If force is true, then |
| 603 |
succeed even if the record doesn't currently exist. |
| 604 |
|
| 605 |
It is not legal to pass NULL for key or zero for keylen |
| 606 |
|
| 607 |
Errors: |
| 608 |
|
| 609 |
- CYRUSDB\_IOERROR - any error to write to the database |
| 610 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 611 |
- CYRUSDB\_NOTFOUND - if force is not set and the key doesn't exist |
| 612 |
- CYRUSDB\_AGAIN - if a deadlock is created. The current transaction |
| 613 |
has been aborted, but a retry may succeed |
| 614 |
|
| 615 |
cyrusdb\_commit(struct db \*db, struct txn \*tid) |
| 616 |
+++++++++++++++++++++++++++++++++++++++++++++++++ |
| 617 |
|
| 618 |
Commit the current transaction. tid will not be valid after this call, |
| 619 |
regardless of success |
| 620 |
|
| 621 |
If the commit fails, it will attempt to abort the transaction |
| 622 |
|
| 623 |
Errors: |
| 624 |
|
| 625 |
- CYRUSDB\_IOERROR - any error to write to the database |
| 626 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 627 |
- CYRUSDB\_AGAIN - if a deadlock is created. The current transaction |
| 628 |
has been aborted, but a retry may succeed |
| 629 |
|
| 630 |
cyrusdb\_abort(struct db \*db, struct txn \*tid) |
| 631 |
++++++++++++++++++++++++++++++++++++++++++++++++ |
| 632 |
|
| 633 |
Abort the current transaction. tid will not be valid after this call, |
| 634 |
regardless of success |
| 635 |
|
| 636 |
Attempt to roll back all changes made in the current transaction. |
| 637 |
|
| 638 |
Errors: |
| 639 |
|
| 640 |
- CYRUSDB\_IOERROR - any error to write to the database |
| 641 |
- CYRUSDB\_LOCKED - if tidptr is incorrect |
| 642 |
|
| 643 |
cyrusdb\_dump(struct db \*db, int detail) |
| 644 |
+++++++++++++++++++++++++++++++++++++++++ |
| 645 |
|
| 646 |
Optional function to dump the internal structure of the database to |
| 647 |
stdout for debugging purposes. Don't use. |
| 648 |
|
| 649 |
cyrusdb\_consistent(struct db \*db) |
| 650 |
+++++++++++++++++++++++++++++++++++ |
| 651 |
|
| 652 |
Check if the DB is internally consistent. Looks pretty bogus, and isn't |
| 653 |
used anywhere. Don't use. |