| 1 |
<!DOCTYPE html> |
| 2 |
<html><head> |
| 3 |
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| 4 |
<meta http-equiv="content-type" content="text/html; charset=UTF-8"> |
| 5 |
<link href="sqlite.css" rel="stylesheet"> |
| 6 |
<title>The Checksum VFS Shim</title> |
| 7 |
<!-- path= --> |
| 8 |
</head> |
| 9 |
<body> |
| 10 |
<div class=nosearch> |
| 11 |
<a href="index.html"> |
| 12 |
<img class="logo" src="images/sqlite370_banner.svg" alt="SQLite" border="0"> |
| 13 |
</a> |
| 14 |
<div><!-- IE hack to prevent disappearing logo --></div> |
| 15 |
<div class="tagline desktoponly"> |
| 16 |
Small. Fast. Reliable.<br>Choose any three. |
| 17 |
</div> |
| 18 |
<div class="menu mainmenu"> |
| 19 |
<ul> |
| 20 |
<li><a href="index.html">Home</a> |
| 21 |
<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a> |
| 22 |
<li class='wideonly'><a href='about.html'>About</a> |
| 23 |
<li class='desktoponly'><a href="docs.html">Documentation</a> |
| 24 |
<li class='desktoponly'><a href="download.html">Download</a> |
| 25 |
<li class='wideonly'><a href='copyright.html'>License</a> |
| 26 |
<li class='desktoponly'><a href="support.html">Support</a> |
| 27 |
<li class='desktoponly'><a href="prosupport.html">Purchase</a> |
| 28 |
<li class='search' id='search_menubutton'> |
| 29 |
<a href="javascript:void(0)" onclick='toggle_search()'>Search</a> |
| 30 |
</ul> |
| 31 |
</div> |
| 32 |
<div class="menu submenu" id="submenu"> |
| 33 |
<ul> |
| 34 |
<li><a href='about.html'>About</a> |
| 35 |
<li><a href='docs.html'>Documentation</a> |
| 36 |
<li><a href='download.html'>Download</a> |
| 37 |
<li><a href='support.html'>Support</a> |
| 38 |
<li><a href='prosupport.html'>Purchase</a> |
| 39 |
</ul> |
| 40 |
</div> |
| 41 |
<div class="searchmenu" id="searchmenu"> |
| 42 |
<form method="GET" action="search"> |
| 43 |
<select name="s" id="searchtype"> |
| 44 |
<option value="d">Search Documentation</option> |
| 45 |
<option value="c">Search Changelog</option> |
| 46 |
</select> |
| 47 |
<input type="text" name="q" id="searchbox" value=""> |
| 48 |
<input type="submit" value="Go"> |
| 49 |
</form> |
| 50 |
</div> |
| 51 |
</div> |
| 52 |
<script> |
| 53 |
function toggle_div(nm) { |
| 54 |
var w = document.getElementById(nm); |
| 55 |
if( w.style.display=="block" ){ |
| 56 |
w.style.display = "none"; |
| 57 |
}else{ |
| 58 |
w.style.display = "block"; |
| 59 |
} |
| 60 |
} |
| 61 |
function toggle_search() { |
| 62 |
var w = document.getElementById("searchmenu"); |
| 63 |
if( w.style.display=="block" ){ |
| 64 |
w.style.display = "none"; |
| 65 |
} else { |
| 66 |
w.style.display = "block"; |
| 67 |
setTimeout(function(){ |
| 68 |
document.getElementById("searchbox").focus() |
| 69 |
}, 30); |
| 70 |
} |
| 71 |
} |
| 72 |
function div_off(nm){document.getElementById(nm).style.display="none";} |
| 73 |
window.onbeforeunload = function(e){div_off("submenu");} |
| 74 |
/* Disable the Search feature if we are not operating from CGI, since */ |
| 75 |
/* Search is accomplished using CGI and will not work without it. */ |
| 76 |
if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){ |
| 77 |
document.getElementById("search_menubutton").style.display = "none"; |
| 78 |
} |
| 79 |
/* Used by the Hide/Show button beside syntax diagrams, to toggle the */ |
| 80 |
function hideorshow(btn,obj){ |
| 81 |
var x = document.getElementById(obj); |
| 82 |
var b = document.getElementById(btn); |
| 83 |
if( x.style.display!='none' ){ |
| 84 |
x.style.display = 'none'; |
| 85 |
b.innerHTML='show'; |
| 86 |
}else{ |
| 87 |
x.style.display = ''; |
| 88 |
b.innerHTML='hide'; |
| 89 |
} |
| 90 |
return false; |
| 91 |
} |
| 92 |
var antiRobot = 0; |
| 93 |
function antiRobotGo(){ |
| 94 |
if( antiRobot!=3 ) return; |
| 95 |
antiRobot = 7; |
| 96 |
var j = document.getElementById("mtimelink"); |
| 97 |
if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href"); |
| 98 |
} |
| 99 |
function antiRobotDefense(){ |
| 100 |
document.body.onmousedown=function(){ |
| 101 |
antiRobot |= 2; |
| 102 |
antiRobotGo(); |
| 103 |
document.body.onmousedown=null; |
| 104 |
} |
| 105 |
document.body.onmousemove=function(){ |
| 106 |
antiRobot |= 2; |
| 107 |
antiRobotGo(); |
| 108 |
document.body.onmousemove=null; |
| 109 |
} |
| 110 |
setTimeout(function(){ |
| 111 |
antiRobot |= 1; |
| 112 |
antiRobotGo(); |
| 113 |
}, 100) |
| 114 |
antiRobotGo(); |
| 115 |
} |
| 116 |
antiRobotDefense(); |
| 117 |
</script> |
| 118 |
<div class=fancy> |
| 119 |
<div class=nosearch> |
| 120 |
<div class="fancy_title"> |
| 121 |
The Checksum VFS Shim |
| 122 |
</div> |
| 123 |
<details class="fancy_toc"> |
| 124 |
<summary>Table Of Contents</summary> |
| 125 |
<div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div> |
| 126 |
<div class="fancy-toc1"><a href="#compiling">2. Compiling</a></div> |
| 127 |
<div class="fancy-toc1"><a href="#loading">3. Loading</a></div> |
| 128 |
<div class="fancy-toc1"><a href="#usage">4. Usage</a></div> |
| 129 |
<div class="fancy-toc1"><a href="#verification_of_checksums">5. Verification Of Checksums</a></div> |
| 130 |
<div class="fancy-toc1"><a href="#controlling_checksum_verification">6. Controlling Checksum Verification</a></div> |
| 131 |
</div> |
| 132 |
</details> |
| 133 |
</div> |
| 134 |
|
| 135 |
|
| 136 |
|
| 137 |
|
| 138 |
|
| 139 |
<h1 id="overview"><span>1. </span>Overview</h1> |
| 140 |
|
| 141 |
<p>The checksum VFS extension is a <a href="vfs.html#shim">VFS shim</a> that adds an 8-byte |
| 142 |
checksum to the end of every page in an SQLite database. The checksum |
| 143 |
is added as each page is written and verified as each page is read. |
| 144 |
The checksum is intended to help detect database corruption caused by |
| 145 |
random bit-flips in the mass storage device. |
| 146 |
|
| 147 |
</p><p>The checksum VFS extension requires |
| 148 |
SQLite version 3.32.0 (2020-05-22) or later. It will not |
| 149 |
work with earlier versions of SQLite. |
| 150 |
|
| 151 |
</p><h1 id="compiling"><span>2. </span>Compiling</h1> |
| 152 |
|
| 153 |
<p>The checksum VFS module is a <a href="loadext.html">loadable extension</a>. It is not |
| 154 |
included in the <a href="amalgamation.html">amalgamation</a>. It must be added to SQLite |
| 155 |
either at compile-time or at run-time. The source code to |
| 156 |
the checksum VFS module is in the |
| 157 |
<a href="https://sqlite.org/src/file/ext/misc/cksumvfs.c">ext/misc/cksumvfs.c</a> |
| 158 |
source file in the |
| 159 |
<a href="https://sqlite.org/src">SQLite source tree</a>. |
| 160 |
|
| 161 |
</p><p>To build the checksum VFS module into a run-time loadable |
| 162 |
extension, use commands similar to the following: |
| 163 |
|
| 164 |
</p><ul> |
| 165 |
<li>(linux) → <tt>gcc -fPIC -shared cksumvfs.c -o cksumvfs.so</tt> |
| 166 |
</li><li>(mac) → <tt>clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib</tt> |
| 167 |
</li><li>(windows) → <tt>cl cksumvfs.c -link -dll -out:cksumvfs.dll</tt> |
| 168 |
</li></ul> |
| 169 |
|
| 170 |
<p>You may want to add additional compiler options, of course, |
| 171 |
according to the needs of your project. |
| 172 |
|
| 173 |
</p><p>To statically link this extension with your product, |
| 174 |
compile it like any other C-language module but add the |
| 175 |
"-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that |
| 176 |
it is being statically linked rather than dynamically linked. |
| 177 |
|
| 178 |
</p><h1 id="loading"><span>3. </span>Loading</h1> |
| 179 |
|
| 180 |
<p>To load this extension as a shared library, you first have to |
| 181 |
bring up a dummy SQLite database connection to use as the argument |
| 182 |
to the <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API call. Then you invoke the |
| 183 |
<a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API and shutdown the dummy database |
| 184 |
connection. All subsequent database connections that are opened |
| 185 |
will include this extension. For example: |
| 186 |
|
| 187 |
</p><div class="codeblock"><pre>sqlite3 *db; |
| 188 |
sqlite3_open(":memory:", &db); |
| 189 |
sqlite3_load_extension(db, "./cksumvfs"); |
| 190 |
sqlite3_close(db); |
| 191 |
</pre></div> |
| 192 |
|
| 193 |
<p>If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and |
| 194 |
statically linked against the application, initialize it using |
| 195 |
a single API call as follows: |
| 196 |
|
| 197 |
</p><div class="codeblock"><pre>sqlite3_cksumvfs_init(); |
| 198 |
</pre></div> |
| 199 |
|
| 200 |
<p>Cksumvfs is a <a href="vfs.html#shim">VFS shim</a>. When loaded, "cksmvfs" becomes the new |
| 201 |
default VFS, and it uses the prior default VFS as the next VFS |
| 202 |
down in the stack. This is normally what you want. However, in |
| 203 |
complex situations where multiple VFS shims are being loaded, |
| 204 |
it might be important to ensure that cksumvfs is loaded in the |
| 205 |
correct order so that it sequences itself into the default VFS |
| 206 |
Shim stack in the right order. |
| 207 |
|
| 208 |
</p><h1 id="usage"><span>4. </span>Usage</h1> |
| 209 |
|
| 210 |
<p>Open database connections using the <a href="c3ref/open.html">sqlite3_open()</a> or |
| 211 |
<a href="c3ref/open.html">sqlite3_open_v2()</a> interfaces, as normal. Ordinary database files |
| 212 |
(without a checksum) will operate normally. Databases with |
| 213 |
checksums will return an SQLITE_IOERR_DATA error if a page is |
| 214 |
encountered that contains an invalid checksum. |
| 215 |
|
| 216 |
</p><p>Checksumming only works on databases that have a <a href="fileformat2.html#resbyte">reserve bytes</a> |
| 217 |
value of exactly 8. The default value for reserve-bytes is 0. |
| 218 |
Hence, newly created database files will omit the checksum by |
| 219 |
default. To create a database that includes a checksum, change |
| 220 |
the reserve-bytes value to 8 by running code similar to this: |
| 221 |
|
| 222 |
</p><div class="codeblock"><pre>int n = 8; |
| 223 |
sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVE_BYTES, &n); |
| 224 |
</pre></div> |
| 225 |
|
| 226 |
<p>If you do this immediately after creating a new database file, |
| 227 |
before anything else has been written into the file, then that |
| 228 |
might be all that you need to do. Otherwise, the API call |
| 229 |
above should be followed by: |
| 230 |
|
| 231 |
</p><div class="codeblock"><pre>sqlite3_exec(db, "VACUUM", 0, 0, 0); |
| 232 |
</pre></div> |
| 233 |
|
| 234 |
<p>It never hurts to run the VACUUM, even if you don't need it. |
| 235 |
If the database is in WAL mode, you should shutdown and |
| 236 |
reopen all database connections before continuing. |
| 237 |
|
| 238 |
</p><p>From the CLI, use the ".filectrl reserve_bytes 8" command, |
| 239 |
followed by "VACUUM;". |
| 240 |
|
| 241 |
</p><p>Note that SQLite allows the number of reserve-bytes to be |
| 242 |
increased but not decreased. So if a database file already |
| 243 |
has a reserve-bytes value greater than 8, there is no way to |
| 244 |
activate checksumming on that database, other than to dump |
| 245 |
and restore the database file. Note also that other extensions |
| 246 |
might also make use of the reserve-bytes. Checksumming will |
| 247 |
be incompatible with those other extensions. |
| 248 |
|
| 249 |
</p><h1 id="verification_of_checksums"><span>5. </span>Verification Of Checksums</h1> |
| 250 |
|
| 251 |
<p>If any checksum is incorrect, the "PRAGMA quick_check" command |
| 252 |
will find it. To verify that checksums are actually enabled |
| 253 |
and running, use SQL like the following: |
| 254 |
|
| 255 |
</p><div class="codeblock"><pre>SELECT count(*), verify_checksum(data) |
| 256 |
FROM sqlite_dbpage |
| 257 |
GROUP BY 2; |
| 258 |
</pre></div> |
| 259 |
|
| 260 |
<p>There are three possible outputs from the verify_checksum() |
| 261 |
function: 1, 0, and NULL. 1 is returned if the checksum is |
| 262 |
correct. 0 is returned if the checksum is incorrect. NULL |
| 263 |
is returned if the page is unreadable. If checksumming is |
| 264 |
enabled, the read will fail if the checksum is wrong, so the |
| 265 |
usual result from verify_checksum() on a bad checksum is NULL. |
| 266 |
|
| 267 |
</p><p>If everything is OK, the query above should return a single |
| 268 |
row where the second column is 1. Any other result indicates |
| 269 |
either that there is a checksum error, or checksum validation |
| 270 |
is disabled. |
| 271 |
|
| 272 |
</p><h1 id="controlling_checksum_verification"><span>6. </span>Controlling Checksum Verification</h1> |
| 273 |
|
| 274 |
<p>The cksumvfs extension implements a new PRAGMA statement that can |
| 275 |
be used to disable, re-enable, or query the status of checksum |
| 276 |
verification: |
| 277 |
|
| 278 |
</p><div class="codeblock"><pre>PRAGMA checksum_verification; -- query status |
| 279 |
PRAGMA checksum_verification=OFF; -- disable verification |
| 280 |
PRAGMA checksum_verification=ON; -- re-enable verification |
| 281 |
</pre></div> |
| 282 |
|
| 283 |
<p>The "checksum_verification" pragma will return "1" (true) or "0" |
| 284 |
(false) if checksum verification is enabled or disabled, respectively. |
| 285 |
"Verification" in this context means the feature that causes |
| 286 |
SQLITE_IOERR_DATA errors if a checksum mismatch is detected while |
| 287 |
reading. Checksums are always kept up-to-date as long as the |
| 288 |
<a href="fileformat2.html#resbyte">reserve bytes</a> value of the database is 8, regardless of the setting |
| 289 |
of this pragma. Checksum verification can be disabled (for example) |
| 290 |
to do forensic analysis of a database that has previously reported |
| 291 |
a checksum error. |
| 292 |
|
| 293 |
</p><p>The "checksum_verification" pragma will always respond with "0" if |
| 294 |
the database file does not have a <a href="fileformat2.html#resbyte">reserve bytes</a> value of 8. The |
| 295 |
pragma will return no rows at all if the cksumvfs extension is |
| 296 |
not loaded. |
| 297 |
</p><p align="center"><small><i>This page was last updated on 2022-01-08 05:02:57Z </small></i></p> |
| 298 |
|