Debian Codegraph @

www/cksumvfs.html

raw
1 <!DOCTYPE html>
2 <html><head>
3 <meta name="viewport" content="width=device-width, initial-scale=1.0">
4 <meta http-equiv="content-type" content="text/html; charset=UTF-8">
5 <link href="sqlite.css" rel="stylesheet">
6 <title>The Checksum VFS Shim</title>
7 <!-- path= -->
8 </head>
9 <body>
10 <div class=nosearch>
11 <a href="index.html">
12 <img class="logo" src="images/sqlite370_banner.svg" alt="SQLite" border="0">
13 </a>
14 <div><!-- IE hack to prevent disappearing logo --></div>
15 <div class="tagline desktoponly">
16 Small. Fast. Reliable.<br>Choose any three.
17 </div>
18 <div class="menu mainmenu">
19 <ul>
20 <li><a href="index.html">Home</a>
21 <li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a>
22 <li class='wideonly'><a href='about.html'>About</a>
23 <li class='desktoponly'><a href="docs.html">Documentation</a>
24 <li class='desktoponly'><a href="download.html">Download</a>
25 <li class='wideonly'><a href='copyright.html'>License</a>
26 <li class='desktoponly'><a href="support.html">Support</a>
27 <li class='desktoponly'><a href="prosupport.html">Purchase</a>
28 <li class='search' id='search_menubutton'>
29 <a href="javascript:void(0)" onclick='toggle_search()'>Search</a>
30 </ul>
31 </div>
32 <div class="menu submenu" id="submenu">
33 <ul>
34 <li><a href='about.html'>About</a>
35 <li><a href='docs.html'>Documentation</a>
36 <li><a href='download.html'>Download</a>
37 <li><a href='support.html'>Support</a>
38 <li><a href='prosupport.html'>Purchase</a>
39 </ul>
40 </div>
41 <div class="searchmenu" id="searchmenu">
42 <form method="GET" action="search">
43 <select name="s" id="searchtype">
44 <option value="d">Search Documentation</option>
45 <option value="c">Search Changelog</option>
46 </select>
47 <input type="text" name="q" id="searchbox" value="">
48 <input type="submit" value="Go">
49 </form>
50 </div>
51 </div>
52 <script>
53 function toggle_div(nm) {
54 var w = document.getElementById(nm);
55 if( w.style.display=="block" ){
56 w.style.display = "none";
57 }else{
58 w.style.display = "block";
59 }
60 }
61 function toggle_search() {
62 var w = document.getElementById("searchmenu");
63 if( w.style.display=="block" ){
64 w.style.display = "none";
65 } else {
66 w.style.display = "block";
67 setTimeout(function(){
68 document.getElementById("searchbox").focus()
69 }, 30);
70 }
71 }
72 function div_off(nm){document.getElementById(nm).style.display="none";}
73 window.onbeforeunload = function(e){div_off("submenu");}
74 /* Disable the Search feature if we are not operating from CGI, since */
75 /* Search is accomplished using CGI and will not work without it. */
76 if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){
77 document.getElementById("search_menubutton").style.display = "none";
78 }
79 /* Used by the Hide/Show button beside syntax diagrams, to toggle the */
80 function hideorshow(btn,obj){
81 var x = document.getElementById(obj);
82 var b = document.getElementById(btn);
83 if( x.style.display!='none' ){
84 x.style.display = 'none';
85 b.innerHTML='show';
86 }else{
87 x.style.display = '';
88 b.innerHTML='hide';
89 }
90 return false;
91 }
92 var antiRobot = 0;
93 function antiRobotGo(){
94 if( antiRobot!=3 ) return;
95 antiRobot = 7;
96 var j = document.getElementById("mtimelink");
97 if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href");
98 }
99 function antiRobotDefense(){
100 document.body.onmousedown=function(){
101 antiRobot |= 2;
102 antiRobotGo();
103 document.body.onmousedown=null;
104 }
105 document.body.onmousemove=function(){
106 antiRobot |= 2;
107 antiRobotGo();
108 document.body.onmousemove=null;
109 }
110 setTimeout(function(){
111 antiRobot |= 1;
112 antiRobotGo();
113 }, 100)
114 antiRobotGo();
115 }
116 antiRobotDefense();
117 </script>
118 <div class=fancy>
119 <div class=nosearch>
120 <div class="fancy_title">
121 The Checksum VFS Shim
122 </div>
123 <details class="fancy_toc">
124 <summary>Table Of Contents</summary>
125 <div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div>
126 <div class="fancy-toc1"><a href="#compiling">2. Compiling</a></div>
127 <div class="fancy-toc1"><a href="#loading">3. Loading</a></div>
128 <div class="fancy-toc1"><a href="#usage">4. Usage</a></div>
129 <div class="fancy-toc1"><a href="#verification_of_checksums">5. Verification Of Checksums</a></div>
130 <div class="fancy-toc1"><a href="#controlling_checksum_verification">6. Controlling Checksum Verification</a></div>
131 </div>
132 </details>
133 </div>
134
135
136
137
138
139 <h1 id="overview"><span>1. </span>Overview</h1>
140
141 <p>The checksum VFS extension is a <a href="vfs.html#shim">VFS shim</a> that adds an 8-byte
142 checksum to the end of every page in an SQLite database. The checksum
143 is added as each page is written and verified as each page is read.
144 The checksum is intended to help detect database corruption caused by
145 random bit-flips in the mass storage device.
146
147 </p><p>The checksum VFS extension requires
148 SQLite version 3.32.0 (2020-05-22) or later. It will not
149 work with earlier versions of SQLite.
150
151 </p><h1 id="compiling"><span>2. </span>Compiling</h1>
152
153 <p>The checksum VFS module is a <a href="loadext.html">loadable extension</a>. It is not
154 included in the <a href="amalgamation.html">amalgamation</a>. It must be added to SQLite
155 either at compile-time or at run-time. The source code to
156 the checksum VFS module is in the
157 <a href="https://sqlite.org/src/file/ext/misc/cksumvfs.c">ext/misc/cksumvfs.c</a>
158 source file in the
159 <a href="https://sqlite.org/src">SQLite source tree</a>.
160
161 </p><p>To build the checksum VFS module into a run-time loadable
162 extension, use commands similar to the following:
163
164 </p><ul>
165 <li>(linux) &rarr; <tt>gcc -fPIC -shared cksumvfs.c -o cksumvfs.so</tt>
166 </li><li>(mac) &rarr; <tt>clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib</tt>
167 </li><li>(windows) &rarr; <tt>cl cksumvfs.c -link -dll -out:cksumvfs.dll</tt>
168 </li></ul>
169
170 <p>You may want to add additional compiler options, of course,
171 according to the needs of your project.
172
173 </p><p>To statically link this extension with your product,
174 compile it like any other C-language module but add the
175 "-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that
176 it is being statically linked rather than dynamically linked.
177
178 </p><h1 id="loading"><span>3. </span>Loading</h1>
179
180 <p>To load this extension as a shared library, you first have to
181 bring up a dummy SQLite database connection to use as the argument
182 to the <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API call. Then you invoke the
183 <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API and shutdown the dummy database
184 connection. All subsequent database connections that are opened
185 will include this extension. For example:
186
187 </p><div class="codeblock"><pre>sqlite3 *db;
188 sqlite3_open(":memory:", &db);
189 sqlite3_load_extension(db, "./cksumvfs");
190 sqlite3_close(db);
191 </pre></div>
192
193 <p>If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and
194 statically linked against the application, initialize it using
195 a single API call as follows:
196
197 </p><div class="codeblock"><pre>sqlite3_cksumvfs_init();
198 </pre></div>
199
200 <p>Cksumvfs is a <a href="vfs.html#shim">VFS shim</a>. When loaded, "cksmvfs" becomes the new
201 default VFS, and it uses the prior default VFS as the next VFS
202 down in the stack. This is normally what you want. However, in
203 complex situations where multiple VFS shims are being loaded,
204 it might be important to ensure that cksumvfs is loaded in the
205 correct order so that it sequences itself into the default VFS
206 Shim stack in the right order.
207
208 </p><h1 id="usage"><span>4. </span>Usage</h1>
209
210 <p>Open database connections using the <a href="c3ref/open.html">sqlite3_open()</a> or
211 <a href="c3ref/open.html">sqlite3_open_v2()</a> interfaces, as normal. Ordinary database files
212 (without a checksum) will operate normally. Databases with
213 checksums will return an SQLITE_IOERR_DATA error if a page is
214 encountered that contains an invalid checksum.
215
216 </p><p>Checksumming only works on databases that have a <a href="fileformat2.html#resbyte">reserve bytes</a>
217 value of exactly 8. The default value for reserve-bytes is 0.
218 Hence, newly created database files will omit the checksum by
219 default. To create a database that includes a checksum, change
220 the reserve-bytes value to 8 by running code similar to this:
221
222 </p><div class="codeblock"><pre>int n = 8;
223 sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVE_BYTES, &n);
224 </pre></div>
225
226 <p>If you do this immediately after creating a new database file,
227 before anything else has been written into the file, then that
228 might be all that you need to do. Otherwise, the API call
229 above should be followed by:
230
231 </p><div class="codeblock"><pre>sqlite3_exec(db, "VACUUM", 0, 0, 0);
232 </pre></div>
233
234 <p>It never hurts to run the VACUUM, even if you don't need it.
235 If the database is in WAL mode, you should shutdown and
236 reopen all database connections before continuing.
237
238 </p><p>From the CLI, use the ".filectrl reserve_bytes 8" command,
239 followed by "VACUUM;".
240
241 </p><p>Note that SQLite allows the number of reserve-bytes to be
242 increased but not decreased. So if a database file already
243 has a reserve-bytes value greater than 8, there is no way to
244 activate checksumming on that database, other than to dump
245 and restore the database file. Note also that other extensions
246 might also make use of the reserve-bytes. Checksumming will
247 be incompatible with those other extensions.
248
249 </p><h1 id="verification_of_checksums"><span>5. </span>Verification Of Checksums</h1>
250
251 <p>If any checksum is incorrect, the "PRAGMA quick_check" command
252 will find it. To verify that checksums are actually enabled
253 and running, use SQL like the following:
254
255 </p><div class="codeblock"><pre>SELECT count(*), verify_checksum(data)
256 FROM sqlite_dbpage
257 GROUP BY 2;
258 </pre></div>
259
260 <p>There are three possible outputs from the verify_checksum()
261 function: 1, 0, and NULL. 1 is returned if the checksum is
262 correct. 0 is returned if the checksum is incorrect. NULL
263 is returned if the page is unreadable. If checksumming is
264 enabled, the read will fail if the checksum is wrong, so the
265 usual result from verify_checksum() on a bad checksum is NULL.
266
267 </p><p>If everything is OK, the query above should return a single
268 row where the second column is 1. Any other result indicates
269 either that there is a checksum error, or checksum validation
270 is disabled.
271
272 </p><h1 id="controlling_checksum_verification"><span>6. </span>Controlling Checksum Verification</h1>
273
274 <p>The cksumvfs extension implements a new PRAGMA statement that can
275 be used to disable, re-enable, or query the status of checksum
276 verification:
277
278 </p><div class="codeblock"><pre>PRAGMA checksum_verification; -- query status
279 PRAGMA checksum_verification=OFF; -- disable verification
280 PRAGMA checksum_verification=ON; -- re-enable verification
281 </pre></div>
282
283 <p>The "checksum_verification" pragma will return "1" (true) or "0"
284 (false) if checksum verification is enabled or disabled, respectively.
285 "Verification" in this context means the feature that causes
286 SQLITE_IOERR_DATA errors if a checksum mismatch is detected while
287 reading. Checksums are always kept up-to-date as long as the
288 <a href="fileformat2.html#resbyte">reserve bytes</a> value of the database is 8, regardless of the setting
289 of this pragma. Checksum verification can be disabled (for example)
290 to do forensic analysis of a database that has previously reported
291 a checksum error.
292
293 </p><p>The "checksum_verification" pragma will always respond with "0" if
294 the database file does not have a <a href="fileformat2.html#resbyte">reserve bytes</a> value of 8. The
295 pragma will return no rows at all if the cksumvfs extension is
296 not loaded.
297 </p><p align="center"><small><i>This page was last updated on 2022-01-08 05:02:57Z </small></i></p>
298