Debian Codegraph @

www/mmap.html

raw
1 <!DOCTYPE html>
2 <html><head>
3 <meta name="viewport" content="width=device-width, initial-scale=1.0">
4 <meta http-equiv="content-type" content="text/html; charset=UTF-8">
5 <link href="sqlite.css" rel="stylesheet">
6 <title>Memory-Mapped I/O</title>
7 <!-- path= -->
8 </head>
9 <body>
10 <div class=nosearch>
11 <a href="index.html">
12 <img class="logo" src="images/sqlite370_banner.svg" alt="SQLite" border="0">
13 </a>
14 <div><!-- IE hack to prevent disappearing logo --></div>
15 <div class="tagline desktoponly">
16 Small. Fast. Reliable.<br>Choose any three.
17 </div>
18 <div class="menu mainmenu">
19 <ul>
20 <li><a href="index.html">Home</a>
21 <li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a>
22 <li class='wideonly'><a href='about.html'>About</a>
23 <li class='desktoponly'><a href="docs.html">Documentation</a>
24 <li class='desktoponly'><a href="download.html">Download</a>
25 <li class='wideonly'><a href='copyright.html'>License</a>
26 <li class='desktoponly'><a href="support.html">Support</a>
27 <li class='desktoponly'><a href="prosupport.html">Purchase</a>
28 <li class='search' id='search_menubutton'>
29 <a href="javascript:void(0)" onclick='toggle_search()'>Search</a>
30 </ul>
31 </div>
32 <div class="menu submenu" id="submenu">
33 <ul>
34 <li><a href='about.html'>About</a>
35 <li><a href='docs.html'>Documentation</a>
36 <li><a href='download.html'>Download</a>
37 <li><a href='support.html'>Support</a>
38 <li><a href='prosupport.html'>Purchase</a>
39 </ul>
40 </div>
41 <div class="searchmenu" id="searchmenu">
42 <form method="GET" action="search">
43 <select name="s" id="searchtype">
44 <option value="d">Search Documentation</option>
45 <option value="c">Search Changelog</option>
46 </select>
47 <input type="text" name="q" id="searchbox" value="">
48 <input type="submit" value="Go">
49 </form>
50 </div>
51 </div>
52 <script>
53 function toggle_div(nm) {
54 var w = document.getElementById(nm);
55 if( w.style.display=="block" ){
56 w.style.display = "none";
57 }else{
58 w.style.display = "block";
59 }
60 }
61 function toggle_search() {
62 var w = document.getElementById("searchmenu");
63 if( w.style.display=="block" ){
64 w.style.display = "none";
65 } else {
66 w.style.display = "block";
67 setTimeout(function(){
68 document.getElementById("searchbox").focus()
69 }, 30);
70 }
71 }
72 function div_off(nm){document.getElementById(nm).style.display="none";}
73 window.onbeforeunload = function(e){div_off("submenu");}
74 /* Disable the Search feature if we are not operating from CGI, since */
75 /* Search is accomplished using CGI and will not work without it. */
76 if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){
77 document.getElementById("search_menubutton").style.display = "none";
78 }
79 /* Used by the Hide/Show button beside syntax diagrams, to toggle the */
80 function hideorshow(btn,obj){
81 var x = document.getElementById(obj);
82 var b = document.getElementById(btn);
83 if( x.style.display!='none' ){
84 x.style.display = 'none';
85 b.innerHTML='show';
86 }else{
87 x.style.display = '';
88 b.innerHTML='hide';
89 }
90 return false;
91 }
92 var antiRobot = 0;
93 function antiRobotGo(){
94 if( antiRobot!=3 ) return;
95 antiRobot = 7;
96 var j = document.getElementById("mtimelink");
97 if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href");
98 }
99 function antiRobotDefense(){
100 document.body.onmousedown=function(){
101 antiRobot |= 2;
102 antiRobotGo();
103 document.body.onmousedown=null;
104 }
105 document.body.onmousemove=function(){
106 antiRobot |= 2;
107 antiRobotGo();
108 document.body.onmousemove=null;
109 }
110 setTimeout(function(){
111 antiRobot |= 1;
112 antiRobotGo();
113 }, 100)
114 antiRobotGo();
115 }
116 antiRobotDefense();
117 </script>
118
119
120
121 <h1 align="center">Memory-Mapped I/O</h1>
122
123 <p>The default mechanism by which SQLite accesses and updates database disk
124 files is the xRead() and xWrite() methods of the
125 <a href="c3ref/io_methods.html">sqlite3_io_methods</a> VFS object. These methods are typically implemented as
126 "read()" and "write()" system calls which cause the operating system
127 to copy disk content between the kernel buffer cache and user space.</p>
128
129 <p>Beginning with <a href="releaselog/3_7_17.html">version 3.7.17</a> (2013-05-20), SQLite has the option of
130 accessing disk content directly using memory-mapped I/O and the new
131 xFetch() and xUnfetch() methods on <a href="c3ref/io_methods.html">sqlite3_io_methods</a>.</p>
132
133 <p>There are advantages and disadvantages to using memory-mapped I/O.
134 Advantages include:</p>
135
136 <ol>
137 <li><p>Many operations, especially I/O intensive operations, can be
138 faster since content need not be copied between kernel space
139 and user space.</p>
140
141 <li><p>The SQLite library may need less RAM since it shares pages with
142 the operating-system page cache and does not always need its own copy of
143 working pages.</p>
144 </ol>
145
146 <p>But there are also disadvantages:</p>
147
148 <ol>
149 <li><p>An I/O error on a memory-mapped file cannot be caught and dealt with by
150 SQLite. Instead, the I/O error causes a signal which, if not caught
151 by the application, results in a program crash.</p>
152
153 <li><p>The operating system must have a unified buffer cache in order for
154 the memory-mapped I/O extension to work correctly, especially in
155 situations where two processes are accessing the same database
156 file and one process is using memory-mapped I/O while the other
157 is not. Not all operating systems have a unified buffer cache.
158 In some operating systems that claim to have a unified buffer cache,
159 the implementation is buggy and can lead to corrupt databases.</p>
160
161 <li><p>Performance does not always increase with memory-mapped I/O. In fact,
162 it is possible to construct test cases where performance is reduced
163 by the use of memory-mapped I/O.</p>
164
165 <li><p>Windows is unable to truncate a memory-mapped file. Hence, on Windows,
166 if an operation such as <a href="lang_vacuum.html">VACUUM</a> or <a href="pragma.html#pragma_auto_vacuum">auto_vacuum</a> tries to reduce the
167 size of a memory-mapped database file, the size reduction attempt will
168 silently fail, leaving unused space at the end of the database file.
169 No data is lost due to this problem, and the unused space will be
170 reused again the next time the database grows. However if a version
171 of SQLite prior to 3.7.0 runs <a href="pragma.html#pragma_integrity_check">PRAGMA integrity_check</a> on such a
172 database, it will (incorrectly) report database corruption due to
173 the unused space at the end. Or if a version of SQLite prior to 3.7.0
174 writes to the database while it still has unused space at the end, it
175 may make that unused space inaccessible and unavailable for reuse until
176 after the next <a href="lang_vacuum.html">VACUUM</a>.
177 </ol>
178
179 <p>Because of the potential disadvantages, memory-mapped I/O is disabled
180 by default. To activate memory-mapped I/O, use the <a href="pragma.html#pragma_mmap_size">mmap_size pragma</a>
181 and set the mmap_size to some large number, usually 256MB or larger, depending
182 on how much address space your application can spare. The rest is
183 automatic. The <a href="pragma.html#pragma_mmap_size">PRAGMA mmap_size</a> statement will be a silent no-op on
184 systems that do not support memory-mapped I/O.</p>
185
186 <h2>How Memory-Mapped I/O Works</h2>
187
188 <p>To read a page of database content using the legacy xRead() method,
189 SQLite first allocates a page-size chunk of heap memory then invokes
190 the xRead() method which causes the database page content to be copied
191 into the newly allocated heap memory. This involves (at a minimum)
192 a copy of the entire page.</p>
193
194 <p>But if SQLite wants to access a page of the database file and
195 memory mapped I/O is enabled, it first calls the xFetch() method.
196 The xFetch() method asks the operating system to return a pointer to
197 the requested page, if possible. If the requested page has been or
198 can be mapped into the application address space, then xFetch returns
199 a pointer to that page for SQLite to use without having to copy anything.
200 Skipping the copy step is what makes memory mapped I/O faster.</p>
201
202 <p>SQLite does not assume that the xFetch() method will work. If
203 a call to xFetch() returns a NULL pointer (indicating that the requested
204 page is not currently mapped into the applications address space) then
205 SQLite silently falls back to using xRead(). An error is only reported
206 if xRead() also fails.</p>
207
208 <p>When updating the database file, SQLite always makes a copy of the
209 page content into heap memory before modifying the page. This is necessary
210 for two reasons. First, changes to the database
211 are not supposed to be visible to other processes until
212 after the transaction commits and so the changes must occur in private memory.
213 Second, SQLite uses a read-only memory map to prevent stray pointers in the
214 application from overwriting and corrupting the database file.
215
216 <p>
217 After all needed changes are completed, xWrite() is used to move the content
218 back into the database file.
219 Hence the use of memory mapped I/O does not significantly change the
220 performance of database changes.
221 Memory mapped I/O is mostly a benefit for queries.</p>
222
223 <h2>Configuring Memory-Mapped I/O</h2>
224
225 <p>The "mmap_size" is the maximum number of bytes of the database file that
226 SQLite will try to map into the process address space at one time. The
227 mmap_size applies separately to each database file, so the total amount
228 of process address space that could potentially be used is the mmap_size
229 times the number of open database files.</p>
230
231 <p>To activate memory-mapped I/O, an application can set the mmap_size to some
232 large value. For example:</p>
233
234 <blockquote><pre>
235 PRAGMA mmap_size=268435456;
236 </pre></blockquote>
237
238 <p>To disable memory-mapped I/O, simply set the mmap_size to zero:</p>
239
240 <blockquote><pre>
241 PRAGMA mmap_size=0;
242 </pre></blockquote>
243
244 <p>If mmap_size is set to N then all current implementations map the first
245 N bytes of the database file and use legacy xRead() calls for any content
246 beyond N bytes. If the database file is smaller than N bytes, then the entire
247 file is mapped. In the future, new OS interfaces could, in theory, map
248 regions of the file other than the first N bytes, but no such
249 implementation currently exists.</p>
250
251 <p>The mmap_size is set separately for each database file using the
252 "<a href="pragma.html#pragma_mmap_size">PRAGMA mmap_size</a>" statement. The usual default mmap_size is zero,
253 meaning that memory mapped I/O is disabled by default. However, the
254 default mmap_size can be increased either at compile-time using
255 the <a href="compile.html#default_mmap_size">SQLITE_DEFAULT_MMAP_SIZE</a> macro or at start-time using the
256 <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_covering_index_scan.html#sqliteconfigmmapsize">SQLITE_CONFIG_MMAP_SIZE</a>,...) interface.</p>
257
258 <p>SQLite also maintains a hard upper bound on the mmap_size. Attempts
259 to increase the mmap_size above this hard upper bound (using
260 <a href="pragma.html#pragma_mmap_size">PRAGMA mmap_size</a>) will automatically cap the mmap_size at the hard
261 upper bound. If the hard upper bound is zero, then memory mapped I/O
262 is impossible. The hard upper bound can be set at compile-time using
263 the <a href="compile.html#max_mmap_size">SQLITE_MAX_MMAP_SIZE</a> macro. If <a href="compile.html#max_mmap_size">SQLITE_MAX_MMAP_SIZE</a> is set to
264 zero, then the code used to implement memory mapped I/O is omitted from
265 the build. The hard upper bound is automatically set to zero on certain
266 platforms (ex: OpenBSD) where memory mapped I/O does not work due to the
267 lack of a unified buffer cache.</p>
268
269 <p>If the hard upper bound on mmap_size is non-zero at compilation time,
270 it may still be reduced or zeroed at start-time using the
271 <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_covering_index_scan.html#sqliteconfigmmapsize">SQLITE_CONFIG_MMAP_SIZE</a>,X,Y) interface. The X and
272 Y parameters must both be 64-bit signed integers. The X parameter
273 is the default mmap_size of the process and the Y is the new hard upper bound.
274 The hard upper bound cannot be increased above its compile-time setting
275 using <a href="c3ref/c_config_covering_index_scan.html#sqliteconfigmmapsize">SQLITE_CONFIG_MMAP_SIZE</a> but it can be reduced or zeroed.</p>
276 <p align="center"><small><i>This page was last updated on 2022-04-18 02:55:50Z </small></i></p>
277