| 1 |
<!DOCTYPE html> |
| 2 |
<html><head> |
| 3 |
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| 4 |
<meta http-equiv="content-type" content="text/html; charset=UTF-8"> |
| 5 |
<link href="sqlite.css" rel="stylesheet"> |
| 6 |
<title>Pointer Passing Interfaces</title> |
| 7 |
<!-- path= --> |
| 8 |
</head> |
| 9 |
<body> |
| 10 |
<div class=nosearch> |
| 11 |
<a href="index.html"> |
| 12 |
<img class="logo" src="images/sqlite370_banner.svg" alt="SQLite" border="0"> |
| 13 |
</a> |
| 14 |
<div><!-- IE hack to prevent disappearing logo --></div> |
| 15 |
<div class="tagline desktoponly"> |
| 16 |
Small. Fast. Reliable.<br>Choose any three. |
| 17 |
</div> |
| 18 |
<div class="menu mainmenu"> |
| 19 |
<ul> |
| 20 |
<li><a href="index.html">Home</a> |
| 21 |
<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a> |
| 22 |
<li class='wideonly'><a href='about.html'>About</a> |
| 23 |
<li class='desktoponly'><a href="docs.html">Documentation</a> |
| 24 |
<li class='desktoponly'><a href="download.html">Download</a> |
| 25 |
<li class='wideonly'><a href='copyright.html'>License</a> |
| 26 |
<li class='desktoponly'><a href="support.html">Support</a> |
| 27 |
<li class='desktoponly'><a href="prosupport.html">Purchase</a> |
| 28 |
<li class='search' id='search_menubutton'> |
| 29 |
<a href="javascript:void(0)" onclick='toggle_search()'>Search</a> |
| 30 |
</ul> |
| 31 |
</div> |
| 32 |
<div class="menu submenu" id="submenu"> |
| 33 |
<ul> |
| 34 |
<li><a href='about.html'>About</a> |
| 35 |
<li><a href='docs.html'>Documentation</a> |
| 36 |
<li><a href='download.html'>Download</a> |
| 37 |
<li><a href='support.html'>Support</a> |
| 38 |
<li><a href='prosupport.html'>Purchase</a> |
| 39 |
</ul> |
| 40 |
</div> |
| 41 |
<div class="searchmenu" id="searchmenu"> |
| 42 |
<form method="GET" action="search"> |
| 43 |
<select name="s" id="searchtype"> |
| 44 |
<option value="d">Search Documentation</option> |
| 45 |
<option value="c">Search Changelog</option> |
| 46 |
</select> |
| 47 |
<input type="text" name="q" id="searchbox" value=""> |
| 48 |
<input type="submit" value="Go"> |
| 49 |
</form> |
| 50 |
</div> |
| 51 |
</div> |
| 52 |
<script> |
| 53 |
function toggle_div(nm) { |
| 54 |
var w = document.getElementById(nm); |
| 55 |
if( w.style.display=="block" ){ |
| 56 |
w.style.display = "none"; |
| 57 |
}else{ |
| 58 |
w.style.display = "block"; |
| 59 |
} |
| 60 |
} |
| 61 |
function toggle_search() { |
| 62 |
var w = document.getElementById("searchmenu"); |
| 63 |
if( w.style.display=="block" ){ |
| 64 |
w.style.display = "none"; |
| 65 |
} else { |
| 66 |
w.style.display = "block"; |
| 67 |
setTimeout(function(){ |
| 68 |
document.getElementById("searchbox").focus() |
| 69 |
}, 30); |
| 70 |
} |
| 71 |
} |
| 72 |
function div_off(nm){document.getElementById(nm).style.display="none";} |
| 73 |
window.onbeforeunload = function(e){div_off("submenu");} |
| 74 |
/* Disable the Search feature if we are not operating from CGI, since */ |
| 75 |
/* Search is accomplished using CGI and will not work without it. */ |
| 76 |
if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){ |
| 77 |
document.getElementById("search_menubutton").style.display = "none"; |
| 78 |
} |
| 79 |
/* Used by the Hide/Show button beside syntax diagrams, to toggle the */ |
| 80 |
function hideorshow(btn,obj){ |
| 81 |
var x = document.getElementById(obj); |
| 82 |
var b = document.getElementById(btn); |
| 83 |
if( x.style.display!='none' ){ |
| 84 |
x.style.display = 'none'; |
| 85 |
b.innerHTML='show'; |
| 86 |
}else{ |
| 87 |
x.style.display = ''; |
| 88 |
b.innerHTML='hide'; |
| 89 |
} |
| 90 |
return false; |
| 91 |
} |
| 92 |
var antiRobot = 0; |
| 93 |
function antiRobotGo(){ |
| 94 |
if( antiRobot!=3 ) return; |
| 95 |
antiRobot = 7; |
| 96 |
var j = document.getElementById("mtimelink"); |
| 97 |
if(j && j.hasAttribute("data-href")) j.href=j.getAttribute("data-href"); |
| 98 |
} |
| 99 |
function antiRobotDefense(){ |
| 100 |
document.body.onmousedown=function(){ |
| 101 |
antiRobot |= 2; |
| 102 |
antiRobotGo(); |
| 103 |
document.body.onmousedown=null; |
| 104 |
} |
| 105 |
document.body.onmousemove=function(){ |
| 106 |
antiRobot |= 2; |
| 107 |
antiRobotGo(); |
| 108 |
document.body.onmousemove=null; |
| 109 |
} |
| 110 |
setTimeout(function(){ |
| 111 |
antiRobot |= 1; |
| 112 |
antiRobotGo(); |
| 113 |
}, 100) |
| 114 |
antiRobotGo(); |
| 115 |
} |
| 116 |
antiRobotDefense(); |
| 117 |
</script> |
| 118 |
<div class=fancy> |
| 119 |
<div class=nosearch> |
| 120 |
<div class="fancy_title"> |
| 121 |
Pointer Passing Interfaces |
| 122 |
</div> |
| 123 |
<details class="fancy_toc"> |
| 124 |
<summary>Table Of Contents</summary> |
| 125 |
<div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div> |
| 126 |
<div class="fancy-toc1"><a href="#a_brief_history_of_pointer_passing_in_sqlite">2. A Brief History Of Pointer Passing In SQLite</a></div> |
| 127 |
<div class="fancy-toc2"><a href="#upping_the_threat_level">2.1. Upping The Threat Level</a></div> |
| 128 |
<div class="fancy-toc2"><a href="#preventing_forged_pointers">2.2. Preventing Forged Pointers</a></div> |
| 129 |
<div class="fancy-toc2"><a href="#pointer_leaks">2.3. Pointer Leaks</a></div> |
| 130 |
<div class="fancy-toc1"><a href="#the_new_pointer_passing_interfaces">3. The New Pointer-Passing Interfaces</a></div> |
| 131 |
<div class="fancy-toc2"><a href="#pointer_types">3.1. Pointer Types</a></div> |
| 132 |
<div class="fancy-toc3"><a href="#pointer_types_are_static_strings">3.1.1. Pointer types are static strings</a></div> |
| 133 |
<div class="fancy-toc2"><a href="#destructor_functions">3.2. Destructor Functions</a></div> |
| 134 |
<div class="fancy-toc1"><a href="#restrictions_on_the_use_of_pointer_values">4. Restrictions On The Use of Pointer Values</a></div> |
| 135 |
<div class="fancy-toc1"><a href="#summary">5. Summary</a></div> |
| 136 |
</div> |
| 137 |
</details> |
| 138 |
</div> |
| 139 |
|
| 140 |
|
| 141 |
|
| 142 |
|
| 143 |
|
| 144 |
<h1 id="overview"><span>1. </span>Overview</h1> |
| 145 |
|
| 146 |
<p> |
| 147 |
Three new "_pointer()" interfaces were added to SQLite 3.20.0 (2017-08-01): |
| 148 |
</p><ul> |
| 149 |
<li> <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a>, |
| 150 |
</li><li> <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a>, and |
| 151 |
</li><li> <a href="c3ref/value_blob.html">sqlite3_value_pointer()</a>. |
| 152 |
</li></ul> |
| 153 |
|
| 154 |
<p>Questions and confusion quickly arose |
| 155 |
on the <a href="support.html#mailinglists">mailing lists</a> about the purpose behind these new interfaces, |
| 156 |
why they were introduced, and what problem they solve. This essay attempts |
| 157 |
to answer those questions and clear up the confusion. |
| 158 |
|
| 159 |
</p><h1 id="a_brief_history_of_pointer_passing_in_sqlite"><span>2. </span>A Brief History Of Pointer Passing In SQLite</h1> |
| 160 |
|
| 161 |
<p> |
| 162 |
It is sometimes convenient for SQLite extensions to |
| 163 |
communicate non-SQL values between subcomponents or between the extension |
| 164 |
and the application. Some examples: |
| 165 |
|
| 166 |
</p><ul> |
| 167 |
<li><p> |
| 168 |
In the <a href="fts3.html">FTS3</a> extension, the |
| 169 |
<a href="fts3.html#full_text_index_queries">MATCH operator</a> (which does the full-text search) |
| 170 |
needs to communicate details of matching entries |
| 171 |
to the <a href="fts3.html#snippet">snippet()</a>, <a href="fts3.html#offsets">offsets()</a>, |
| 172 |
and <a href="fts3.html#matchinfo">matchinfo()</a> functions so that those functions can convert the details |
| 173 |
of the match into useful output. |
| 174 |
|
| 175 |
</p></li><li><p> |
| 176 |
In order for an application to |
| 177 |
<a href="fts5.html#extending_fts5">add new extensions to FTS5</a>, such as new tokenizers, |
| 178 |
the application needs a pointer to the "fts5_api" object. |
| 179 |
|
| 180 |
</p></li><li><p> |
| 181 |
In the <a href="carray.html">CARRAY extension</a>, the application needs to tell the |
| 182 |
extension the |
| 183 |
location of a C-language array that contains the data for the table-valued |
| 184 |
function that the extension implements. |
| 185 |
</p></li></ul> |
| 186 |
|
| 187 |
<p> |
| 188 |
The traditional way of communicating this information was to transform a |
| 189 |
C-language pointer into a BLOB or a 64-bit integer, then move that BLOB or |
| 190 |
integer through SQLite using the usual interfaces like |
| 191 |
<a href="c3ref/bind_blob.html">sqlite3_bind_blob()</a>, <a href="c3ref/result_blob.html">sqlite3_result_blob()</a>, <a href="c3ref/value_blob.html">sqlite3_value_blob()</a> or |
| 192 |
the integer equivalents. |
| 193 |
|
| 194 |
</p><h2 id="upping_the_threat_level"><span>2.1. </span>Upping The Threat Level</h2> |
| 195 |
|
| 196 |
<p> |
| 197 |
Passing around pointers as if they were integers or BLOBs is easy, |
| 198 |
effective, and works well in an environment where the application |
| 199 |
components are all friendly toward one another. However, passing pointers |
| 200 |
as integers and BLOBs allows hostile SQL text to forge invalid pointers that |
| 201 |
can carry out mischief. |
| 202 |
|
| 203 |
</p><p> |
| 204 |
For example, the first argument to the <a href="fts3.html#snippet">snippet()</a> function is supposed to |
| 205 |
be a special column of the FTS3 table that contains a pointer to an fts3cursor |
| 206 |
object that contains information about the current full text search match. |
| 207 |
That pointer was formerly passed as a BLOB. |
| 208 |
For example, if the FTS3 table is named "t1" and has a column named "cx", |
| 209 |
one might write: |
| 210 |
|
| 211 |
</p><div class="codeblock"><pre>SELECT snippet(t1) FROM t1 WHERE cx MATCH $pattern; |
| 212 |
</pre></div> |
| 213 |
|
| 214 |
<p> |
| 215 |
But if a hacker is able to run arbitrary SQL, he might run a slightly |
| 216 |
different query, like this: |
| 217 |
|
| 218 |
</p><div class="codeblock"><pre>SELECT hex(t1) FROM t1 WHERE cx MATCH $pattern; |
| 219 |
</pre></div> |
| 220 |
|
| 221 |
<p> |
| 222 |
Because the pointer is passed in the t1 column of the t1 |
| 223 |
table as a BLOB (in older versions of SQLite), such a query would have |
| 224 |
shown the value of the |
| 225 |
pointer in hex. The attacker could then modify that pointer to try to |
| 226 |
get the snippet() function to modify memory in some other part of |
| 227 |
the application address space instead of the fts3cursor object it |
| 228 |
was supposed to be operating on: |
| 229 |
|
| 230 |
</p><div class="codeblock"><pre>SELECT snippet(x'6092310100000000') FROM t1 WHERE cx MATCH $pattern; |
| 231 |
</pre></div> |
| 232 |
|
| 233 |
<p> |
| 234 |
Historically, this was not considered a threat. The argument was that if |
| 235 |
a hostile agent is able to inject arbitrary SQL text into the application, |
| 236 |
then that agent is already in full control of the application, so |
| 237 |
letting the hostile agent forge a pointer does not give the agent |
| 238 |
any new capability. |
| 239 |
|
| 240 |
</p><p> |
| 241 |
For most cases, it is true that potential attackers have no way of injecting |
| 242 |
arbitrary SQL, and so most uses of SQLite are immune to the attack above. |
| 243 |
But there are some notable exceptions. To wit: |
| 244 |
|
| 245 |
</p><ul> |
| 246 |
<li><p> |
| 247 |
The <a href="https://en.wikipedia.org/wiki/Web_SQL_Database">WebSQL</a> interface |
| 248 |
to webkit allowed any webpage to run arbitrary SQL in the browser |
| 249 |
for Chrome and Safari. That arbitrary SQL was supposed to be run inside |
| 250 |
a sandbox where it could do no harm even if exploited, but that sandbox |
| 251 |
turned out to be less secure than people supposed. In the spring of 2017, |
| 252 |
one team of hackers was able to root an iMac using a long sequence of |
| 253 |
exploits, one of which involved corrupting the pointers passed as BLOB |
| 254 |
values to the snippet() FTS3 function of an SQLite database running via |
| 255 |
the WebSQL interface inside of Safari. |
| 256 |
|
| 257 |
</p></li><li><p> |
| 258 |
On Android, we are told, there are many services that will blindly |
| 259 |
run arbitrary SQL that is passed to them by untrustworthy apps |
| 260 |
that have been downloaded from dodgy corners of the internet. |
| 261 |
Android services are suppose to be more guarded about running SQL |
| 262 |
from unvetted sources. This author does not have any specific examples |
| 263 |
to the contrary, but he has heard rumors that they exist. Even if |
| 264 |
all Android services are more careful and properly vet all the SQL |
| 265 |
they run, it would be difficult to audit them |
| 266 |
all in order to verify that they are safe. Hence, security-minded people |
| 267 |
are keen to ensure that no exploits are possible by passing arbitrary |
| 268 |
SQL text. |
| 269 |
|
| 270 |
</p></li><li><p> |
| 271 |
The <a href="https://www.fossil-scm.org/">Fossil</a> version control system (designed |
| 272 |
and written for the purpose of supporting SQLite development) allows |
| 273 |
mildly trusted users to enter arbitrary SQL for generating trouble-ticket |
| 274 |
reports. That SQL is sanitized using the |
| 275 |
<a href="c3ref/set_authorizer.html">sqlite3_set_authorizer()</a> interface, and no exploits have ever been |
| 276 |
found. But this is an example of potentially hostile agents being able to |
| 277 |
inject arbitrary SQL into the system. |
| 278 |
</p></li></ul> |
| 279 |
|
| 280 |
<h2 id="preventing_forged_pointers"><span>2.2. </span>Preventing Forged Pointers</h2> |
| 281 |
|
| 282 |
<p> |
| 283 |
The first attempt at closing security gaps in pointer passing was to |
| 284 |
prevent pointer values from being forged. This was accomplished by |
| 285 |
having the sender attach a subtype to each pointer using |
| 286 |
<a href="c3ref/result_subtype.html">sqlite3_result_subtype()</a> and having the receiver verify that subtype |
| 287 |
using <a href="c3ref/value_subtype.html">sqlite3_value_subtype()</a> and reject pointers that had an incorrect |
| 288 |
subtype. Since there is no way to attach a subtype to a result using |
| 289 |
pure SQL, this prevents pointers from being forged using SQL. The only |
| 290 |
way to send a pointer is to use C code. If an attacker can set a subtype, |
| 291 |
then he is also able to forge a pointer without the help of SQLite. |
| 292 |
|
| 293 |
</p><p> |
| 294 |
Using subtypes to identify valid pointers prevented the WebSQL exploit. |
| 295 |
But it turned out to be an incomplete solution. |
| 296 |
|
| 297 |
<a name="ptrleak"></a> |
| 298 |
|
| 299 |
</p><h2 id="pointer_leaks"><span>2.3. </span>Pointer Leaks</h2> |
| 300 |
|
| 301 |
<p> |
| 302 |
The use of subtypes on pointers prevented pointer forgery using |
| 303 |
pure SQL. But subtypes do nothing to prevent an attacker from reading |
| 304 |
the values of pointers. In other words, subtypes on pointer values |
| 305 |
prevent attacks using SQL statements like this: |
| 306 |
|
| 307 |
</p><div class="codeblock"><pre>SELECT snippet(x'6092310100000000') FROM t1 WHERE cx MATCH $pattern; |
| 308 |
</pre></div> |
| 309 |
|
| 310 |
<p> |
| 311 |
The BLOB argument to snippet() does not have the correct subtype, so the |
| 312 |
snippet function ignores it, makes no changes to any data structures, |
| 313 |
and harmlessly returns NULL. |
| 314 |
|
| 315 |
</p><p> |
| 316 |
But the use of subtypes does nothing to prevent the value of a |
| 317 |
pointer from being read using SQL code like this: |
| 318 |
|
| 319 |
</p><div class="codeblock"><pre>SELECT hex(t1) FROM t1 WHERE cx MATCH $pattern; |
| 320 |
</pre></div> |
| 321 |
|
| 322 |
<p> |
| 323 |
What harm could come of that, you ask? The SQLite developers (including |
| 324 |
this author) wondered the same thing. But then security researchers |
| 325 |
pointed out that knowledge of pointers can help attackers to circumvent |
| 326 |
address-space randomization defenses. This is called a "pointer leak". |
| 327 |
A pointer leak is not itself a vulnerability, but it can aid an attacker |
| 328 |
in effectively exploiting other vulnerabilities. |
| 329 |
|
| 330 |
</p><h1 id="the_new_pointer_passing_interfaces"><span>3. </span>The New Pointer-Passing Interfaces</h1> |
| 331 |
|
| 332 |
<p> |
| 333 |
Allowing extension components to pass private information to one another |
| 334 |
securely and without introducing pointer leaks requires new interfaces: |
| 335 |
|
| 336 |
</p><ul> |
| 337 |
<li><b><a href="c3ref/bind_blob.html">sqlite3_bind_pointer</a>(S,I,P,T,D)</b> → |
| 338 |
Bind pointer P of type T to the I-th parameter of prepared statement S. |
| 339 |
D is an optional destructor function for P. |
| 340 |
</li><li><b><a href="c3ref/result_blob.html">sqlite3_result_pointer</a>(C,P,T,D)</b> → |
| 341 |
Return pointer P of type T as the argument of function C. |
| 342 |
D is an optional destructor function for P. |
| 343 |
</li><li><b><a href="c3ref/value_blob.html">sqlite3_value_pointer</a>(V,T)</b> → |
| 344 |
Return the pointer of type T associated with value V, or if V has no |
| 345 |
associated pointer, or if the pointer on V is of a type different from |
| 346 |
T, then return NULL. |
| 347 |
</li></ul> |
| 348 |
|
| 349 |
<p> |
| 350 |
To SQL, the values created by <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a> and |
| 351 |
<a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> are indistinguishable from NULL. An |
| 352 |
SQL statement that tries to use the <a href="lang_corefunc.html#hex">hex()</a> function to read the |
| 353 |
value of a pointer will get an SQL NULL answer. The only way to |
| 354 |
discover whether or not a value has an associated pointer is to |
| 355 |
use the <a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> interface with the appropriate |
| 356 |
type string T. |
| 357 |
|
| 358 |
</p><p> |
| 359 |
Pointer values read by <a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> |
| 360 |
cannot be generated by pure SQL. Hence, it is not possible for SQL to |
| 361 |
forge pointers. |
| 362 |
|
| 363 |
</p><p> |
| 364 |
Pointer values generated by <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a> and |
| 365 |
<a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> cannot be read by pure SQL. |
| 366 |
Hence, it is not possible for SQL to leak the value of pointers. |
| 367 |
|
| 368 |
</p><p> |
| 369 |
In this way the new pointer-passing interface seems to solve all of the |
| 370 |
security problems associated with passing pointer values from one |
| 371 |
extension to another in SQLite. |
| 372 |
|
| 373 |
<a name="ptrtyp"></a> |
| 374 |
|
| 375 |
</p><h2 id="pointer_types"><span>3.1. </span>Pointer Types</h2> |
| 376 |
|
| 377 |
<p> |
| 378 |
The "pointer type" in the last parameter to <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a>, |
| 379 |
<a href="c3ref/result_blob.html">sqlite3_result_pointer()</a>, and <a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> is used to prevent |
| 380 |
pointers intended for one extension from being redirected to a different |
| 381 |
extension. For example, without the use of pointer types, an attacker |
| 382 |
could still get access to pointer information in a system that included |
| 383 |
both the <a href="fts3.html">FTS3</a> and the <a href="carray.html">CARRAY extension</a> using SQL like this: |
| 384 |
|
| 385 |
</p><div class="codeblock"><pre>SELECT ca.value FROM t1, carray(t1,10) AS ca WHERE cx MATCH $pattern |
| 386 |
</pre></div> |
| 387 |
|
| 388 |
<p> |
| 389 |
In the statement above, the FTS3 cursor pointer generated by the |
| 390 |
MATCH operator is sent into the carray() table-valued function instead |
| 391 |
of its intended recipient snippet(). The carray() function treats the |
| 392 |
pointer as a pointer to an array of integers and returns each integer |
| 393 |
one by one, thus leaking the content of the FTS3 cursor object. Since |
| 394 |
the FTS3 cursor object contains pointers to other objects, the statement |
| 395 |
above would be a pointer leak. |
| 396 |
|
| 397 |
</p><p> |
| 398 |
Except, the statement above does not work, thanks to pointer types. |
| 399 |
The pointer generated by the MATCH operator has a type of "fts3cursor" |
| 400 |
but the carray() function expects to receives a pointer of type "carray". |
| 401 |
Because the pointer type on the <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> does not match |
| 402 |
the pointer type on the <a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> call, |
| 403 |
<a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> returns NULL in carray() and thus signals |
| 404 |
the CARRAY extension that it has been passed an invalid pointer. |
| 405 |
|
| 406 |
</p><h3 id="pointer_types_are_static_strings"><span>3.1.1. </span>Pointer types are static strings</h3> |
| 407 |
|
| 408 |
<p> |
| 409 |
Pointer types are static strings, which ideally should be string literals |
| 410 |
embedded directly in the SQLite API call, not parameters passed in from |
| 411 |
other functions. Consideration was given to using integer values as |
| 412 |
the pointer type, but static strings provides a much larger name space |
| 413 |
which reduces the chance of accidental type-name collisions between |
| 414 |
unrelated extensions. |
| 415 |
|
| 416 |
</p><p> |
| 417 |
By "static string", we mean a zero-terminated array of bytes that is |
| 418 |
fixed and unchanging for the life of the program. In other words, the |
| 419 |
pointer type string should be a string constant. |
| 420 |
In contrast, a "dynamic string" is a zero-terminated array of bytes |
| 421 |
that is held in memory allocated |
| 422 |
from the heap, and which must be freed to avoid a memory leak. |
| 423 |
Do not use dynamic strings as the pointer type string. |
| 424 |
|
| 425 |
</p><p> |
| 426 |
Multiple commentators have expressed a desire to use dynamic strings |
| 427 |
for the pointer type, and to have SQLite take ownership of the type strings |
| 428 |
and to automatically free the type string |
| 429 |
when it has finished using it. That design is rejected for the |
| 430 |
following reasons: |
| 431 |
|
| 432 |
</p><ol> |
| 433 |
<li><p> |
| 434 |
The pointer type is not intended to be flexible and dynamic. The |
| 435 |
pointer type is intended to be a design-time constant. Applications |
| 436 |
should not synthesize pointer type strings at run-time. Providing |
| 437 |
support for dynamic pointer type strings would lead developers |
| 438 |
to misuse the pointer-passing interfaces by creating run-time |
| 439 |
synthesized pointer type strings. Requiring the pointer type strings |
| 440 |
to be static encourages developers to do the right thing by choosing |
| 441 |
fixed pointer type names at design-time and encoding those names |
| 442 |
as constant strings. |
| 443 |
|
| 444 |
</p></li><li><p> |
| 445 |
All string values at the SQL level in SQLite are dynamic strings. |
| 446 |
Requiring type strings to be static makes it difficult to |
| 447 |
create an application-defined SQL function that |
| 448 |
can synthesize a pointer of an arbitrary type. We do not want users |
| 449 |
to create such SQL functions, since such functions would compromise the |
| 450 |
security of the system. Thus, the requirement to use static strings |
| 451 |
helps to defend the integrity of the pointer-passing interfaces against |
| 452 |
ill-designed SQL functions. The static string requirement is not |
| 453 |
a perfect defense, since a sophisticated programmer can code around |
| 454 |
it, and a novice programmer can simply take the memory leak. But by |
| 455 |
stating that the pointer type string must be static, we hope to encourage |
| 456 |
developers who might otherwise use a dynamic string for the pointer type |
| 457 |
to think more carefully about the problem and avoid introducing |
| 458 |
security issues. |
| 459 |
|
| 460 |
</p></li><li><p> |
| 461 |
Having SQLite take ownership of the type strings would impose a performance |
| 462 |
cost on all applications, even applications that do not use the |
| 463 |
pointer-passing interfaces. SQLite passes values around as instances |
| 464 |
of the <a href="c3ref/value.html">sqlite3_value</a> object. That object has a destructor, which because |
| 465 |
of the fact that sqlite3_value objects are used for nearly everything, is |
| 466 |
invoked frequently. If the destructor needs to check to see if there is |
| 467 |
a pointer type string that needs to be freed, that is a few extra CPU |
| 468 |
cycles that need to be burned on each call to the destructor. Those |
| 469 |
cycles add up. We would be willing to bear the cost of the extra CPU |
| 470 |
cycles if pointer-passing was a commonly used programming paradigm, but |
| 471 |
pointer-passing is rare, and so it seems unwise to impose a run-time cost |
| 472 |
on billions and billions of applications that do not use pointer passing |
| 473 |
just for convenience of a few applications that do. |
| 474 |
</p></li></ol> |
| 475 |
|
| 476 |
<p> |
| 477 |
If you feel that you need dynamic pointer type strings in your application, |
| 478 |
that is a strong indicator that you are misusing the pointer-passing interface. |
| 479 |
Your intended use may be unsafe. |
| 480 |
Please rethink your design. Determine if you really need to be passing |
| 481 |
pointers through SQL in the first place. Or perhaps find a different |
| 482 |
mechanism other than the pointer-passing interfaces described by this |
| 483 |
article. |
| 484 |
|
| 485 |
</p><h2 id="destructor_functions"><span>3.2. </span>Destructor Functions</h2> |
| 486 |
|
| 487 |
<p> |
| 488 |
The last parameter to the <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a> and |
| 489 |
<a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> routines is a pointer to a procedure |
| 490 |
used to dispose of the P pointer once SQLite has finished with it. |
| 491 |
This pointer can be NULL, in which case no destructor is called. |
| 492 |
|
| 493 |
</p><p> |
| 494 |
When the D parameter is not NULL, that means that ownership of the |
| 495 |
pointer is being transferred to SQLite. SQLite will take responsibility |
| 496 |
for freeing resources associated with the pointer when it has finished |
| 497 |
using the pointer. If the D parameter is NULL, that means that ownership |
| 498 |
of the pointer remains with the caller and the caller is responsible for |
| 499 |
disposing of the pointer. |
| 500 |
|
| 501 |
</p><p> |
| 502 |
Note that the destructor function D is for the pointer value P, not for |
| 503 |
the type string T. The type string T should be a static string with an |
| 504 |
infinite lifetime. |
| 505 |
|
| 506 |
</p><p> |
| 507 |
If ownership of the pointer is passed into SQLite by providing a |
| 508 |
non-NULL D parameter to <a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a> or <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> |
| 509 |
then the ownership remains with SQLite until the object is destroyed. |
| 510 |
There is no way to transfer ownership out of SQLite and back into the |
| 511 |
application again. |
| 512 |
|
| 513 |
</p><h1 id="restrictions_on_the_use_of_pointer_values"><span>4. </span>Restrictions On The Use of Pointer Values</h1> |
| 514 |
|
| 515 |
<p> |
| 516 |
The pointers that piggy-back on SQL NULL values using the |
| 517 |
<a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a>, <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a>, and |
| 518 |
<a href="c3ref/value_blob.html">sqlite3_value_pointer()</a> interface are transient and ephemeral. |
| 519 |
The pointers are never written into the database. The pointers |
| 520 |
will not survive sorting. The latter fact is why there is no |
| 521 |
sqlite3_column_pointer() interface, since it is impossible to |
| 522 |
predict whether or not the query planner will insert a sort operation |
| 523 |
prior to returning a value from a query, so it would be impossible to |
| 524 |
know if a pointer value inserted into a query by |
| 525 |
<a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a> or <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a> would survive |
| 526 |
through to the result set. |
| 527 |
|
| 528 |
</p><p> |
| 529 |
Pointer values must flow directly from their producer into their |
| 530 |
consumer, with no intermediate operators or functions. Any transformation |
| 531 |
of a pointer value destroys the pointer and transforms the value into |
| 532 |
an ordinary SQL NULL. |
| 533 |
|
| 534 |
</p><h1 id="summary"><span>5. </span>Summary</h1> |
| 535 |
|
| 536 |
<p>Key take-aways from this essay: |
| 537 |
|
| 538 |
</p><ol> |
| 539 |
<li><p> |
| 540 |
The internet is an increasingly hostile place. These days, developers |
| 541 |
should assume that attackers will find a way to execute arbitrary SQL |
| 542 |
in an application. |
| 543 |
Applications should be designed to prevent the execution of arbitrary |
| 544 |
SQL from escalating into a more severe exploit. |
| 545 |
|
| 546 |
</p></li><li><p> |
| 547 |
A few SQLite extensions benefit from passing pointers: |
| 548 |
</p><ul> |
| 549 |
<li>The <a href="fts3.html">FTS3</a> MATCH operator passes pointers into <a href="fts3.html#snippet">snippet()</a>, |
| 550 |
<a href="fts3.html#offsets">offsets()</a>, and <a href="fts3.html#matchinfo">matchinfo()</a>. |
| 551 |
</li><li>The <a href="carray.html">carray table-valued function</a> needs to accept a pointer to |
| 552 |
an array of C-language values from the application. |
| 553 |
</li><li>The <a href="https://sqlite.org/src/file/ext/misc/remember.c">remember() extension</a> |
| 554 |
needs a pointer to a C-language integer variable in which to remember |
| 555 |
the value it passes. |
| 556 |
</li><li>Applications need to receive a pointer to the "fts5_api" object in order |
| 557 |
to add extensions, such as custom tokenizers, to the <a href="fts5.html">FTS5</a> extension. |
| 558 |
</li></ul> |
| 559 |
|
| 560 |
</li><li><p> |
| 561 |
Pointers should <u>never</u> be exchanged by encoding them as some other |
| 562 |
SQL datatype, such as integers or BLOBs. Instead, use the interfaces |
| 563 |
designed to facilitate secure pointer passing: |
| 564 |
<a href="c3ref/bind_blob.html">sqlite3_bind_pointer()</a>, <a href="c3ref/result_blob.html">sqlite3_result_pointer()</a>, and |
| 565 |
<a href="c3ref/value_blob.html">sqlite3_value_pointer()</a>. |
| 566 |
|
| 567 |
</p></li><li><p> |
| 568 |
The use of pointer-passing is an advanced technique that should be used |
| 569 |
infrequently and cautiously. Pointer-passing should not be |
| 570 |
used haphazardly or carelessly. Pointer-passing is a sharp tool |
| 571 |
that can leave deep scars if misused. |
| 572 |
|
| 573 |
</p></li><li><p> |
| 574 |
The "pointer type" string which is the last parameter to each of the |
| 575 |
pointer-passing interfaces should be a distinct, application-specific |
| 576 |
string literal that appears directly in the API call. The pointer type |
| 577 |
should not be a parameter passed in from a higher-level function. |
| 578 |
</p></li></ol> |
| 579 |
<p align="center"><small><i>This page was last updated on 2025-05-31 13:08:22Z </small></i></p> |
| 580 |
|