| 1 |
<!-- doc/src/sgml/jit.sgml --> |
| 2 |
|
| 3 |
<chapter id="jit"> |
| 4 |
<title>Just-in-Time Compilation (<acronym>JIT</acronym>)</title> |
| 5 |
|
| 6 |
<indexterm zone="jit"> |
| 7 |
<primary><acronym>JIT</acronym></primary> |
| 8 |
</indexterm> |
| 9 |
|
| 10 |
<indexterm> |
| 11 |
<primary>Just-In-Time compilation</primary> |
| 12 |
<see><acronym>JIT</acronym></see> |
| 13 |
</indexterm> |
| 14 |
|
| 15 |
<para> |
| 16 |
This chapter explains what just-in-time compilation is, and how it can be |
| 17 |
configured in <productname>PostgreSQL</productname>. |
| 18 |
</para> |
| 19 |
|
| 20 |
<sect1 id="jit-reason"> |
| 21 |
<title>What Is <acronym>JIT</acronym> compilation?</title> |
| 22 |
|
| 23 |
<para> |
| 24 |
Just-in-Time (<acronym>JIT</acronym>) compilation is the process of turning |
| 25 |
some form of interpreted program evaluation into a native program, and |
| 26 |
doing so at run time. |
| 27 |
For example, instead of using general-purpose code that can evaluate |
| 28 |
arbitrary SQL expressions to evaluate a particular SQL predicate |
| 29 |
like <literal>WHERE a.col = 3</literal>, it is possible to generate a |
| 30 |
function that is specific to that expression and can be natively executed |
| 31 |
by the CPU, yielding a speedup. |
| 32 |
</para> |
| 33 |
|
| 34 |
<para> |
| 35 |
<productname>PostgreSQL</productname> has builtin support to perform |
| 36 |
<acronym>JIT</acronym> compilation using <ulink |
| 37 |
url="https://llvm.org/"><productname>LLVM</productname></ulink> when |
| 38 |
<productname>PostgreSQL</productname> is built with |
| 39 |
<link linkend="configure-with-llvm"><literal>--with-llvm</literal></link>. |
| 40 |
</para> |
| 41 |
|
| 42 |
<para> |
| 43 |
See <filename>src/backend/jit/README</filename> for further details. |
| 44 |
</para> |
| 45 |
|
| 46 |
<sect2 id="jit-accelerated-operations"> |
| 47 |
<title><acronym>JIT</acronym> Accelerated Operations</title> |
| 48 |
<para> |
| 49 |
Currently <productname>PostgreSQL</productname>'s <acronym>JIT</acronym> |
| 50 |
implementation has support for accelerating expression evaluation and |
| 51 |
tuple deforming. Several other operations could be accelerated in the |
| 52 |
future. |
| 53 |
</para> |
| 54 |
<para> |
| 55 |
Expression evaluation is used to evaluate <literal>WHERE</literal> |
| 56 |
clauses, target lists, aggregates and projections. It can be accelerated |
| 57 |
by generating code specific to each case. |
| 58 |
</para> |
| 59 |
<para> |
| 60 |
Tuple deforming is the process of transforming an on-disk tuple (see <xref |
| 61 |
linkend="storage-tuple-layout"/>) into its in-memory representation. |
| 62 |
It can be accelerated by creating a function specific to the table layout |
| 63 |
and the number of columns to be extracted. |
| 64 |
</para> |
| 65 |
</sect2> |
| 66 |
|
| 67 |
<sect2 id="jit-inlining"> |
| 68 |
<title>Inlining</title> |
| 69 |
<para> |
| 70 |
<productname>PostgreSQL</productname> is very extensible and allows new |
| 71 |
data types, functions, operators and other database objects to be defined; |
| 72 |
see <xref linkend="extend"/>. In fact the built-in objects are implemented |
| 73 |
using nearly the same mechanisms. This extensibility implies some |
| 74 |
overhead, for example due to function calls (see <xref linkend="xfunc"/>). |
| 75 |
To reduce that overhead, <acronym>JIT</acronym> compilation can inline the |
| 76 |
bodies of small functions into the expressions using them. That allows a |
| 77 |
significant percentage of the overhead to be optimized away. |
| 78 |
</para> |
| 79 |
</sect2> |
| 80 |
|
| 81 |
<sect2 id="jit-optimization"> |
| 82 |
<title>Optimization</title> |
| 83 |
<para> |
| 84 |
<productname>LLVM</productname> has support for optimizing generated |
| 85 |
code. Some of the optimizations are cheap enough to be performed whenever |
| 86 |
<acronym>JIT</acronym> is used, while others are only beneficial for |
| 87 |
longer-running queries. |
| 88 |
See <ulink url="https://llvm.org/docs/Passes.html#transform-passes"/> for |
| 89 |
more details about optimizations. |
| 90 |
</para> |
| 91 |
</sect2> |
| 92 |
|
| 93 |
</sect1> |
| 94 |
|
| 95 |
<sect1 id="jit-decision"> |
| 96 |
<title>When to <acronym>JIT</acronym>?</title> |
| 97 |
|
| 98 |
<para> |
| 99 |
<acronym>JIT</acronym> compilation is beneficial primarily for long-running |
| 100 |
CPU-bound queries. Frequently these will be analytical queries. For short |
| 101 |
queries the added overhead of performing <acronym>JIT</acronym> compilation |
| 102 |
will often be higher than the time it can save. |
| 103 |
</para> |
| 104 |
|
| 105 |
<para> |
| 106 |
To determine whether <acronym>JIT</acronym> compilation should be used, |
| 107 |
the total estimated cost of a query (see |
| 108 |
<xref linkend="planner-stats-details"/> and |
| 109 |
<xref linkend="runtime-config-query-constants"/>) is used. |
| 110 |
The estimated cost of the query will be compared with the setting of <xref |
| 111 |
linkend="guc-jit-above-cost"/>. If the cost is higher, |
| 112 |
<acronym>JIT</acronym> compilation will be performed. |
| 113 |
Two further decisions are then needed. |
| 114 |
Firstly, if the estimated cost is more |
| 115 |
than the setting of <xref linkend="guc-jit-inline-above-cost"/>, short |
| 116 |
functions and operators used in the query will be inlined. |
| 117 |
Secondly, if the estimated cost is more than the setting of <xref |
| 118 |
linkend="guc-jit-optimize-above-cost"/>, expensive optimizations are |
| 119 |
applied to improve the generated code. |
| 120 |
Each of these options increases the <acronym>JIT</acronym> compilation |
| 121 |
overhead, but can reduce query execution time considerably. |
| 122 |
</para> |
| 123 |
|
| 124 |
<para> |
| 125 |
These cost-based decisions will be made at plan time, not execution |
| 126 |
time. This means that when prepared statements are in use, and a generic |
| 127 |
plan is used (see <xref linkend="sql-prepare"/>), the values of the |
| 128 |
configuration parameters in effect at prepare time control the decisions, |
| 129 |
not the settings at execution time. |
| 130 |
</para> |
| 131 |
|
| 132 |
<note> |
| 133 |
<para> |
| 134 |
If <xref linkend="guc-jit"/> is set to <literal>off</literal>, or if no |
| 135 |
<acronym>JIT</acronym> implementation is available (for example because |
| 136 |
the server was compiled without <literal>--with-llvm</literal>), |
| 137 |
<acronym>JIT</acronym> will not be performed, even if it would be |
| 138 |
beneficial based on the above criteria. Setting <xref linkend="guc-jit"/> |
| 139 |
to <literal>off</literal> has effects at both plan and execution time. |
| 140 |
</para> |
| 141 |
</note> |
| 142 |
|
| 143 |
<para> |
| 144 |
<xref linkend="sql-explain"/> can be used to see whether |
| 145 |
<acronym>JIT</acronym> is used or not. As an example, here is a query that |
| 146 |
is not using <acronym>JIT</acronym>: |
| 147 |
<screen> |
| 148 |
=# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class; |
| 149 |
QUERY PLAN |
| 150 |
-------------------------------------------------------------------&zwsp;------------------------------------------ |
| 151 |
Aggregate (cost=16.27..16.29 rows=1 width=8) (actual time=0.303..0.303 rows=1.00 loops=1) |
| 152 |
Buffers: shared hit=14 |
| 153 |
-> Seq Scan on pg_class (cost=0.00..15.42 rows=342 width=4) (actual time=0.017..0.111 rows=356.00 loops=1) |
| 154 |
Buffers: shared hit=14 |
| 155 |
Planning Time: 0.116 ms |
| 156 |
Execution Time: 0.365 ms |
| 157 |
</screen> |
| 158 |
Given the cost of the plan, it is entirely reasonable that no |
| 159 |
<acronym>JIT</acronym> was used; the cost of <acronym>JIT</acronym> would |
| 160 |
have been bigger than the potential savings. Adjusting the cost limits |
| 161 |
will lead to <acronym>JIT</acronym> use: |
| 162 |
<screen> |
| 163 |
=# SET jit_above_cost = 10; |
| 164 |
SET |
| 165 |
=# EXPLAIN ANALYZE SELECT SUM(relpages) FROM pg_class; |
| 166 |
QUERY PLAN |
| 167 |
-------------------------------------------------------------------&zwsp;------------------------------------------ |
| 168 |
Aggregate (cost=16.27..16.29 rows=1 width=8) (actual time=6.049..6.049 rows=1.00 loops=1) |
| 169 |
Buffers: shared hit=14 |
| 170 |
-> Seq Scan on pg_class (cost=0.00..15.42 rows=342 width=4) (actual time=0.019..0.052 rows=356.00 loops=1) |
| 171 |
Buffers: shared hit=14 |
| 172 |
Planning Time: 0.133 ms |
| 173 |
JIT: |
| 174 |
Functions: 3 |
| 175 |
Options: Inlining false, Optimization false, Expressions true, Deforming true |
| 176 |
Timing: Generation 1.259 ms (Deform 0.000 ms), Inlining 0.000 ms, Optimization 0.797 ms, Emission 5.048 ms, Total 7.104 ms |
| 177 |
Execution Time: 7.416 ms |
| 178 |
</screen> |
| 179 |
As visible here, <acronym>JIT</acronym> was used, but inlining and |
| 180 |
expensive optimization were not. If <xref |
| 181 |
linkend="guc-jit-inline-above-cost"/> or <xref |
| 182 |
linkend="guc-jit-optimize-above-cost"/> were also lowered, |
| 183 |
that would change. |
| 184 |
</para> |
| 185 |
</sect1> |
| 186 |
|
| 187 |
<sect1 id="jit-configuration" xreflabel="JIT Configuration"> |
| 188 |
<title>Configuration</title> |
| 189 |
|
| 190 |
<para> |
| 191 |
The configuration variable |
| 192 |
<xref linkend="guc-jit"/> determines whether <acronym>JIT</acronym> |
| 193 |
compilation is enabled or disabled. |
| 194 |
If it is enabled, the configuration variables |
| 195 |
<xref linkend="guc-jit-above-cost"/>, <xref |
| 196 |
linkend="guc-jit-inline-above-cost"/>, and <xref |
| 197 |
linkend="guc-jit-optimize-above-cost"/> determine |
| 198 |
whether <acronym>JIT</acronym> compilation is performed for a query, |
| 199 |
and how much effort is spent doing so. |
| 200 |
</para> |
| 201 |
|
| 202 |
<para> |
| 203 |
<xref linkend="guc-jit-provider"/> determines which <acronym>JIT</acronym> |
| 204 |
implementation is used. It is rarely required to be changed. See <xref |
| 205 |
linkend="jit-pluggable"/>. |
| 206 |
</para> |
| 207 |
|
| 208 |
<para> |
| 209 |
For development and debugging purposes a few additional configuration |
| 210 |
parameters exist, as described in |
| 211 |
<xref linkend="runtime-config-developer"/>. |
| 212 |
</para> |
| 213 |
</sect1> |
| 214 |
|
| 215 |
<sect1 id="jit-extensibility"> |
| 216 |
<title>Extensibility</title> |
| 217 |
|
| 218 |
<sect2 id="jit-extensibility-bitcode"> |
| 219 |
<title>Inlining Support for Extensions</title> |
| 220 |
<para> |
| 221 |
<productname>PostgreSQL</productname>'s <acronym>JIT</acronym> |
| 222 |
implementation can inline the bodies of functions |
| 223 |
of types <literal>C</literal> and <literal>internal</literal>, as well as |
| 224 |
operators based on such functions. To do so for functions in extensions, |
| 225 |
the definitions of those functions need to be made available. |
| 226 |
When using <link linkend="extend-pgxs">PGXS</link> to build an extension |
| 227 |
against a server that has been compiled with LLVM JIT support, the |
| 228 |
relevant files will be built and installed automatically. |
| 229 |
</para> |
| 230 |
|
| 231 |
<para> |
| 232 |
The relevant files have to be installed into |
| 233 |
<filename>$pkglibdir/bitcode/$extension/</filename> and a summary of them |
| 234 |
into <filename>$pkglibdir/bitcode/$extension.index.bc</filename>, where |
| 235 |
<literal>$pkglibdir</literal> is the directory returned by |
| 236 |
<literal>pg_config --pkglibdir</literal> and <literal>$extension</literal> |
| 237 |
is the base name of the extension's shared library. |
| 238 |
|
| 239 |
<note> |
| 240 |
<para> |
| 241 |
For functions built into <productname>PostgreSQL</productname> itself, |
| 242 |
the bitcode is installed into |
| 243 |
<literal>$pkglibdir/bitcode/postgres</literal>. |
| 244 |
</para> |
| 245 |
</note> |
| 246 |
</para> |
| 247 |
</sect2> |
| 248 |
|
| 249 |
<sect2 id="jit-pluggable"> |
| 250 |
<title>Pluggable <acronym>JIT</acronym> Providers</title> |
| 251 |
|
| 252 |
<para> |
| 253 |
<productname>PostgreSQL</productname> provides a <acronym>JIT</acronym> |
| 254 |
implementation based on <productname>LLVM</productname>. The interface to |
| 255 |
the <acronym>JIT</acronym> provider is pluggable and the provider can be |
| 256 |
changed without recompiling (although currently, the build process only |
| 257 |
provides inlining support data for <productname>LLVM</productname>). |
| 258 |
The active provider is chosen via the setting |
| 259 |
<xref linkend="guc-jit-provider"/>. |
| 260 |
</para> |
| 261 |
|
| 262 |
<sect3 id="jit-pluggable-provider-interface"> |
| 263 |
<title><acronym>JIT</acronym> Provider Interface</title> |
| 264 |
<para> |
| 265 |
A <acronym>JIT</acronym> provider is loaded by dynamically loading the |
| 266 |
named shared library. The normal library search path is used to locate |
| 267 |
the library. To provide the required <acronym>JIT</acronym> provider |
| 268 |
callbacks and to indicate that the library is actually a |
| 269 |
<acronym>JIT</acronym> provider, it needs to provide a C function named |
| 270 |
<function>_PG_jit_provider_init</function>. This function is passed a |
| 271 |
struct that needs to be filled with the callback function pointers for |
| 272 |
individual actions: |
| 273 |
<programlisting> |
| 274 |
struct JitProviderCallbacks |
| 275 |
{ |
| 276 |
JitProviderResetAfterErrorCB reset_after_error; |
| 277 |
JitProviderReleaseContextCB release_context; |
| 278 |
JitProviderCompileExprCB compile_expr; |
| 279 |
}; |
| 280 |
|
| 281 |
extern void _PG_jit_provider_init(JitProviderCallbacks *cb); |
| 282 |
</programlisting> |
| 283 |
</para> |
| 284 |
</sect3> |
| 285 |
</sect2> |
| 286 |
</sect1> |
| 287 |
|
| 288 |
</chapter> |