| 1 |
<!-- |
| 2 |
doc/src/sgml/ref/pg_receivewal.sgml |
| 3 |
PostgreSQL documentation |
| 4 |
--> |
| 5 |
|
| 6 |
<refentry id="app-pgreceivewal"> |
| 7 |
<indexterm zone="app-pgreceivewal"> |
| 8 |
<primary>pg_receivewal</primary> |
| 9 |
</indexterm> |
| 10 |
|
| 11 |
<refmeta> |
| 12 |
<refentrytitle><application>pg_receivewal</application></refentrytitle> |
| 13 |
<manvolnum>1</manvolnum> |
| 14 |
<refmiscinfo>Application</refmiscinfo> |
| 15 |
</refmeta> |
| 16 |
|
| 17 |
<refnamediv> |
| 18 |
<refname>pg_receivewal</refname> |
| 19 |
<refpurpose>stream write-ahead logs from a <productname>PostgreSQL</productname> server</refpurpose> |
| 20 |
</refnamediv> |
| 21 |
|
| 22 |
<refsynopsisdiv> |
| 23 |
<cmdsynopsis> |
| 24 |
<command>pg_receivewal</command> |
| 25 |
<arg rep="repeat"><replaceable>option</replaceable></arg> |
| 26 |
</cmdsynopsis> |
| 27 |
</refsynopsisdiv> |
| 28 |
|
| 29 |
<refsect1> |
| 30 |
<title>Description</title> |
| 31 |
<para> |
| 32 |
<application>pg_receivewal</application> is used to stream the write-ahead log |
| 33 |
from a running <productname>PostgreSQL</productname> cluster. The write-ahead |
| 34 |
log is streamed using the streaming replication protocol, and is written |
| 35 |
to a local directory of files. This directory can be used as the archive |
| 36 |
location for doing a restore using point-in-time recovery (see |
| 37 |
<xref linkend="continuous-archiving"/>). |
| 38 |
</para> |
| 39 |
|
| 40 |
<para> |
| 41 |
<application>pg_receivewal</application> streams the write-ahead |
| 42 |
log in real time as it's being generated on the server, and does not wait |
| 43 |
for segments to complete like <xref linkend="guc-archive-command"/> and |
| 44 |
<xref linkend="guc-archive-library"/> do. |
| 45 |
For this reason, it is not necessary to set |
| 46 |
<xref linkend="guc-archive-timeout"/> when using |
| 47 |
<application>pg_receivewal</application>. |
| 48 |
</para> |
| 49 |
|
| 50 |
<para> |
| 51 |
Unlike the WAL receiver of a PostgreSQL standby server, <application>pg_receivewal</application> |
| 52 |
by default flushes WAL data only when a WAL file is closed. |
| 53 |
The option <option>--synchronous</option> must be specified to flush WAL data |
| 54 |
in real time. Since <application>pg_receivewal</application> does not |
| 55 |
apply WAL, you should not allow it to become a synchronous standby when |
| 56 |
<xref linkend="guc-synchronous-commit"/> equals |
| 57 |
<literal>remote_apply</literal>. If it does, it will appear to be a |
| 58 |
standby that never catches up, and will cause transaction commits to |
| 59 |
block. To avoid this, you should either configure an appropriate value |
| 60 |
for <xref linkend="guc-synchronous-standby-names"/>, or specify |
| 61 |
<varname>application_name</varname> for |
| 62 |
<application>pg_receivewal</application> that does not match it, or |
| 63 |
change the value of <varname>synchronous_commit</varname> to |
| 64 |
something other than <literal>remote_apply</literal>. |
| 65 |
</para> |
| 66 |
|
| 67 |
<para> |
| 68 |
The write-ahead log is streamed over a regular |
| 69 |
<productname>PostgreSQL</productname> connection and uses the replication |
| 70 |
protocol. The connection must be made with a user having |
| 71 |
<literal>REPLICATION</literal> permissions (see |
| 72 |
<xref linkend="role-attributes"/>) or a superuser, and |
| 73 |
<filename>pg_hba.conf</filename> must permit the replication connection. |
| 74 |
The server must also be configured with |
| 75 |
<xref linkend="guc-max-wal-senders"/> set high enough to leave at least |
| 76 |
one session available for the stream. |
| 77 |
</para> |
| 78 |
|
| 79 |
<para> |
| 80 |
The starting point of the write-ahead log streaming is calculated when |
| 81 |
<application>pg_receivewal</application> starts: |
| 82 |
<orderedlist> |
| 83 |
<listitem> |
| 84 |
<para> |
| 85 |
First, scan the directory where the WAL segment files are written and |
| 86 |
find the newest completed segment file, using as the starting point the |
| 87 |
beginning of the next WAL segment file. |
| 88 |
</para> |
| 89 |
</listitem> |
| 90 |
|
| 91 |
<listitem> |
| 92 |
<para> |
| 93 |
If a starting point cannot be calculated with the previous method, |
| 94 |
and if a replication slot is used, an extra |
| 95 |
<command>READ_REPLICATION_SLOT</command> command is issued to retrieve |
| 96 |
the slot's <literal>restart_lsn</literal> to use as the starting point. |
| 97 |
This option is only available when streaming write-ahead logs from |
| 98 |
<productname>PostgreSQL</productname> 15 and up. |
| 99 |
</para> |
| 100 |
</listitem> |
| 101 |
|
| 102 |
<listitem> |
| 103 |
<para> |
| 104 |
If a starting point cannot be calculated with the previous method, |
| 105 |
the latest WAL flush location is used as reported by the server from |
| 106 |
an <literal>IDENTIFY_SYSTEM</literal> command. |
| 107 |
</para> |
| 108 |
</listitem> |
| 109 |
</orderedlist> |
| 110 |
</para> |
| 111 |
|
| 112 |
<para> |
| 113 |
If the connection is lost, or if it cannot be initially established, |
| 114 |
with a non-fatal error, <application>pg_receivewal</application> will |
| 115 |
retry the connection indefinitely, and reestablish streaming as soon |
| 116 |
as possible. To avoid this behavior, use the <literal>-n</literal> |
| 117 |
parameter. |
| 118 |
</para> |
| 119 |
|
| 120 |
<para> |
| 121 |
In the absence of fatal errors, <application>pg_receivewal</application> |
| 122 |
will run until terminated by the <systemitem>SIGINT</systemitem> |
| 123 |
(<keycombo action="simul"><keycap>Control</keycap><keycap>C</keycap></keycombo>) |
| 124 |
or <systemitem>SIGTERM</systemitem> signal. |
| 125 |
</para> |
| 126 |
</refsect1> |
| 127 |
|
| 128 |
<refsect1> |
| 129 |
<title>Options</title> |
| 130 |
|
| 131 |
<variablelist> |
| 132 |
<varlistentry> |
| 133 |
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term> |
| 134 |
<term><option>--directory=<replaceable class="parameter">directory</replaceable></option></term> |
| 135 |
<listitem> |
| 136 |
<para> |
| 137 |
Directory to write the output to. |
| 138 |
</para> |
| 139 |
<para> |
| 140 |
This parameter is required. |
| 141 |
</para> |
| 142 |
</listitem> |
| 143 |
</varlistentry> |
| 144 |
|
| 145 |
<varlistentry> |
| 146 |
<term><option>-E <replaceable>lsn</replaceable></option></term> |
| 147 |
<term><option>--endpos=<replaceable>lsn</replaceable></option></term> |
| 148 |
<listitem> |
| 149 |
<para> |
| 150 |
Automatically stop replication and exit with normal exit status 0 when |
| 151 |
receiving reaches the specified LSN. |
| 152 |
</para> |
| 153 |
|
| 154 |
<para> |
| 155 |
If there is a record with LSN exactly equal to <replaceable>lsn</replaceable>, |
| 156 |
the record will be processed. |
| 157 |
</para> |
| 158 |
</listitem> |
| 159 |
</varlistentry> |
| 160 |
|
| 161 |
<varlistentry> |
| 162 |
<term><option>--if-not-exists</option></term> |
| 163 |
<listitem> |
| 164 |
<para> |
| 165 |
Do not error out when <option>--create-slot</option> is specified |
| 166 |
and a slot with the specified name already exists. |
| 167 |
</para> |
| 168 |
</listitem> |
| 169 |
</varlistentry> |
| 170 |
|
| 171 |
<varlistentry> |
| 172 |
<term><option>-n</option></term> |
| 173 |
<term><option>--no-loop</option></term> |
| 174 |
<listitem> |
| 175 |
<para> |
| 176 |
Don't loop on connection errors. Instead, exit right away with |
| 177 |
an error. |
| 178 |
</para> |
| 179 |
</listitem> |
| 180 |
</varlistentry> |
| 181 |
|
| 182 |
<varlistentry> |
| 183 |
<term><option>--no-sync</option></term> |
| 184 |
<listitem> |
| 185 |
<para> |
| 186 |
This option causes <command>pg_receivewal</command> to not force WAL |
| 187 |
data to be flushed to disk. This is faster, but means that a |
| 188 |
subsequent operating system crash can leave the WAL segments corrupt. |
| 189 |
Generally, this option is useful for testing but should not be used |
| 190 |
when doing WAL archiving on a production deployment. |
| 191 |
</para> |
| 192 |
|
| 193 |
<para> |
| 194 |
This option is incompatible with <literal>--synchronous</literal>. |
| 195 |
</para> |
| 196 |
</listitem> |
| 197 |
</varlistentry> |
| 198 |
|
| 199 |
<varlistentry> |
| 200 |
<term><option>-s <replaceable class="parameter">interval</replaceable></option></term> |
| 201 |
<term><option>--status-interval=<replaceable class="parameter">interval</replaceable></option></term> |
| 202 |
<listitem> |
| 203 |
<para> |
| 204 |
Specifies the number of seconds between status packets sent back to the |
| 205 |
server. This allows for easier monitoring of the progress from server. |
| 206 |
A value of zero disables the periodic status updates completely, |
| 207 |
although an update will still be sent when requested by the server, to |
| 208 |
avoid timeout disconnect. The default value is 10 seconds. |
| 209 |
</para> |
| 210 |
</listitem> |
| 211 |
</varlistentry> |
| 212 |
|
| 213 |
<varlistentry> |
| 214 |
<term><option>-S <replaceable>slotname</replaceable></option></term> |
| 215 |
<term><option>--slot=<replaceable class="parameter">slotname</replaceable></option></term> |
| 216 |
<listitem> |
| 217 |
<para> |
| 218 |
Require <application>pg_receivewal</application> to use an existing |
| 219 |
replication slot (see <xref linkend="streaming-replication-slots"/>). |
| 220 |
When this option is used, <application>pg_receivewal</application> will report |
| 221 |
a flush position to the server, indicating when each segment has been |
| 222 |
synchronized to disk so that the server can remove that segment if it |
| 223 |
is not otherwise needed. |
| 224 |
</para> |
| 225 |
|
| 226 |
<para> |
| 227 |
When the replication client |
| 228 |
of <application>pg_receivewal</application> is configured on the |
| 229 |
server as a synchronous standby, then using a replication slot will |
| 230 |
report the flush position to the server, but only when a WAL file is |
| 231 |
closed. Therefore, that configuration will cause transactions on the |
| 232 |
primary to wait for a long time and effectively not work |
| 233 |
satisfactorily. The option <literal>--synchronous</literal> (see |
| 234 |
below) must be specified in addition to make this work correctly. |
| 235 |
</para> |
| 236 |
</listitem> |
| 237 |
</varlistentry> |
| 238 |
|
| 239 |
<varlistentry> |
| 240 |
<term><option>--synchronous</option></term> |
| 241 |
<listitem> |
| 242 |
<para> |
| 243 |
Flush the WAL data to disk immediately after it has been received. Also |
| 244 |
send a status packet back to the server immediately after flushing, |
| 245 |
regardless of <literal>--status-interval</literal>. |
| 246 |
</para> |
| 247 |
|
| 248 |
<para> |
| 249 |
This option should be specified if the replication client |
| 250 |
of <application>pg_receivewal</application> is configured on the |
| 251 |
server as a synchronous standby, to ensure that timely feedback is |
| 252 |
sent to the server. |
| 253 |
</para> |
| 254 |
</listitem> |
| 255 |
</varlistentry> |
| 256 |
|
| 257 |
<varlistentry> |
| 258 |
<term><option>-v</option></term> |
| 259 |
<term><option>--verbose</option></term> |
| 260 |
<listitem> |
| 261 |
<para> |
| 262 |
Enables verbose mode. |
| 263 |
</para> |
| 264 |
</listitem> |
| 265 |
</varlistentry> |
| 266 |
|
| 267 |
<varlistentry> |
| 268 |
<term><option>-Z <replaceable class="parameter">level</replaceable></option></term> |
| 269 |
<term><option>-Z <replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term> |
| 270 |
<term><option>--compress=<replaceable class="parameter">level</replaceable></option></term> |
| 271 |
<term><option>--compress=<replaceable class="parameter">method</replaceable>[:<replaceable>detail</replaceable>]</option></term> |
| 272 |
<listitem> |
| 273 |
<para> |
| 274 |
Enables compression of write-ahead logs. |
| 275 |
</para> |
| 276 |
<para> |
| 277 |
The compression method can be set to <literal>gzip</literal>, |
| 278 |
<literal>lz4</literal> (if <productname>PostgreSQL</productname> |
| 279 |
was compiled with <option>--with-lz4</option>) or |
| 280 |
<literal>none</literal> for no compression. |
| 281 |
A compression detail string can optionally be specified. If the |
| 282 |
detail string is an integer, it specifies the compression level. |
| 283 |
Otherwise, it should be a comma-separated list of items, each of the |
| 284 |
form <replaceable>keyword</replaceable> or |
| 285 |
<replaceable>keyword=value</replaceable>. |
| 286 |
Currently, the only supported keyword is <literal>level</literal>. |
| 287 |
</para> |
| 288 |
<para> |
| 289 |
If no compression level is specified, the default compression level |
| 290 |
will be used. If only a level is specified without mentioning an |
| 291 |
algorithm, <literal>gzip</literal> compression will be used if the |
| 292 |
level is greater than 0, and no compression will be used if the level |
| 293 |
is 0. |
| 294 |
</para> |
| 295 |
<para> |
| 296 |
The suffix <filename>.gz</filename> will automatically be added to |
| 297 |
all filenames when using <literal>gzip</literal>, and the suffix |
| 298 |
<filename>.lz4</filename> is added when using <literal>lz4</literal>. |
| 299 |
</para> |
| 300 |
</listitem> |
| 301 |
</varlistentry> |
| 302 |
</variablelist> |
| 303 |
|
| 304 |
<para> |
| 305 |
The following command-line options control the database connection parameters. |
| 306 |
|
| 307 |
<variablelist> |
| 308 |
<varlistentry> |
| 309 |
<term><option>-d <replaceable class="parameter">connstr</replaceable></option></term> |
| 310 |
<term><option>--dbname=<replaceable class="parameter">connstr</replaceable></option></term> |
| 311 |
<listitem> |
| 312 |
<para> |
| 313 |
Specifies parameters used to connect to the server, as a <link |
| 314 |
linkend="libpq-connstring">connection string</link>; these |
| 315 |
will override any conflicting command line options. |
| 316 |
</para> |
| 317 |
<para> |
| 318 |
This option is called <literal>--dbname</literal> for consistency with other |
| 319 |
client applications, but because <application>pg_receivewal</application> |
| 320 |
doesn't connect to any particular database in the cluster, any database |
| 321 |
name included in the connection string will be ignored by the server. |
| 322 |
However, a database name supplied that way overrides the default |
| 323 |
database name (<literal>replication</literal>) for purposes of |
| 324 |
looking up the replication connection's password |
| 325 |
in <filename>~/.pgpass</filename>. Similarly, middleware or proxies |
| 326 |
used in connecting to <productname>PostgreSQL</productname> might |
| 327 |
utilize the name for purposes such as connection routing. |
| 328 |
</para> |
| 329 |
</listitem> |
| 330 |
</varlistentry> |
| 331 |
|
| 332 |
<varlistentry> |
| 333 |
<term><option>-h <replaceable class="parameter">host</replaceable></option></term> |
| 334 |
<term><option>--host=<replaceable class="parameter">host</replaceable></option></term> |
| 335 |
<listitem> |
| 336 |
<para> |
| 337 |
Specifies the host name of the machine on which the server is |
| 338 |
running. If the value begins with a slash, it is used as the |
| 339 |
directory for the Unix domain socket. The default is taken |
| 340 |
from the <envar>PGHOST</envar> environment variable, if set, |
| 341 |
else a Unix domain socket connection is attempted. |
| 342 |
</para> |
| 343 |
</listitem> |
| 344 |
</varlistentry> |
| 345 |
|
| 346 |
<varlistentry> |
| 347 |
<term><option>-p <replaceable class="parameter">port</replaceable></option></term> |
| 348 |
<term><option>--port=<replaceable class="parameter">port</replaceable></option></term> |
| 349 |
<listitem> |
| 350 |
<para> |
| 351 |
Specifies the TCP port or local Unix domain socket file |
| 352 |
extension on which the server is listening for connections. |
| 353 |
Defaults to the <envar>PGPORT</envar> environment variable, if |
| 354 |
set, or a compiled-in default. |
| 355 |
</para> |
| 356 |
</listitem> |
| 357 |
</varlistentry> |
| 358 |
|
| 359 |
<varlistentry> |
| 360 |
<term><option>-U <replaceable>username</replaceable></option></term> |
| 361 |
<term><option>--username=<replaceable class="parameter">username</replaceable></option></term> |
| 362 |
<listitem> |
| 363 |
<para> |
| 364 |
User name to connect as. |
| 365 |
</para> |
| 366 |
</listitem> |
| 367 |
</varlistentry> |
| 368 |
|
| 369 |
<varlistentry> |
| 370 |
<term><option>-w</option></term> |
| 371 |
<term><option>--no-password</option></term> |
| 372 |
<listitem> |
| 373 |
<para> |
| 374 |
Never issue a password prompt. If the server requires |
| 375 |
password authentication and a password is not available by |
| 376 |
other means such as a <filename>.pgpass</filename> file, the |
| 377 |
connection attempt will fail. This option can be useful in |
| 378 |
batch jobs and scripts where no user is present to enter a |
| 379 |
password. |
| 380 |
</para> |
| 381 |
</listitem> |
| 382 |
</varlistentry> |
| 383 |
|
| 384 |
<varlistentry> |
| 385 |
<term><option>-W</option></term> |
| 386 |
<term><option>--password</option></term> |
| 387 |
<listitem> |
| 388 |
<para> |
| 389 |
Force <application>pg_receivewal</application> to prompt for a |
| 390 |
password before connecting to a database. |
| 391 |
</para> |
| 392 |
|
| 393 |
<para> |
| 394 |
This option is never essential, since |
| 395 |
<application>pg_receivewal</application> will automatically prompt |
| 396 |
for a password if the server demands password authentication. |
| 397 |
However, <application>pg_receivewal</application> will waste a |
| 398 |
connection attempt finding out that the server wants a password. |
| 399 |
In some cases it is worth typing <option>-W</option> to avoid the extra |
| 400 |
connection attempt. |
| 401 |
</para> |
| 402 |
</listitem> |
| 403 |
</varlistentry> |
| 404 |
</variablelist> |
| 405 |
</para> |
| 406 |
|
| 407 |
<para> |
| 408 |
<application>pg_receivewal</application> can perform one of the two |
| 409 |
following actions in order to control physical replication slots: |
| 410 |
|
| 411 |
<variablelist> |
| 412 |
<varlistentry> |
| 413 |
<term><option>--create-slot</option></term> |
| 414 |
<listitem> |
| 415 |
<para> |
| 416 |
Create a new physical replication slot with the name specified in |
| 417 |
<option>--slot</option>, then exit. |
| 418 |
</para> |
| 419 |
</listitem> |
| 420 |
</varlistentry> |
| 421 |
|
| 422 |
<varlistentry> |
| 423 |
<term><option>--drop-slot</option></term> |
| 424 |
<listitem> |
| 425 |
<para> |
| 426 |
Drop the replication slot with the name specified in |
| 427 |
<option>--slot</option>, then exit. |
| 428 |
</para> |
| 429 |
</listitem> |
| 430 |
</varlistentry> |
| 431 |
</variablelist> |
| 432 |
</para> |
| 433 |
|
| 434 |
<para> |
| 435 |
Other options are also available: |
| 436 |
|
| 437 |
<variablelist> |
| 438 |
<varlistentry> |
| 439 |
<term><option>-V</option></term> |
| 440 |
<term><option>--version</option></term> |
| 441 |
<listitem> |
| 442 |
<para> |
| 443 |
Print the <application>pg_receivewal</application> version and exit. |
| 444 |
</para> |
| 445 |
</listitem> |
| 446 |
</varlistentry> |
| 447 |
|
| 448 |
<varlistentry> |
| 449 |
<term><option>-?</option></term> |
| 450 |
<term><option>--help</option></term> |
| 451 |
<listitem> |
| 452 |
<para> |
| 453 |
Show help about <application>pg_receivewal</application> command line |
| 454 |
arguments, and exit. |
| 455 |
</para> |
| 456 |
</listitem> |
| 457 |
</varlistentry> |
| 458 |
|
| 459 |
</variablelist> |
| 460 |
</para> |
| 461 |
|
| 462 |
</refsect1> |
| 463 |
|
| 464 |
<refsect1> |
| 465 |
<title>Exit Status</title> |
| 466 |
|
| 467 |
<para> |
| 468 |
<application>pg_receivewal</application> will exit with status 0 when |
| 469 |
terminated by the <systemitem>SIGINT</systemitem> or |
| 470 |
<systemitem>SIGTERM</systemitem> signal. (That is the |
| 471 |
normal way to end it. Hence it is not an error.) For fatal errors or |
| 472 |
other signals, the exit status will be nonzero. |
| 473 |
</para> |
| 474 |
|
| 475 |
</refsect1> |
| 476 |
|
| 477 |
<refsect1> |
| 478 |
<title>Environment</title> |
| 479 |
|
| 480 |
<para> |
| 481 |
This utility, like most other <productname>PostgreSQL</productname> utilities, |
| 482 |
uses the environment variables supported by <application>libpq</application> |
| 483 |
(see <xref linkend="libpq-envars"/>). |
| 484 |
</para> |
| 485 |
|
| 486 |
<para> |
| 487 |
The environment variable <envar>PG_COLOR</envar> specifies whether to use |
| 488 |
color in diagnostic messages. Possible values are |
| 489 |
<literal>always</literal>, <literal>auto</literal> and |
| 490 |
<literal>never</literal>. |
| 491 |
</para> |
| 492 |
</refsect1> |
| 493 |
|
| 494 |
<refsect1> |
| 495 |
<title>Notes</title> |
| 496 |
|
| 497 |
<para> |
| 498 |
When using <application>pg_receivewal</application> instead of |
| 499 |
<xref linkend="guc-archive-command"/> or |
| 500 |
<xref linkend="guc-archive-library"/> as the main WAL backup method, it is |
| 501 |
strongly recommended to use replication slots. Otherwise, the server is |
| 502 |
free to recycle or remove write-ahead log files before they are backed up, |
| 503 |
because it does not have any information, either |
| 504 |
from <xref linkend="guc-archive-command"/> or |
| 505 |
<xref linkend="guc-archive-library"/> or the replication slots, about |
| 506 |
how far the WAL stream has been archived. Note, however, that a |
| 507 |
replication slot will fill up the server's disk space if the receiver does |
| 508 |
not keep up with fetching the WAL data. |
| 509 |
</para> |
| 510 |
|
| 511 |
<para> |
| 512 |
<application>pg_receivewal</application> will preserve group permissions on |
| 513 |
the received WAL files if group permissions are enabled on the source |
| 514 |
cluster. |
| 515 |
</para> |
| 516 |
|
| 517 |
</refsect1> |
| 518 |
|
| 519 |
<refsect1> |
| 520 |
<title>Examples</title> |
| 521 |
|
| 522 |
<para> |
| 523 |
To stream the write-ahead log from the server at |
| 524 |
<literal>mydbserver</literal> and store it in the local directory |
| 525 |
<filename>/usr/local/pgsql/archive</filename>: |
| 526 |
<screen> |
| 527 |
<prompt>$</prompt> <userinput>pg_receivewal -h mydbserver -D /usr/local/pgsql/archive</userinput> |
| 528 |
</screen></para> |
| 529 |
</refsect1> |
| 530 |
|
| 531 |
<refsect1> |
| 532 |
<title>See Also</title> |
| 533 |
|
| 534 |
<simplelist type="inline"> |
| 535 |
<member><xref linkend="app-pgbasebackup"/></member> |
| 536 |
</simplelist> |
| 537 |
</refsect1> |
| 538 |
|
| 539 |
</refentry> |