Debian Codegraph @

doc/src/sgml/ref/pg_rewind.sgml

raw
1 <!--
2 doc/src/sgml/ref/pg_rewind.sgml
3 PostgreSQL documentation
4 -->
5
6 <refentry id="app-pgrewind">
7 <indexterm zone="app-pgrewind">
8 <primary>pg_rewind</primary>
9 </indexterm>
10
11 <refmeta>
12 <refentrytitle><application>pg_rewind</application></refentrytitle>
13 <manvolnum>1</manvolnum>
14 <refmiscinfo>Application</refmiscinfo>
15 </refmeta>
16
17 <refnamediv>
18 <refname>pg_rewind</refname>
19 <refpurpose>synchronize a <productname>PostgreSQL</productname> data directory with another data directory that was forked from it</refpurpose>
20 </refnamediv>
21
22 <refsynopsisdiv>
23 <cmdsynopsis>
24 <command>pg_rewind</command>
25 <arg rep="repeat"><replaceable>option</replaceable></arg>
26 <group choice="plain">
27 <group choice="req">
28 <arg choice="plain"><option>-D</option></arg>
29 <arg choice="plain"><option>--target-pgdata</option></arg>
30 </group>
31 <replaceable> directory</replaceable>
32 <group choice="req">
33 <arg choice="plain"><option>--source-pgdata=<replaceable>directory</replaceable></option></arg>
34 <arg choice="plain"><option>--source-server=<replaceable>connstr</replaceable></option></arg>
35 </group>
36 </group>
37 </cmdsynopsis>
38 </refsynopsisdiv>
39
40 <refsect1>
41 <title>Description</title>
42
43 <para>
44 <application>pg_rewind</application> is a tool for synchronizing a PostgreSQL cluster
45 with another copy of the same cluster, after the clusters' timelines have
46 diverged. A typical scenario is to bring an old primary server back online
47 after failover as a standby that follows the new primary.
48 </para>
49
50 <para>
51 After a successful rewind, the state of the target data directory is
52 analogous to a base backup of the source data directory. Unlike taking
53 a new base backup or using a tool like <application>rsync</application>,
54 <application>pg_rewind</application> does not require comparing or copying
55 unchanged relation blocks in the cluster. Only changed blocks from existing
56 relation files are copied; all other files, including new relation files,
57 configuration files, and WAL segments, are copied in full. As such the
58 rewind operation is significantly faster than other approaches when the
59 database is large and only a small fraction of blocks differ between the
60 clusters.
61 </para>
62
63 <para>
64 <application>pg_rewind</application> examines the timeline histories of the source
65 and target clusters to determine the point where they diverged, and
66 expects to find WAL in the target cluster's <filename>pg_wal</filename> directory
67 reaching all the way back to the point of divergence. The point of divergence
68 can be found either on the target timeline, the source timeline, or their common
69 ancestor. In the typical failover scenario where the target cluster was
70 shut down soon after the divergence, this is not a problem, but if the
71 target cluster ran for a long time after the divergence, its old WAL
72 files might no longer be present. In this case, you can manually copy them
73 from the WAL archive to the <filename>pg_wal</filename> directory, or run
74 <application>pg_rewind</application> with the <literal>-c</literal> option to
75 automatically retrieve them from the WAL archive. The use of
76 <application>pg_rewind</application> is not limited to failover, e.g., a standby
77 server can be promoted, run some write transactions, and then rewound
78 to become a standby again.
79 </para>
80
81 <para>
82 After running <application>pg_rewind</application>, WAL replay needs to
83 complete for the data directory to be in a consistent state. When the
84 target server is started again it will enter archive recovery and replay
85 all WAL generated in the source server from the last checkpoint before
86 the point of divergence. If some of the WAL was no longer available in the
87 source server when <application>pg_rewind</application> was run, and
88 therefore could not be copied by the <application>pg_rewind</application>
89 session, it must be made available when the target server is started.
90 This can be done by creating a <filename>recovery.signal</filename> file
91 in the target data directory and by configuring a suitable
92 <xref linkend="guc-restore-command"/> in
93 <filename>postgresql.conf</filename>.
94 </para>
95
96 <para>
97 <application>pg_rewind</application> requires that the target server either has
98 the <xref linkend="guc-wal-log-hints"/> option enabled
99 in <filename>postgresql.conf</filename> or data checksums enabled when
100 the cluster was initialized with <application>initdb</application> (the
101 default). <xref linkend="guc-full-page-writes"/> must also be set to
102 <literal>on</literal>, but is enabled by default.
103 </para>
104
105 <warning>
106 <title>Warning: Failures While Rewinding</title>
107 <para>
108 If <application>pg_rewind</application> fails while processing, then
109 the data folder of the target is likely not in a state that can be
110 recovered. In such a case, taking a new fresh backup is recommended.
111 </para>
112
113 <para>
114 As <application>pg_rewind</application> copies configuration files
115 entirely from the source, it may be required to correct the configuration
116 used for recovery before restarting the target server, especially if
117 the target is reintroduced as a standby of the source. If you restart
118 the server after the rewind operation has finished but without configuring
119 recovery, the target may again diverge from the primary.
120 </para>
121
122 <para>
123 <application>pg_rewind</application> will fail immediately if it finds
124 files it cannot write directly to. This can happen for example when
125 the source and the target server use the same file mapping for read-only
126 SSL keys and certificates. If such files are present on the target server
127 it is recommended to remove them before running
128 <application>pg_rewind</application>. After doing the rewind, some of
129 those files may have been copied from the source, in which case it may
130 be necessary to remove the data copied and restore back the set of links
131 used before the rewind.
132 </para>
133 </warning>
134 </refsect1>
135
136 <refsect1>
137 <title>Options</title>
138
139 <para>
140 <application>pg_rewind</application> accepts the following command-line
141 arguments:
142
143 <variablelist>
144 <varlistentry>
145 <term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
146 <term><option>--target-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
147 <listitem>
148 <para>
149 This option specifies the target data directory that is synchronized
150 with the source. The target server must be shut down cleanly before
151 running <application>pg_rewind</application>
152 </para>
153 </listitem>
154 </varlistentry>
155
156 <varlistentry>
157 <term><option>--source-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
158 <listitem>
159 <para>
160 Specifies the file system path to the data directory of the source
161 server to synchronize the target with. This option requires the
162 source server to be cleanly shut down.
163 </para>
164 </listitem>
165 </varlistentry>
166
167 <varlistentry>
168 <term><option>--source-server=<replaceable class="parameter">connstr</replaceable></option></term>
169 <listitem>
170 <para>
171 Specifies a libpq connection string to connect to the source
172 <productname>PostgreSQL</productname> server to synchronize the target
173 with. The connection must be a normal (non-replication) connection
174 with a role having sufficient permissions to execute the functions
175 used by <application>pg_rewind</application> on the source server
176 (see Notes section for details) or a superuser role. This option
177 requires the source server to be running and accepting connections.
178 </para>
179 </listitem>
180 </varlistentry>
181
182 <varlistentry>
183 <term><option>-R</option></term>
184 <term><option>--write-recovery-conf</option></term>
185 <listitem>
186 <para>
187 Create <filename>standby.signal</filename> and append connection
188 settings to <filename>postgresql.auto.conf</filename> in the output
189 directory. The dbname will be recorded only if the dbname was
190 specified explicitly in the connection string or <link linkend="libpq-envars">
191 environment variable</link>. <literal>--source-server</literal> is
192 mandatory with this option.
193 </para>
194 </listitem>
195 </varlistentry>
196
197 <varlistentry>
198 <term><option>-n</option></term>
199 <term><option>--dry-run</option></term>
200 <listitem>
201 <para>
202 Do everything except actually modifying the target directory.
203 </para>
204 </listitem>
205 </varlistentry>
206
207 <varlistentry>
208 <term><option>-N</option></term>
209 <term><option>--no-sync</option></term>
210 <listitem>
211 <para>
212 By default, <command>pg_rewind</command> will wait for all files
213 to be written safely to disk. This option causes
214 <command>pg_rewind</command> to return without waiting, which is
215 faster, but means that a subsequent operating system crash can leave
216 the data directory corrupt. Generally, this option is useful for
217 testing but should not be used on a production
218 installation.
219 </para>
220 </listitem>
221 </varlistentry>
222
223 <varlistentry>
224 <term><option>-P</option></term>
225 <term><option>--progress</option></term>
226 <listitem>
227 <para>
228 Enables progress reporting. Turning this on will deliver an approximate
229 progress report while copying data from the source cluster.
230 </para>
231 </listitem>
232 </varlistentry>
233
234 <varlistentry>
235 <term><option>-c</option></term>
236 <term><option>--restore-target-wal</option></term>
237 <listitem>
238 <para>
239 Use <varname>restore_command</varname> defined in the target cluster
240 configuration to retrieve WAL files from the WAL archive if these
241 files are no longer available in the <filename>pg_wal</filename>
242 directory.
243 </para>
244 </listitem>
245 </varlistentry>
246
247 <varlistentry>
248 <term><option>--config-file=<replaceable class="parameter">filename</replaceable></option></term>
249 <listitem>
250 <para>
251 Use the specified main server configuration file for the target
252 cluster. This affects <application>pg_rewind</application> when
253 it uses internally the <application>postgres</application> command
254 for the rewind operation on this cluster (when retrieving
255 <varname>restore_command</varname> with the option
256 <option>-c/--restore-target-wal</option> and when forcing a
257 completion of crash recovery).
258 </para>
259 </listitem>
260 </varlistentry>
261
262 <varlistentry>
263 <term><option>--debug</option></term>
264 <listitem>
265 <para>
266 Print verbose debugging output that is mostly useful for developers
267 debugging <application>pg_rewind</application>.
268 </para>
269 </listitem>
270 </varlistentry>
271
272 <varlistentry>
273 <term><option>--no-ensure-shutdown</option></term>
274 <listitem>
275 <para>
276 <application>pg_rewind</application> requires that the target server
277 is cleanly shut down before rewinding. By default, if the target server
278 is not shut down cleanly, <application>pg_rewind</application> starts
279 the target server in single-user mode to complete crash recovery first,
280 and stops it.
281 By passing this option, <application>pg_rewind</application> skips
282 this and errors out immediately if the server is not cleanly shut
283 down. Users are expected to handle the situation themselves in that
284 case.
285 </para>
286 </listitem>
287 </varlistentry>
288
289 <varlistentry>
290 <term><option>--sync-method=<replaceable class="parameter">method</replaceable></option></term>
291 <listitem>
292 <para>
293 When set to <literal>fsync</literal>, which is the default,
294 <command>pg_rewind</command> will recursively open and synchronize all
295 files in the data directory. The search for files will follow symbolic
296 links for the WAL directory and each configured tablespace.
297 </para>
298 <para>
299 On Linux, <literal>syncfs</literal> may be used instead to ask the
300 operating system to synchronize the whole file systems that contain the
301 data directory, the WAL files, and each tablespace. See
302 <xref linkend="guc-recovery-init-sync-method"/> for information about
303 the caveats to be aware of when using <literal>syncfs</literal>.
304 </para>
305 <para>
306 This option has no effect when <option>--no-sync</option> is used.
307 </para>
308 </listitem>
309 </varlistentry>
310
311 <varlistentry>
312 <term><option>-V</option></term>
313 <term><option>--version</option></term>
314 <listitem><para>Display version information, then exit.</para></listitem>
315 </varlistentry>
316
317 <varlistentry>
318 <term><option>-?</option></term>
319 <term><option>--help</option></term>
320 <listitem><para>Show help, then exit.</para></listitem>
321 </varlistentry>
322
323 </variablelist>
324 </para>
325 </refsect1>
326
327 <refsect1>
328 <title>Environment</title>
329
330 <para>
331 When <option>--source-server</option> option is used,
332 <application>pg_rewind</application> also uses the environment variables
333 supported by <application>libpq</application> (see <xref linkend="libpq-envars"/>).
334 </para>
335
336 <para>
337 The environment variable <envar>PG_COLOR</envar> specifies whether to use
338 color in diagnostic messages. Possible values are
339 <literal>always</literal>, <literal>auto</literal> and
340 <literal>never</literal>.
341 </para>
342 </refsect1>
343
344 <refsect1>
345 <title>Notes</title>
346
347 <para>
348 When executing <application>pg_rewind</application> using an online
349 cluster as source, a role having sufficient permissions to execute the
350 functions used by <application>pg_rewind</application> on the source
351 cluster can be used instead of a superuser. Here is how to create such
352 a role, named <literal>rewind_user</literal> here:
353 <programlisting>
354 CREATE USER rewind_user LOGIN;
355 GRANT EXECUTE ON function pg_catalog.pg_ls_dir(text, boolean, boolean) TO rewind_user;
356 GRANT EXECUTE ON function pg_catalog.pg_stat_file(text, boolean) TO rewind_user;
357 GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text) TO rewind_user;
358 GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean) TO rewind_user;
359 </programlisting>
360 </para>
361
362 <refsect2>
363 <title>How It Works</title>
364
365 <para>
366 The basic idea is to copy all file system-level changes from the source
367 cluster to the target cluster:
368 </para>
369
370 <procedure>
371 <step>
372 <para>
373 Scan the WAL log of the target cluster, starting from the last
374 checkpoint before the point where the source cluster's timeline
375 history forked off from the target cluster. For each WAL record,
376 record each data block that was touched. This yields a list of all
377 the data blocks that were changed in the target cluster, after the
378 source cluster forked off. If some of the WAL files are no longer
379 available, try re-running <application>pg_rewind</application> with
380 the <option>-c</option> option to search for the missing files in
381 the WAL archive.
382 </para>
383 </step>
384 <step>
385 <para>
386 Copy all those changed blocks from the source cluster to
387 the target cluster, either using direct file system access
388 (<option>--source-pgdata</option>) or SQL (<option>--source-server</option>).
389 Relation files are now in a state equivalent to the moment of the last
390 completed checkpoint prior to the point at which the WAL timelines of the
391 source and target diverged plus the current state on the source of any
392 blocks changed on the target after that divergence.
393 </para>
394 </step>
395 <step>
396 <para>
397 Copy all other files, including new relation files, WAL segments,
398 <filename>pg_xact</filename>, and configuration files from the source
399 cluster to the target cluster. Similarly to base backups, the contents
400 of the directories <filename>pg_dynshmem/</filename>,
401 <filename>pg_notify/</filename>, <filename>pg_replslot/</filename>,
402 <filename>pg_serial/</filename>, <filename>pg_snapshots/</filename>,
403 <filename>pg_stat_tmp/</filename>, and <filename>pg_subtrans/</filename>
404 are omitted from the data copied from the source cluster. The files
405 <filename>backup_label</filename>,
406 <filename>tablespace_map</filename>,
407 <filename>pg_internal.init</filename>,
408 <filename>postmaster.opts</filename>,
409 <filename>postmaster.pid</filename> and
410 <filename>.DS_Store</filename> as well as any file or directory
411 beginning with <filename>pgsql_tmp</filename>, are omitted.
412 </para>
413 </step>
414 <step>
415 <para>
416 Create a <filename>backup_label</filename> file to begin WAL replay at
417 the checkpoint created at failover and configure the
418 <filename>pg_control</filename> file with a minimum consistency LSN
419 defined as the result of <literal>pg_current_wal_insert_lsn()</literal>
420 when rewinding from a live source or the last checkpoint LSN when
421 rewinding from a stopped source.
422 </para>
423 </step>
424 <step>
425 <para>
426 When starting the target, <productname>PostgreSQL</productname> replays
427 all the required WAL, resulting in a data directory in a consistent
428 state.
429 </para>
430 </step>
431 </procedure>
432 </refsect2>
433 </refsect1>
434
435 </refentry>