Debian Codegraph @

src/backend/storage/page/README

raw
1 src/backend/storage/page/README
2
3 Checksums
4 ---------
5
6 Checksums on data pages are designed to detect corruption by the I/O system.
7 We do not protect buffers against uncorrectable memory errors, since these
8 have a very low measured incidence according to research on large server farms,
9 http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf, discussed
10 2010/12/22 on -hackers list.
11
12 Current implementation requires this be enabled system-wide at initdb time, or
13 by using the pg_checksums tool on an offline cluster.
14
15 The checksum is not valid at all times on a data page!!
16 The checksum is valid when the page leaves the shared pool and is checked
17 when it later re-enters the shared pool as a result of I/O.
18 We set the checksum on a buffer in the shared pool immediately before we
19 flush the buffer. As a result we implicitly invalidate the page's checksum
20 when we modify the page for a data change or even a hint. This means that
21 many or even most pages in shared buffers have invalid page checksums,
22 so be careful how you interpret the pd_checksum field.
23
24 That means that WAL-logged changes to a page do NOT update the page checksum,
25 so full page images may not have a valid checksum. But those page images have
26 the WAL CRC covering them and so are verified separately from this
27 mechanism. WAL replay should not test the checksum of a full-page image.
28
29 The best way to understand this is that WAL CRCs protect records entering the
30 WAL stream, and data page verification protects blocks entering the shared
31 buffer pool. They are similar in purpose, yet completely separate. Together
32 they ensure we are able to detect errors in data re-entering
33 PostgreSQL-controlled memory. Note also that the WAL checksum is a 32-bit CRC,
34 whereas the page checksum is only 16-bits.
35
36 Any write of a data block can cause a torn page if the write is unsuccessful.
37 Full page writes protect us from that, which are stored in WAL. Setting hint
38 bits when a page is already dirty is OK because a full page write must already
39 have been written for it since the last checkpoint. Setting hint bits on an
40 otherwise clean page can allow torn pages; this doesn't normally matter since
41 they are just hints, but when the page has checksums, then losing a few bits
42 would cause the checksum to be invalid. So if we have full_page_writes = on
43 and checksums enabled then we must write a WAL record specifically so that we
44 record a full page image in WAL. Hint bits updates should be protected using
45 MarkBufferDirtyHint(), which is responsible for writing the full-page image
46 when necessary.
47
48 Note that when we write a page checksum we include the hopefully zeroed bytes
49 that form the hole in the centre of a standard page. Thus, when we read the
50 block back from storage we implicitly check that the hole is still all zeroes.
51 We do this to ensure that we spot errors that could have destroyed data even
52 if they haven't actually done so. Full page images stored in WAL do *not*
53 check that the hole is all zero; the data in the hole is simply skipped and
54 re-zeroed if the backup block is reapplied. We do this because a failure in
55 WAL is a fatal error and prevents further recovery, whereas a checksum failure
56 on a normal data block is a hard error but not a critical one for the server,
57 even if it is a very bad thing for the user.
58
59 New WAL records cannot be written during recovery, so hint bits set during
60 recovery must not dirty the page if the buffer is not already dirty, when
61 checksums are enabled. Systems in Hot-Standby mode may benefit from hint bits
62 being set, but with checksums enabled, a page cannot be dirtied after setting a
63 hint bit (due to the torn page risk). So, it must wait for full-page images
64 containing the hint bit updates to arrive from the primary.