Debian Codegraph @

upstream/fedora-44/man5/btrfs.5

raw
1 '\" t
2 .\" Man page generated from reStructuredText
3 .\" by the Docutils 0.22.4 manpage writer.
4 .
5 .
6 .nr rst2man-indent-level 0
7 .
8 .de1 rstReportMargin
9 \\$1 \\n[an-margin]
10 level \\n[rst2man-indent-level]
11 level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
12 -
13 \\n[rst2man-indent0]
14 \\n[rst2man-indent1]
15 \\n[rst2man-indent2]
16 ..
17 .de1 INDENT
18 .\" .rstReportMargin pre:
19 . RS \\$1
20 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
21 . nr rst2man-indent-level +1
22 .\" .rstReportMargin post:
23 ..
24 .de UNINDENT
25 . RE
26 .\" indent \\n[an-margin]
27 .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
28 .nr rst2man-indent-level -1
29 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
30 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
31 ..
32 .TH "BTRFS" "5" "Mar 18, 2026" "6.19.1" "BTRFS"
33 .SH NAME
34 btrfs \- topics about the BTRFS filesystem (mount options, supported file attributes and other)
35 .SH DESCRIPTION
36 .sp
37 This document describes topics related to BTRFS that are not specific to the
38 tools. Currently covers:
39 .INDENT 0.0
40 .IP 1. 4
41 mount options
42 .IP 2. 4
43 filesystem features
44 .IP 3. 4
45 swapfile support
46 .IP 4. 4
47 checksum algorithms
48 .IP 5. 4
49 compression
50 .IP 6. 4
51 sysfs interface
52 .IP 7. 4
53 filesystem exclusive operations
54 .IP 8. 4
55 filesystem limits
56 .IP 9. 4
57 bootloader support
58 .IP 10. 4
59 file attributes
60 .IP 11. 4
61 zoned mode
62 .IP 12. 4
63 control device
64 .IP 13. 4
65 filesystems with multiple block group profiles
66 .IP 14. 4
67 seeding device
68 .IP 15. 4
69 RAID56 status and recommended practices
70 .IP 16. 4
71 glossary
72 .IP 17. 4
73 storage model, hardware considerations
74 .UNINDENT
75 .SH MOUNT OPTIONS
76 .SS BTRFS SPECIFIC MOUNT OPTIONS
77 .sp
78 This section describes mount options specific to BTRFS. For the generic mount
79 options please refer to mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> manual page and also see the section
80 with BTRFS specifics below \%<btrfs-man5//#\:mount-options-generic>\&. The options are
81 sorted alphabetically (discarding the \fIno\fP prefix).
82 .sp
83 \fBNote:\fP
84 .INDENT 0.0
85 .INDENT 3.5
86 Most mount options apply to the whole filesystem and only options in the
87 first mounted subvolume will take effect. This is due to lack of implementation
88 and may change in the future. This means that (for example) you can\(aqt set
89 per\-subvolume \fInodatacow\fP, \fInodatasum\fP, or \fIcompress\fP using mount options. This
90 should eventually be fixed, but it has proved to be difficult to implement
91 correctly within the Linux VFS framework.
92 .UNINDENT
93 .UNINDENT
94 .sp
95 Mount options are processed in order, only the last occurrence of an option
96 takes effect and may disable other options due to constraints (see e.g.
97 \fInodatacow\fP and \fIcompress\fP). The output of \fBmount\fP command shows which options
98 have been applied.
99 .INDENT 0.0
100 .TP
101 .B acl, noacl
102 (default: on)
103 .sp
104 Enable/disable support for POSIX Access Control Lists (ACLs). See the
105 acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html> manual page for more information about ACLs.
106 .sp
107 The support for ACL is build\-time configurable (BTRFS_FS_POSIX_ACL) and
108 mount fails if \fIacl\fP is requested but the feature is not compiled in.
109 .UNINDENT
110 .INDENT 0.0
111 .TP
112 .B autodefrag, noautodefrag
113 (since: 3.0, default: off)
114 .sp
115 Enable automatic file defragmentation.
116 When enabled, small random writes into files (in a range of tens of kilobytes,
117 currently it\(aqs 64KiB) are detected and queued up for the defragmentation process.
118 May not be well suited for large database workloads.
119 .sp
120 The read latency may increase due to reading the adjacent blocks that make up the
121 range for defragmentation, successive write will merge the blocks in the new
122 location.
123 .sp
124 \fBWarning:\fP
125 .INDENT 7.0
126 .INDENT 3.5
127 Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14\-rc2 as
128 well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or
129 ≥ 3.13.4 will break up the reflinks of COW data (for example files
130 copied with \fBcp \-\-reflink\fP, snapshots or de\-duplicated data).
131 This may cause considerable increase of space usage depending on the
132 broken up reflinks.
133 .UNINDENT
134 .UNINDENT
135 .TP
136 .B barrier, nobarrier
137 (default: on)
138 .sp
139 Ensure that all IO write operations make it through the device cache and are stored
140 permanently when the filesystem is at its consistency checkpoint. This
141 typically means that a flush command is sent to the device that will
142 synchronize all pending data and ordinary metadata blocks, then writes the
143 superblock and issues another flush.
144 .sp
145 The write flushes incur a slight hit and also prevent the IO block
146 scheduler to reorder requests in a more effective way. Disabling barriers gets
147 rid of that penalty but will most certainly lead to a corrupted filesystem in
148 case of a crash or power loss. The ordinary metadata blocks could be yet
149 unwritten at the time the new superblock is stored permanently, expecting that
150 the block pointers to metadata were stored permanently before.
151 .sp
152 On a device with a volatile battery\-backed write\-back cache, the \fInobarrier\fP
153 option will not lead to filesystem corruption as the pending blocks are
154 supposed to make it to the permanent storage.
155 .TP
156 .B clear_cache
157 Force clearing and rebuilding of the free space cache if something
158 has gone wrong.
159 This option only takes effect on the first read\-write mount.
160 .sp
161 For free space cache \fIv1\fP, this only clears (and, unless \fInospace_cache\fP is
162 used, rebuilds) the free space cache for block groups that are modified while
163 the filesystem is mounted with that option. To actually clear an entire free
164 space cache \fIv1\fP, see \fBbtrfs check \-\-clear\-space\-cache v1\fP\&.
165 .sp
166 For free space cache \fIv2\fP, this clears the entire free space cache.
167 To do so without requiring to mounting the filesystem, see
168 \fBbtrfs check \-\-clear\-space\-cache v2\fP\&.
169 .sp
170 See also: \fIspace_cache\fP\&.
171 .TP
172 .B commit=<seconds>
173 (since: 3.12, default: 30)
174 .sp
175 Set the interval of periodic transaction commit when data are synchronized
176 to permanent storage. Higher interval values lead to larger amount of unwritten
177 data to accumulate in memory, which has obvious consequences when the
178 system crashes. The upper bound is not forced, but a warning is
179 printed if it\(aqs more than 300 seconds (5 minutes). Use with care.
180 .sp
181 The periodic commit is not the only mechanism to do the transaction commit,
182 this can also happen by explicit \fBsync\fP or indirectly by other
183 commands that affect the global filesystem state or internal kernel
184 mechanisms that flush based on various thresholds or policies (e.g. cgroups).
185 .TP
186 .B compress, compress=<type[:level]>, compress\-force, compress\-force=<type[:level]>
187 (default: off, level support since: 5.1)
188 .sp
189 Control BTRFS file data compression. Type may be specified as \fIzlib\fP,
190 \fIlzo\fP, \fIzstd\fP or \fIno\fP (for no compression, used for remounting). If no type
191 is specified, \fIzlib\fP is used. If \fIcompress\-force\fP is specified,
192 then compression will always be attempted, but the data may end up uncompressed
193 if the compression would make them larger.
194 .sp
195 Both \fIzlib\fP and \fIzstd\fP (since version 5.1) expose the compression level as a
196 tunable knob with higher levels trading speed and memory (\fIzstd\fP) for higher
197 compression ratios. This can be set by appending a colon and the desired level.
198 ZLIB accepts the range [1, 9] and ZSTD accepts [1, 15]. If no level is set,
199 both currently use a default level of 3. The value 0 is an alias for the
200 default level.
201 .sp
202 Otherwise some simple heuristics are applied to detect an incompressible file.
203 If the first blocks written to a file are not compressible, the whole file is
204 permanently marked to skip compression. As this is too simple, the
205 \fIcompress\-force\fP is a workaround that will compress most of the files at the
206 cost of some wasted CPU cycles on failed attempts.
207 Since kernel 4.15, a set of heuristic algorithms have been improved by using
208 frequency sampling, repeated pattern detection and Shannon entropy calculation
209 to avoid that.
210 .sp
211 \fBNote:\fP
212 .INDENT 7.0
213 .INDENT 3.5
214 If compression is enabled, \fInodatacow\fP and \fInodatasum\fP are disabled.
215 .UNINDENT
216 .UNINDENT
217 .TP
218 .B datacow, nodatacow
219 (default: on)
220 .sp
221 Enable data copy\-on\-write for newly created files.
222 \fINodatacow\fP implies \fInodatasum\fP, and disables \fIcompression\fP\&. All files created
223 under \fInodatacow\fP are also set the NOCOW file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>).
224 .sp
225 \fBNote:\fP
226 .INDENT 7.0
227 .INDENT 3.5
228 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
229 .UNINDENT
230 .UNINDENT
231 .sp
232 Updates in\-place improve performance for workloads that do frequent overwrites,
233 at the cost of potential partial writes, in case the write is interrupted
234 (system crash, device failure).
235 .TP
236 .B datasum, nodatasum
237 (default: on)
238 .sp
239 Enable data checksumming for newly created files.
240 \fIDatasum\fP implies \fIdatacow\fP, i.e. the normal mode of operation. All files created
241 under \fInodatasum\fP inherit the \(dqno checksums\(dq property, however there\(aqs no
242 corresponding file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>).
243 .sp
244 \fBNote:\fP
245 .INDENT 7.0
246 .INDENT 3.5
247 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
248 .UNINDENT
249 .UNINDENT
250 .sp
251 There is a slight performance gain when checksums are turned off, the
252 corresponding metadata blocks holding the checksums do not need to updated.
253 The cost of checksumming of the blocks in memory is much lower than the IO,
254 modern CPUs feature hardware support of the checksumming algorithm.
255 .UNINDENT
256 .INDENT 0.0
257 .TP
258 .B degraded
259 (default: off)
260 .sp
261 Allow mounts with fewer devices than the RAID profile constraints
262 require. A read\-write mount (or remount) may fail when there are too many devices
263 missing, for example if a stripe member is completely missing from RAID0.
264 .sp
265 Since 4.14, the constraint checks have been improved and are verified on the
266 chunk level, not at the device level. This allows degraded mounts of
267 filesystems with mixed RAID profiles for data and metadata, even if the
268 device number constraints would not be satisfied for some of the profiles.
269 .sp
270 Example: metadata \-\- raid1, data \-\- single, devices \-\- \fB/dev/sda\fP, \fB/dev/sdb\fP
271 .sp
272 Suppose the data are completely stored on \fIsda\fP, then missing \fIsdb\fP will not
273 prevent the mount, even if 1 missing device would normally prevent (any)
274 \fIsingle\fP profile to mount. In case some of the data chunks are stored on \fIsdb\fP,
275 then the constraint of single/data is not satisfied and the filesystem
276 cannot be mounted.
277 .UNINDENT
278 .INDENT 0.0
279 .TP
280 .B device=<devicepath>
281 Specify a path to a device that will be scanned for BTRFS filesystem during
282 mount. This is usually done automatically by a device manager (like udev) or
283 using the \fBbtrfs device scan\fP command (e.g. run from the initial ramdisk). In
284 cases where this is not possible the \fIdevice\fP mount option can help.
285 .sp
286 \fBNote:\fP
287 .INDENT 7.0
288 .INDENT 3.5
289 Booting e.g. a RAID1 system may fail even if all filesystem\(aqs \fIdevice\fP
290 paths are provided as the actual device nodes may not be discovered by the
291 system at that point.
292 .UNINDENT
293 .UNINDENT
294 .TP
295 .B discard, discard=sync, discard=async, nodiscard
296 (default: async when devices support it since 6.2, async support since: 5.6)
297 .sp
298 Enable discarding of freed file blocks. This is useful for SSD/NVMe
299 devices, thinly provisioned LUNs, or virtual machine images; however,
300 every storage layer must support discard for it to work.
301 .sp
302 In the synchronous mode (\fIsync\fP or without option value), lack of asynchronous
303 queued TRIM on the backing device TRIM can severely degrade performance,
304 because a synchronous TRIM operation will be attempted instead. Queued TRIM
305 requires SATA devices with chipsets revision newer than 3.1 and devices.
306 .sp
307 The asynchronous mode (\fIasync\fP) gathers extents in larger chunks before sending
308 them to the devices for TRIM. The overhead and performance impact should be
309 negligible compared to the previous mode and it\(aqs supposed to be the preferred
310 mode if needed.
311 .sp
312 If it is not necessary to immediately discard freed blocks, then the \fBfstrim\fP
313 tool can be used to discard all free blocks in a batch. Scheduling a TRIM
314 during a period of low system activity will prevent latent interference with
315 the performance of other operations. Also, a device may ignore the TRIM command
316 if the range is too small, so running a batch discard has a greater probability
317 of actually discarding the blocks.
318 .TP
319 .B enospc_debug, noenospc_debug
320 (default: off)
321 .sp
322 Enable verbose output for some ENOSPC conditions. It\(aqs safe to use but can
323 be noisy if the system reaches near\-full state.
324 .TP
325 .B fatal_errors=<action>
326 (since: 3.4, default: bug)
327 .sp
328 Action to take when encountering a fatal error.
329 .INDENT 7.0
330 .TP
331 .B bug
332 \fIBUG()\fP on a fatal error, the system will stay in the crashed state and may be
333 still partially usable, but reboot is required for full operation
334 .TP
335 .B panic
336 \fIpanic()\fP on a fatal error, depending on other system configuration, this may
337 be followed by a reboot. Please refer to the documentation of kernel boot
338 parameters, e.g. \fIpanic\fP, \fIoops\fP or \fIcrashkernel\fP\&.
339 .UNINDENT
340 .TP
341 .B flushoncommit, noflushoncommit
342 (default: off)
343 .sp
344 This option forces any data dirtied by a write in a prior transaction to commit
345 as part of the current commit, effectively a full filesystem sync.
346 .sp
347 This makes the committed state a fully consistent view of the file system from
348 the application\(aqs perspective (i.e. it includes all completed file system
349 operations). This was previously the behavior only when a snapshot was
350 created.
351 .sp
352 When off, the filesystem is consistent but buffered writes may last more than
353 one transaction commit.
354 .TP
355 .B fragment=<type>
356 (depends on compile\-time option CONFIG_BTRFS_DEBUG, since: 4.4, default: off)
357 .sp
358 A debugging helper to intentionally fragment given \fItype\fP of block groups. The
359 type can be \fIdata\fP, \fImetadata\fP or \fIall\fP\&. This mount option should not be used
360 outside of debugging environments and is not recognized if the kernel config
361 option \fICONFIG_BTRFS_DEBUG\fP is not enabled.
362 .TP
363 .B nologreplay
364 (default: off, even read\-only)
365 .sp
366 The tree\-log contains pending updates to the filesystem until the full commit.
367 The log is replayed on next mount, this can be disabled by this option. See
368 also \fItreelog\fP\&. Note that \fInologreplay\fP is the same as \fInorecovery\fP\&.
369 .sp
370 \fBWarning:\fP
371 .INDENT 7.0
372 .INDENT 3.5
373 Currently, the tree log is replayed even with a read\-only mount! To
374 disable that behaviour, mount also with \fInologreplay\fP\&.
375 .UNINDENT
376 .UNINDENT
377 .TP
378 .B max_inline=<bytes>
379 (default: min(2048, page size) )
380 .sp
381 Specify the maximum amount of space, that can be inlined in
382 a metadata b\-tree leaf. The value is specified in bytes, optionally
383 with a K suffix (case insensitive). In practice, this value
384 is limited by the filesystem block size (named \fIsectorsize\fP at mkfs time),
385 and memory page size of the system. In case of sectorsize limit, there\(aqs
386 some space unavailable due to b\-tree leaf headers. For example, a 4KiB
387 sectorsize, maximum size of inline data is about 3900 bytes.
388 .sp
389 Inlining can be completely turned off by specifying 0. This will increase data
390 block slack if file sizes are much smaller than block size but will reduce
391 metadata consumption in return.
392 .sp
393 \fBNote:\fP
394 .INDENT 7.0
395 .INDENT 3.5
396 The default value has changed to 2048 in kernel 4.6.
397 .UNINDENT
398 .UNINDENT
399 .TP
400 .B metadata_ratio=<value>
401 (default: 0, internal logic)
402 .sp
403 Specifies that 1 metadata chunk should be allocated after every \fIvalue\fP data
404 chunks. Default behaviour depends on internal logic, some percent of unused
405 metadata space is attempted to be maintained but is not always possible if
406 there\(aqs not enough space left for chunk allocation. The option could be useful to
407 override the internal logic in favor of the metadata allocation if the expected
408 workload is supposed to be metadata intense (snapshots, reflinks, xattrs,
409 inlined files).
410 .TP
411 .B norecovery
412 (since: 4.5, default: off)
413 .sp
414 Do not attempt any data recovery at mount time. This will disable \fIlogreplay\fP
415 and avoids other write operations. Note that this option is the same as
416 \fInologreplay\fP\&.
417 .sp
418 \fBNote:\fP
419 .INDENT 7.0
420 .INDENT 3.5
421 The opposite option \fIrecovery\fP used to have different meaning but was
422 changed for consistency with other filesystems, where \fInorecovery\fP is used for
423 skipping log replay. BTRFS does the same and in general will try to avoid any
424 write operations.
425 .UNINDENT
426 .UNINDENT
427 .UNINDENT
428 .INDENT 0.0
429 .TP
430 .B rescan_uuid_tree
431 (since: 3.12, default: off)
432 .sp
433 Force check and rebuild procedure of the UUID tree. This should not
434 normally be needed. Alternatively the tree can be cleared from
435 userspace by command btrfs rescue clear\-uuid\-tree \%<#\:man-rescue-clear-uuid-tree>
436 and then it will be automatically rebuilt in kernel (the mount option
437 is not needed in that case).
438 .TP
439 .B rescue
440 (since: 5.9)
441 .sp
442 Modes allowing mount with damaged filesystem structures, all requires
443 the filesystem to be mounted read\-only and doesn\(aqt allow remount to read\-write.
444 This is supposed to provide unified and more fine grained tuning of
445 errors that affect filesystem operation.
446 .INDENT 7.0
447 .IP \(bu 2
448 \fIusebackuproot\fP (since 5.9)
449 .sp
450 Try to use backup root slots inside super block.
451 Replaces standalone option \fIusebackuproot\fP
452 .IP \(bu 2
453 \fInologreplay\fP (since 5.9)
454 .sp
455 Do not replay any dirty logs.
456 Replaces standalone option \fInologreplay\fP
457 .IP \(bu 2
458 \fIignorebadroots\fP, \fIibadroots\fP (since: 5.11)
459 .sp
460 Ignore bad tree roots, greatly improve the chance for data salvage.
461 .IP \(bu 2
462 \fIignoredatacsums\fP, \fIidatacsums\fP (since: 5.11)
463 .sp
464 Ignore data checksum verification.
465 .IP \(bu 2
466 \fIignoremetacsums\fP, \fIimetacsums\fP (since 6.12)
467 .sp
468 Ignore metadata checksum verification, useful for interrupted checksum conversion.
469 .IP \(bu 2
470 \fIall\fP (since: 5.9)
471 .sp
472 Enable all supported rescue options.
473 .UNINDENT
474 .TP
475 .B skip_balance
476 (since: 3.3, default: off)
477 .sp
478 Skip automatic resume of an interrupted balance operation. The operation can
479 later be resumed with \fBbtrfs balance resume\fP, or the paused state can be
480 removed with \fBbtrfs balance cancel\fP\&. The default behaviour is to resume an
481 interrupted balance immediately after the filesystem is mounted.
482 .TP
483 .B space_cache, space_cache=<version>, nospace_cache
484 (\fInospace_cache\fP since: 3.2, \fIspace_cache=v1\fP and \fIspace_cache=v2\fP since 4.5, default: \fIspace_cache=v2\fP)
485 .sp
486 Options to control the free space cache. The free space cache greatly improves
487 performance when reading block group free space into memory. However, managing
488 the space cache consumes some resources, including a small amount of disk
489 space.
490 .sp
491 There are two implementations of the free space cache. The original
492 one, referred to as \fIv1\fP, used to be a safe default but has been
493 superseded by \fIv2\fP\&. The \fIv1\fP space cache can be disabled at mount time
494 with \fInospace_cache\fP without clearing.
495 .sp
496 On very large filesystems (many terabytes) and certain workloads, the
497 performance of the \fIv1\fP space cache may degrade drastically. The \fIv2\fP
498 implementation, which adds a new b\-tree called the free space tree, addresses
499 this issue. Once enabled, the \fIv2\fP space cache will always be used and cannot
500 be disabled unless it is cleared. Use \fIclear_cache,space_cache=v1\fP or
501 \fIclear_cache,nospace_cache\fP to do so. If \fIv2\fP is enabled, and \fIv1\fP space
502 cache will be cleared (at the first mount) and kernels without \fIv2\fP
503 support will only be able to mount the filesystem in read\-only mode.
504 On an unmounted filesystem the caches (both versions) can be cleared by
505 \(dqbtrfs check \-\-clear\-space\-cache\(dq.
506 .sp
507 The btrfs\-check(8) \%<> and \fI:doc:\(gamkfs.btrfs\fP commands have full \fIv2\fP free space
508 cache support since v4.19.
509 .sp
510 If a version is not explicitly specified, the default implementation will be
511 chosen, which is \fIv2\fP\&.
512 .TP
513 .B ssd, ssd_spread, nossd, nossd_spread
514 (default: SSD autodetected)
515 .sp
516 Options to control SSD allocation schemes. By default, BTRFS will
517 enable or disable SSD optimizations depending on status of a device with
518 respect to rotational or non\-rotational type. This is determined by the
519 contents of \fB/sys/block/DEV/queue/rotational\fP). If it is 0, the
520 \fIssd\fP option is turned on. The option \fInossd\fP will disable the
521 autodetection.
522 .sp
523 The optimizations make use of the absence of the seek penalty that\(aqs inherent
524 for the rotational devices. The blocks can be typically written faster and
525 are not offloaded to separate threads.
526 .sp
527 \fBNote:\fP
528 .INDENT 7.0
529 .INDENT 3.5
530 Since 4.14, the block layout optimizations have been dropped. This used
531 to help with first generations of SSD devices. Their FTL (flash translation
532 layer) was not effective and the optimization was supposed to improve the wear
533 by better aligning blocks. This is no longer true with modern SSD devices and
534 the optimization had no real benefit. Furthermore it caused increased
535 fragmentation. The layout tuning has been kept intact for the option
536 \fIssd_spread\fP\&.
537 .UNINDENT
538 .UNINDENT
539 .sp
540 The \fIssd_spread\fP mount option attempts to allocate into bigger and aligned
541 chunks of unused space, and may perform better on low\-end SSDs. \fIssd_spread\fP
542 implies \fIssd\fP, enabling all other SSD heuristics as well. The option \fInossd\fP
543 will disable all SSD options while \fInossd_spread\fP only disables \fIssd_spread\fP\&.
544 .TP
545 .B subvol=<path>
546 Mount subvolume from \fIpath\fP rather than the toplevel subvolume. The
547 \fIpath\fP is always treated as relative to the toplevel subvolume.
548 This mount option overrides the default subvolume set for the given filesystem.
549 .TP
550 .B subvolid=<subvolid>
551 Mount subvolume specified by a \fIsubvolid\fP number rather than the toplevel
552 subvolume. You can use \fBbtrfs subvolume list\fP of \fBbtrfs subvolume show\fP to see
553 subvolume ID numbers.
554 This mount option overrides the default subvolume set for the given filesystem.
555 .sp
556 \fBNote:\fP
557 .INDENT 7.0
558 .INDENT 3.5
559 If both \fIsubvolid\fP and \fIsubvol\fP are specified, they must point at the
560 same subvolume, otherwise the mount will fail.
561 .UNINDENT
562 .UNINDENT
563 .TP
564 .B thread_pool=<number>
565 (default: min(NRCPUS + 2, 8) )
566 .sp
567 The number of worker threads to start. NRCPUS is number of on\-line CPUs
568 detected at the time of mount. Small number leads to less parallelism in
569 processing data and metadata, higher numbers could lead to a performance hit
570 due to increased locking contention, process scheduling, cache\-line bouncing or
571 costly data transfers between local CPU memories.
572 .TP
573 .B treelog, notreelog
574 (default: on)
575 .sp
576 Enable the tree logging used for \fIfsync\fP and \fIO_SYNC\fP writes. The tree log
577 stores changes without the need of a full filesystem sync. The log operations
578 are flushed at sync and transaction commit. If the system crashes between two
579 such syncs, the pending tree log operations are replayed during mount.
580 .sp
581 \fBWarning:\fP
582 .INDENT 7.0
583 .INDENT 3.5
584 Currently, the tree log is replayed even with a read\-only mount! To
585 disable that behaviour, also mount with \fInologreplay\fP\&.
586 .UNINDENT
587 .UNINDENT
588 .sp
589 The tree log could contain new files/directories, these would not exist on
590 a mounted filesystem if the log is not replayed.
591 .TP
592 .B usebackuproot
593 (since: 4.6, default: off)
594 .sp
595 Enable autorecovery attempts if a bad tree root is found at mount time.
596 Currently this scans a backup list of several previous tree roots and tries to
597 use the first readable. This can be used with read\-only mounts as well.
598 .sp
599 \fBNote:\fP
600 .INDENT 7.0
601 .INDENT 3.5
602 This option has replaced \fIrecovery\fP which has been deprecated.
603 .UNINDENT
604 .UNINDENT
605 .TP
606 .B user_subvol_rm_allowed
607 (default: off)
608 .sp
609 Allow subvolumes to be deleted by their respective owner. Otherwise, only the
610 root user can do that.
611 .sp
612 \fBNote:\fP
613 .INDENT 7.0
614 .INDENT 3.5
615 Historically, any user could create a snapshot even if he was not owner
616 of the source subvolume, the subvolume deletion has been restricted for that
617 reason. The subvolume creation has been restricted but this mount option is
618 still required. This is a usability issue.
619 Since 4.18, the rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an
620 ordinary directory. Whether this is possible can be detected at runtime, see
621 \fIrmdir_subvol\fP feature in \fIFILESYSTEM FEATURES\fP\&.
622 .UNINDENT
623 .UNINDENT
624 .UNINDENT
625 .SS DEPRECATED MOUNT OPTIONS
626 .sp
627 List of mount options that have been removed, kept for backward compatibility.
628 .INDENT 0.0
629 .TP
630 .B recovery
631 (since: 3.2, default: off, deprecated since: 4.5)
632 .sp
633 \fBNote:\fP
634 .INDENT 7.0
635 .INDENT 3.5
636 This option has been replaced by \fIusebackuproot\fP and should not be used
637 but will work on 4.5+ kernels.
638 .UNINDENT
639 .UNINDENT
640 .TP
641 .B inode_cache, noinode_cache
642 (removed in: 5.11, since: 3.0, default: off)
643 .sp
644 \fBNote:\fP
645 .INDENT 7.0
646 .INDENT 3.5
647 The functionality has been removed in 5.11, any stale data created by
648 previous use of the \fIinode_cache\fP option can be removed by
649 btrfs rescue clear\-ino\-cache \%<#\:man-rescue-clear-ino-cache>\&.
650 .UNINDENT
651 .UNINDENT
652 .TP
653 .B check_int, check_int_data, check_int_print_mask=<value>
654 (removed in: 6.7, since: 3.0, default: off)
655 .sp
656 These debugging options control the behavior of the integrity checking
657 module (the BTRFS_FS_CHECK_INTEGRITY config option required). The main goal is
658 to verify that all blocks from a given transaction period are properly linked.
659 .sp
660 \fIcheck_int\fP enables the integrity checker module, which examines all
661 block write requests to ensure on\-disk consistency, at a large
662 memory and CPU cost.
663 .sp
664 \fIcheck_int_data\fP includes extent data in the integrity checks, and
665 implies the \fIcheck_int\fP option.
666 .sp
667 \fIcheck_int_print_mask\fP takes a bit mask of BTRFSIC_PRINT_MASK_* values
668 as defined in \fIfs/btrfs/check\-integrity.c\fP, to control the integrity
669 checker module behavior.
670 .sp
671 See comments at the top of \fIfs/btrfs/check\-integrity.c\fP
672 for more information.
673 .UNINDENT
674 .SS NOTES ON GENERIC MOUNT OPTIONS
675 .sp
676 Some of the general mount options from mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> that affect BTRFS and are
677 worth mentioning.
678 .INDENT 0.0
679 .TP
680 .B context
681 The context refers to the SELinux contexts and policy definitions passed
682 as mount options. This works properly since version v6.8 (because the
683 mount option parser of BTRFS was ported to new API that also understood
684 the options).
685 .TP
686 .B noatime
687 under read intensive work\-loads, specifying \fInoatime\fP significantly improves
688 performance because no new access time information needs to be written. Without
689 this option, the default is \fIrelatime\fP, which only reduces the number of
690 inode atime updates in comparison to the traditional \fIstrictatime\fP\&. The worst
691 case for atime updates under \fIrelatime\fP occurs when many files are read whose
692 atime is older than 24 h and which are freshly snapshotted. In that case the
693 atime is updated and COW happens \- for each file \- in bulk. See also
694 \%<https://\:lwn\:.net/\:Articles/\:499293/> \- \fIAtime and btrfs: a bad combination? (LWN, 2012\-05\-31)\fP\&.
695 .sp
696 Note that \fInoatime\fP may break applications that rely on atime uptimes like
697 the venerable Mutt (unless you use \fImaildir\fP mailboxes).
698 .UNINDENT
699 .SH FILESYSTEM FEATURES
700 .sp
701 The basic set of filesystem features gets extended over time. The backward
702 compatibility is maintained and the features are optional, need to be
703 explicitly asked for so accidental use will not create incompatibilities.
704 .sp
705 There are several classes and the respective tools to manage the features:
706 .INDENT 0.0
707 .TP
708 .B at mkfs time only
709 This is namely for core structures, like the b\-tree nodesize or checksum
710 algorithm, see mkfs.btrfs(8) \%<> for more details.
711 .TP
712 .B after mkfs, on an unmounted filesystem
713 Features that may optimize internal structures or add new structures to support
714 new functionality, see btrfstune(8) \%<>\&. The command
715 \fBbtrfs inspect\-internal dump\-super /dev/sdx\fP
716 will dump a superblock, you can map the value of
717 \fIincompat_flags\fP to the features listed below
718 .TP
719 .B after mkfs, on a mounted filesystem
720 The features of a filesystem (with a given UUID) are listed in
721 \fB/sys/fs/btrfs/UUID/features/\fP, one file per feature. The status is stored
722 inside the file. The value \fI1\fP is for enabled and active, while \fI0\fP means the
723 feature was enabled at mount time but turned off afterwards.
724 .sp
725 Whether a particular feature can be turned on a mounted filesystem can be found
726 in the directory \fB/sys/fs/btrfs/features/\fP, one file per feature. The value \fI1\fP
727 means the feature can be enabled.
728 .UNINDENT
729 .sp
730 List of features (see also mkfs.btrfs(8) \%<> section
731 FILESYSTEM FEATURES \%<#\:man-mkfs-filesystem-features>):
732 .INDENT 0.0
733 .TP
734 .B big_metadata
735 (since: 3.4)
736 .sp
737 the filesystem uses \fInodesize\fP for metadata blocks, this can be bigger than the
738 page size
739 .TP
740 .B block_group_tree
741 (since: 6.1)
742 .sp
743 block group item representation using a dedicated b\-tree, this can greatly
744 reduce mount time for large filesystems
745 .TP
746 .B compress_lzo
747 (since: 2.6.38)
748 .sp
749 the \fIlzo\fP compression has been used on the filesystem, either as a mount option
750 or via \fBbtrfs filesystem defrag\fP\&.
751 .TP
752 .B compress_zstd
753 (since: 4.14)
754 .sp
755 the \fIzstd\fP compression has been used on the filesystem, either as a mount option
756 or via \fBbtrfs filesystem defrag\fP\&.
757 .TP
758 .B default_subvol
759 (since: 2.6.34)
760 .sp
761 the default subvolume has been set on the filesystem
762 .TP
763 .B extended_iref
764 (since: 3.7)
765 .sp
766 increased hardlink limit per file in a directory to 65536, older kernels
767 supported a varying number of hardlinks depending on the sum of all file name
768 sizes that can be stored into one metadata block
769 .TP
770 .B free_space_tree
771 (since: 4.5)
772 .sp
773 free space representation using a dedicated b\-tree, successor of v1 space cache
774 .TP
775 .B metadata_uuid
776 (since: 5.0)
777 .sp
778 the main filesystem UUID is the metadata_uuid, which stores the new UUID only
779 in the superblock while all metadata blocks still have the UUID set at mkfs
780 time, see btrfstune(8) \%<> for more
781 .TP
782 .B mixed_backref
783 (since: 2.6.31)
784 .sp
785 the last major disk format change, improved backreferences, now default
786 .TP
787 .B mixed_groups
788 (since: 2.6.37)
789 .sp
790 mixed data and metadata block groups, i.e. the data and metadata are not
791 separated and occupy the same block groups, this mode is suitable for small
792 volumes as there are no constraints how the remaining space should be used
793 (compared to the split mode, where empty metadata space cannot be used for data
794 and vice versa)
795 .sp
796 on the other hand, the final layout is quite unpredictable and possibly highly
797 fragmented, which means worse performance
798 .TP
799 .B no_holes
800 (since: 3.14)
801 .sp
802 improved representation of file extents where holes are not explicitly
803 stored as an extent, saves a few percent of metadata if sparse files are used
804 .TP
805 .B raid1c34
806 (since: 5.5)
807 .sp
808 extended RAID1 mode with copies on 3 or 4 devices respectively
809 .TP
810 .B raid_stripe_tree
811 (since: 6.7)
812 .sp
813 a separate tree for tracking file extents on RAID profiles
814 .TP
815 .B RAID56
816 (since: 3.9)
817 .sp
818 the filesystem contains or contained a RAID56 profile of block groups
819 .TP
820 .B rmdir_subvol
821 (since: 4.18)
822 .sp
823 indicate that rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an
824 ordinary directory. Note that this feature only depends on the kernel version.
825 .TP
826 .B skinny_metadata
827 (since: 3.10)
828 .sp
829 reduced\-size metadata for extent references, saves a few percent of metadata
830 .TP
831 .B send_stream_version
832 (since: 5.10)
833 .sp
834 number of the highest supported send stream version
835 .TP
836 .B simple_quota
837 (since: 6.7)
838 .sp
839 simplified quota accounting
840 .TP
841 .B supported_checksums
842 (since: 5.5)
843 .sp
844 list of checksum algorithms supported by the kernel module, the respective
845 modules or built\-in implementing the algorithms need to be present to mount
846 the filesystem, see section CHECKSUM ALGORITHMS\&.
847 .TP
848 .B supported_sectorsizes
849 (since: 5.13)
850 .sp
851 list of values that are accepted as sector sizes (\fBmkfs.btrfs \-\-sectorsize\fP) by
852 the running kernel
853 .TP
854 .B supported_rescue_options
855 (since: 5.11)
856 .sp
857 list of values for the mount option \fIrescue\fP that are supported by the running
858 kernel, see btrfs(5) \%<>
859 .TP
860 .B zoned
861 (since: 5.12)
862 .sp
863 zoned mode is allocation/write friendly to host\-managed zoned devices,
864 allocation space is partitioned into fixed\-size zones that must be updated
865 sequentially, see section ZONED MODE
866 .UNINDENT
867 .SH SWAPFILE SUPPORT
868 .sp
869 A swapfile, when active, is a file\-backed swap area. It is supported since kernel 5.0.
870 Use swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html> to activate it, until then (respectively again after deactivating it
871 with swapoff(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapoff\:.8\:.html>) it\(aqs just a normal file (with NODATACOW set), for which the special
872 restrictions for active swapfiles don\(aqt apply.
873 .sp
874 There are some limitations of the implementation in BTRFS and Linux swap
875 subsystem:
876 .INDENT 0.0
877 .IP \(bu 2
878 filesystem \- must be only single device
879 .IP \(bu 2
880 filesystem \- must have only \fIsingle\fP data profile
881 .IP \(bu 2
882 subvolume \- cannot be snapshotted if it contains any active swapfiles
883 .IP \(bu 2
884 swapfile \- must be preallocated (i.e. no holes)
885 .IP \(bu 2
886 swapfile \- must be NODATACOW (i.e. also NODATASUM, no compression)
887 .UNINDENT
888 .sp
889 The limitations come namely from the COW\-based design and mapping layer of
890 blocks that allows the advanced features like relocation and multi\-device
891 filesystems. However, the swap subsystem expects simpler mapping and no
892 background changes of the file block location once they\(aqve been assigned to
893 swap. The constraints mentioned above (single device and single profile) are
894 related to the swapfile itself, i.e. the extents and their placement. It is
895 possible to create swapfile on multi\-device filesystem as long as the extents
896 are on one device but this cannot be affected by user and depends on free space
897 fragmentation and available unused space for new chunks.
898 .sp
899 With active swapfiles, the following whole\-filesystem operations will skip
900 swapfile extents or may fail:
901 .INDENT 0.0
902 .IP \(bu 2
903 balance \- block groups with extents of any active swapfiles are skipped and
904 reported, the rest will be processed normally
905 .IP \(bu 2
906 resize grow \- unaffected
907 .IP \(bu 2
908 resize shrink \- works as long as the extents of any active swapfiles are
909 outside of the shrunk range
910 .IP \(bu 2
911 device add \- if the new devices do not interfere with any already active swapfiles
912 this operation will work, though no new swapfile can be activated
913 afterwards
914 .IP \(bu 2
915 device delete \- if the device has been added as above, it can be also deleted
916 .IP \(bu 2
917 device replace \- ditto
918 .UNINDENT
919 .sp
920 When there are no active swapfiles and a whole\-filesystem exclusive operation
921 is running (e.g. balance, device delete, shrink), the swapfiles cannot be
922 temporarily activated. The operation must finish first.
923 .sp
924 To create and activate a swapfile run the following commands:
925 .INDENT 0.0
926 .INDENT 3.5
927 .sp
928 .EX
929 # truncate \-s 0 swapfile
930 # chattr +C swapfile
931 # fallocate \-l 2G swapfile
932 # chmod 0600 swapfile
933 # mkswap swapfile
934 # swapon swapfile
935 .EE
936 .UNINDENT
937 .UNINDENT
938 .sp
939 Since version 6.1 it\(aqs possible to create the swapfile in a single command
940 (except the activation):
941 .INDENT 0.0
942 .INDENT 3.5
943 .sp
944 .EX
945 # btrfs filesystem mkswapfile \-\-size 2G swapfile
946 # swapon swapfile
947 .EE
948 .UNINDENT
949 .UNINDENT
950 .sp
951 Please note that the UUID returned by the \fImkswap\fP utility identifies the swap
952 \(dqfilesystem\(dq and because it\(aqs stored in a file, it\(aqs not generally visible and
953 usable as an identifier unlike if it was on a block device.
954 .sp
955 Once activated the file will appear in \fB/proc/swaps\fP:
956 .INDENT 0.0
957 .INDENT 3.5
958 .sp
959 .EX
960 # cat /proc/swaps
961 Filename Type Size Used Priority
962 /path/swapfile file 2097152 0 \-2
963 .EE
964 .UNINDENT
965 .UNINDENT
966 .sp
967 The swapfile can be created as one\-time operation or, once properly created,
968 activated on each boot by the \fBswapon \-a\fP command (usually started by the
969 service manager). Add the following entry to \fI/etc/fstab\fP, assuming the
970 filesystem that provides the \fI/path\fP has been already mounted at this point.
971 Additional mount options relevant for the swapfile can be set too (like
972 priority, not the BTRFS mount options).
973 .INDENT 0.0
974 .INDENT 3.5
975 .sp
976 .EX
977 /path/swapfile none swap defaults 0 0
978 .EE
979 .UNINDENT
980 .UNINDENT
981 .sp
982 From now on the subvolume with the active swapfile cannot be snapshotted until
983 the swapfile is deactivated again by \fBswapoff\fP\&. Then the swapfile is a
984 regular file and the subvolume can be snapshotted again, though this would prevent
985 another activation any swapfile that has been snapshotted. New swapfiles (not
986 snapshotted) can be created and activated.
987 .sp
988 Otherwise, an inactive swapfile does not affect the containing subvolume. Activation
989 creates a temporary in\-memory status and prevents some file operations, but is
990 not stored permanently.
991 .SH HIBERNATION
992 .sp
993 A swapfile can be used for hibernation but it\(aqs not straightforward. Before
994 hibernation a resume offset must be written to file \fI/sys/power/resume_offset\fP
995 or the kernel command line parameter \fIresume_offset\fP must be set.
996 .sp
997 The value is the physical offset on the device. Note that \fBthis is not the same
998 value that\fP \fBfilefrag\fP \fBprints as physical offset!\fP
999 .sp
1000 Btrfs filesystem uses mapping between logical and physical addresses but here
1001 the physical can still map to one or more device\-specific physical block
1002 addresses. It\(aqs the device\-specific physical offset that is suitable as resume
1003 offset.
1004 .sp
1005 Since version 6.1 there\(aqs a command btrfs inspect\-internal map\-swapfile \%<#\:man-inspect-map-swapfile>
1006 that will print the device physical offset and the adjusted value for
1007 \fB/sys/power/resume_offset\fP\&. Note that the value is divided by page size, i.e.
1008 it\(aqs not the offset itself.
1009 .INDENT 0.0
1010 .INDENT 3.5
1011 .sp
1012 .EX
1013 # btrfs filesystem mkswapfile swapfile
1014 # btrfs inspect\-internal map\-swapfile swapfile
1015 Physical start: 811511726080
1016 Resume offset: 198122980
1017 .EE
1018 .UNINDENT
1019 .UNINDENT
1020 .sp
1021 For scripting and convenience the option \fI\-r\fP will print just the offset:
1022 .INDENT 0.0
1023 .INDENT 3.5
1024 .sp
1025 .EX
1026 # btrfs inspect\-internal map\-swapfile \-r swapfile
1027 198122980
1028 .EE
1029 .UNINDENT
1030 .UNINDENT
1031 .sp
1032 The command \fBmap\-swapfile\fP also verifies all the requirements, i.e. no holes,
1033 single device, etc.
1034 .SH TROUBLESHOOTING
1035 .sp
1036 If the swapfile activation fails please verify that you followed all the steps
1037 above or check the system log (e.g. \fBdmesg\fP or \fBjournalctl\fP) for more
1038 information.
1039 .sp
1040 Notably, the \fBswapon\fP utility exits with a message that does not say what
1041 failed:
1042 .INDENT 0.0
1043 .INDENT 3.5
1044 .sp
1045 .EX
1046 # swapon /path/swapfile
1047 swapon: /path/swapfile: swapon failed: Invalid argument
1048 .EE
1049 .UNINDENT
1050 .UNINDENT
1051 .sp
1052 The specific reason is likely to be printed to the system log by the btrfs
1053 module:
1054 .INDENT 0.0
1055 .INDENT 3.5
1056 .sp
1057 .EX
1058 # journalctl \-t kernel | grep swapfile
1059 kernel: BTRFS warning (device sda): swapfile must have single data profile
1060 .EE
1061 .UNINDENT
1062 .UNINDENT
1063 .SH CHECKSUM ALGORITHMS
1064 .sp
1065 Data and metadata are checksummed by default. The checksum is calculated before
1066 writing and verified after reading the blocks from devices. The whole metadata
1067 block has an inline checksum stored in the b\-tree node header. Each data block
1068 has a detached checksum stored in the checksum tree.
1069 .sp
1070 \fBNote:\fP
1071 .INDENT 0.0
1072 .INDENT 3.5
1073 Since a data checksum is calculated just before submitting to the block
1074 device, btrfs has a strong requirement that the corresponding data block must
1075 not be modified until the writeback is finished.
1076 .sp
1077 This requirement is met for a buffered write as btrfs has the full control on
1078 its page cache, but a direct write (\fBO_DIRECT\fP) bypasses page cache, and
1079 btrfs can not control the direct IO buffer (as it can be in user space memory).
1080 Thus it\(aqs possible that a user space program modifies its direct write buffer
1081 before the buffer is fully written back, and this can lead to a data
1082 checksum mismatch.
1083 .sp
1084 To avoid this, kernel starting with version 6.14 will force a direct
1085 write to fall back to buffered, if the inode requires a data checksum.
1086 This will bring a small performance penalty. If you require true zero\-copy
1087 direct writes, then set the \fBNODATASUM\fP flag for the inode and make
1088 sure the direct IO buffer is fully aligned to block size.
1089 .UNINDENT
1090 .UNINDENT
1091 .sp
1092 There are several checksum algorithms supported. The default and backward
1093 compatible algorithm is \fIcrc32c\fP\&. Since kernel 5.5 there are three more with different
1094 characteristics and trade\-offs regarding speed and strength. The following list
1095 may help you to decide which one to select.
1096 .INDENT 0.0
1097 .TP
1098 .B CRC32C (32 bits digest)
1099 Default, best backward compatibility. Very fast, modern CPUs have
1100 instruction\-level support, not collision\-resistant but still good error
1101 detection capabilities.
1102 .TP
1103 .B XXHASH (64 bits digest)
1104 Can be used as CRC32C successor. Very fast, optimized for modern CPUs utilizing
1105 instruction pipelining, good collision resistance and error detection.
1106 .TP
1107 .B SHA256 (256 bits digest)
1108 Cryptographic\-strength hash. Relatively slow but with possible CPU
1109 instruction acceleration or specialized hardware cards. FIPS certified and
1110 in wide use.
1111 .TP
1112 .B BLAKE2b (256 bits digest)
1113 Cryptographic\-strength hash. Relatively fast, with possible CPU acceleration
1114 using SIMD extensions. Not standardized but based on BLAKE which was a SHA3
1115 finalist, in wide use. The algorithm used is BLAKE2b\-256 that\(aqs optimized for
1116 64\-bit platforms.
1117 .UNINDENT
1118 .sp
1119 The \fIdigest size\fP affects overall size of data block checksums stored in the
1120 filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so
1121 there\(aqs no increase. Each data block has a separate checksum stored, with
1122 additional overhead of the b\-tree leaves.
1123 .sp
1124 Approximate relative performance of the algorithms, measured against CRC32C
1125 using implementations on a 11th gen 3.6GHz intel CPU:
1126 .TS
1127 box center;
1128 l|l|l|l.
1129 T{
1130 Digest
1131 T} T{
1132 Cycles/4KiB
1133 T} T{
1134 Ratio
1135 T} T{
1136 Implementation
1137 T}
1138 _
1139 T{
1140 CRC32C
1141 T} T{
1142 470
1143 T} T{
1144 1.00
1145 T} T{
1146 CPU instruction, PCL combination
1147 T}
1148 _
1149 T{
1150 XXHASH
1151 T} T{
1152 870
1153 T} T{
1154 1.9
1155 T} T{
1156 reference impl.
1157 T}
1158 _
1159 T{
1160 SHA256
1161 T} T{
1162 7600
1163 T} T{
1164 16
1165 T} T{
1166 libgcrypt
1167 T}
1168 _
1169 T{
1170 SHA256
1171 T} T{
1172 8500
1173 T} T{
1174 18
1175 T} T{
1176 openssl
1177 T}
1178 _
1179 T{
1180 SHA256
1181 T} T{
1182 8700
1183 T} T{
1184 18
1185 T} T{
1186 botan
1187 T}
1188 _
1189 T{
1190 SHA256
1191 T} T{
1192 32000
1193 T} T{
1194 68
1195 T} T{
1196 builtin, CPU instruction
1197 T}
1198 _
1199 T{
1200 SHA256
1201 T} T{
1202 37000
1203 T} T{
1204 78
1205 T} T{
1206 libsodium
1207 T}
1208 _
1209 T{
1210 SHA256
1211 T} T{
1212 78000
1213 T} T{
1214 166
1215 T} T{
1216 builtin, reference impl.
1217 T}
1218 _
1219 T{
1220 BLAKE2b
1221 T} T{
1222 10000
1223 T} T{
1224 21
1225 T} T{
1226 builtin/AVX2
1227 T}
1228 _
1229 T{
1230 BLAKE2b
1231 T} T{
1232 10900
1233 T} T{
1234 23
1235 T} T{
1236 libgcrypt
1237 T}
1238 _
1239 T{
1240 BLAKE2b
1241 T} T{
1242 13500
1243 T} T{
1244 29
1245 T} T{
1246 builtin/SSE41
1247 T}
1248 _
1249 T{
1250 BLAKE2b
1251 T} T{
1252 13700
1253 T} T{
1254 29
1255 T} T{
1256 libsodium
1257 T}
1258 _
1259 T{
1260 BLAKE2b
1261 T} T{
1262 14100
1263 T} T{
1264 30
1265 T} T{
1266 openssl
1267 T}
1268 _
1269 T{
1270 BLAKE2b
1271 T} T{
1272 14500
1273 T} T{
1274 31
1275 T} T{
1276 kcapi
1277 T}
1278 _
1279 T{
1280 BLAKE2b
1281 T} T{
1282 14500
1283 T} T{
1284 34
1285 T} T{
1286 builtin, reference impl.
1287 T}
1288 .TE
1289 .sp
1290 Many kernels are configured with SHA256 as built\-in and not as a module.
1291 Up to kernel v6.15 the accelerated versions are however provided by the
1292 modules and must be loaded
1293 explicitly (\fBmodprobe sha256\fP) before mounting the filesystem to make use of
1294 them. You can check in \fB/sys/fs/btrfs/FSID/checksum\fP which one is used. If you
1295 see \fIsha256\-generic\fP, then you may want to unmount and mount the filesystem
1296 again. Changing that on a mounted filesystem is not possible.
1297 .sp
1298 Since kernel v6.16 the accelereated implementation is always used if available.
1299 .sp
1300 Check the file \fB/proc/crypto\fP, when the implementation is built\-in, you\(aqd find:
1301 .INDENT 0.0
1302 .INDENT 3.5
1303 .sp
1304 .EX
1305 name : sha256
1306 driver : sha256\-generic
1307 module : kernel
1308 priority : 100
1309 \&...
1310 .EE
1311 .UNINDENT
1312 .UNINDENT
1313 .sp
1314 While accelerated implementation is e.g.:
1315 .INDENT 0.0
1316 .INDENT 3.5
1317 .sp
1318 .EX
1319 name : sha256
1320 driver : sha256\-avx2
1321 module : sha256_ssse3
1322 priority : 170
1323 \&...
1324 .EE
1325 .UNINDENT
1326 .UNINDENT
1327 .SH COMPRESSION
1328 .sp
1329 Btrfs supports transparent file compression. There are three algorithms
1330 available: ZLIB, LZO and ZSTD (since v4.14), with various levels.
1331 The compression happens on the level of file extents and the algorithm is
1332 selected by file property, mount option or by a defrag command.
1333 You can have a single btrfs mount point that has some files that are
1334 uncompressed, some that are compressed with LZO, some with ZLIB, for instance
1335 (though you may not want it that way, it is supported).
1336 .sp
1337 Once the compression is set, all newly written data will be compressed, i.e.
1338 existing data are untouched. Data are split into smaller chunks (128KiB) before
1339 compression to make random rewrites possible without a high performance hit. Due
1340 to the increased number of extents the metadata consumption is higher. The
1341 chunks are compressed in parallel.
1342 .sp
1343 The algorithms can be characterized as follows regarding the speed/ratio
1344 trade\-offs:
1345 .INDENT 0.0
1346 .TP
1347 .B ZLIB
1348 .INDENT 7.0
1349 .IP \(bu 2
1350 slower, higher compression ratio
1351 .IP \(bu 2
1352 levels: 1 to 9, mapped directly, default level is 3
1353 .IP \(bu 2
1354 good backward compatibility
1355 .UNINDENT
1356 .TP
1357 .B LZO
1358 .INDENT 7.0
1359 .IP \(bu 2
1360 faster compression and decompression than ZLIB, worse compression ratio, designed to be fast
1361 .IP \(bu 2
1362 no levels
1363 .IP \(bu 2
1364 good backward compatibility
1365 .UNINDENT
1366 .TP
1367 .B ZSTD
1368 .INDENT 7.0
1369 .IP \(bu 2
1370 compression comparable to ZLIB with higher compression/decompression speeds and different ratio
1371 .IP \(bu 2
1372 levels: \-15..15, mapped directly, default is 3
1373 .IP \(bu 2
1374 support since 4.14
1375 .IP \(bu 2
1376 levels 1..15 supported since 5.1
1377 .IP \(bu 2
1378 levels \-15..\-1 supported since 6.15
1379 .UNINDENT
1380 .UNINDENT
1381 .sp
1382 The differences depend on the actual data set and cannot be expressed by a
1383 single number or recommendation. Higher levels consume more CPU time and may
1384 not bring a significant improvement, lower levels are close to real time.
1385 .SH HOW TO ENABLE COMPRESSION
1386 .sp
1387 Typically the compression can be enabled on the whole filesystem, specified for
1388 the mount point. Note that the compression mount options are shared among all
1389 mounts of the same filesystem, either bind mounts or subvolume mounts.
1390 Please refer to btrfs(5) \%<> section
1391 MOUNT OPTIONS\&.
1392 .INDENT 0.0
1393 .INDENT 3.5
1394 .sp
1395 .EX
1396 $ mount \-o compress=zstd /dev/sdx /mnt
1397 .EE
1398 .UNINDENT
1399 .UNINDENT
1400 .sp
1401 This will enable the \fBzstd\fP algorithm on the default level (which is 3).
1402 The level can be specified manually too like \fBzstd:3\fP\&. Higher levels compress
1403 better at the cost of time. This in turn may cause increased write latency, low
1404 levels are suitable for real\-time compression and on reasonably fast CPU don\(aqt
1405 cause noticeable performance drops.
1406 .INDENT 0.0
1407 .INDENT 3.5
1408 .sp
1409 .EX
1410 $ btrfs filesystem defrag \-czstd file
1411 .EE
1412 .UNINDENT
1413 .UNINDENT
1414 .sp
1415 The command above will start defragmentation of the whole \fIfile\fP and apply
1416 the compression, regardless of the mount option. The compression level can be
1417 also specified with the \fB\-\-level\fP or \fB\-L\fP argument as of version \fB6.14\fP\&.
1418 The compression algorithm is not persistent and applies only
1419 to the defragmentation command, for any other writes other compression settings
1420 apply.
1421 .sp
1422 Persistent settings on a per\-file basis can be set in two ways:
1423 .INDENT 0.0
1424 .INDENT 3.5
1425 .sp
1426 .EX
1427 $ chattr +c file
1428 $ btrfs property set file compression zstd
1429 .EE
1430 .UNINDENT
1431 .UNINDENT
1432 .sp
1433 The first command is using legacy interface of file attributes inherited from
1434 ext2 filesystem and is not flexible, so by default the \fIzlib\fP compression is
1435 set. The other command sets a property on the file with the given algorithm.
1436 (Note: setting level that way is not yet implemented.)
1437 .SH COMPRESSION LEVELS
1438 .sp
1439 The level support of ZLIB has been added in v4.14, LZO does not support levels
1440 (the kernel implementation provides only one), ZSTD level support has been added
1441 in v5.1 and the negative levels in v6.15.
1442 .sp
1443 There are 9 levels of ZLIB supported (1 to 9), mapping 1:1 from the mount option
1444 to the algorithm defined level. The default is level 3, which provides the
1445 reasonably good compression ratio and is still reasonably fast. The difference
1446 in compression gain of levels 7, 8 and 9 is comparable but the higher levels
1447 take longer.
1448 .sp
1449 The ZSTD support includes levels \-15..15, a subset of full range of what ZSTD
1450 provides. Levels \-15..\-1 are real\-time with worse compression ratio, levels
1451 1..3 are near real\-time with good compression, 4..8 are slower with improved
1452 compression and 9..15 try even harder though the resulting size may not be
1453 significantly improved. Higher levels also require more memory and as they need
1454 more CPU the system performance is affected.
1455 .sp
1456 Level 0 always maps to the default. The compression level does not affect
1457 compatibility.
1458 .SH EXCEPTIONS
1459 .sp
1460 Any file that has been touched by the \fIfallocate\fP system call will always be
1461 excepted from compression even if \fIforce\-compress\fP mount option is used.
1462 .sp
1463 The reason for this is that a successful \fIfallocate\fP call must guarantee that
1464 future writes to the allocated range will not fail because of lack of space.
1465 This is difficult to guarantee in a COW filesystem. To reduce the chances of
1466 it happening, btrfs preallocates space and disables compression for the file.
1467 .sp
1468 As a workaround, one can trigger a compressed rewrite for such a file using the
1469 \fIbtrfs defrag\fP command. Be aware that if the file is touched again by the
1470 \fIfallocate\fP system call, it will be excepted again from compression for all the
1471 new data written to it.
1472 .SH INCOMPRESSIBLE DATA
1473 .sp
1474 Files with already compressed data or with data that won\(aqt compress well with
1475 the CPU and memory constraints of the kernel implementations are using a simple
1476 decision logic. If the first portion of data being compressed is not smaller
1477 than the original, the compression of the whole file is disabled. Unless the
1478 filesystem is mounted with \fIcompress\-force\fP in which case btrfs will try
1479 compressing every block, falling back to storing the uncompressed version for
1480 each block that ends up larger after compression. This is not optimal and
1481 subject to optimizations and further development.
1482 .sp
1483 If a file is identified as incompressible, a flag is set (\fINOCOMPRESS\fP) and it\(aqs
1484 sticky. On that file compression won\(aqt be performed unless forced. The flag
1485 can be also set by \fBchattr +m\fP (since e2fsprogs 1.46.2) or by properties with
1486 value \fIno\fP or \fInone\fP\&. Empty value will reset it to the default that\(aqs currently
1487 applicable on the mounted filesystem.
1488 .sp
1489 There are two ways to detect incompressible data:
1490 .INDENT 0.0
1491 .IP \(bu 2
1492 actual compression attempt \- data are compressed, if the result is not smaller,
1493 it\(aqs discarded, so this depends on the algorithm and level
1494 .IP \(bu 2
1495 pre\-compression heuristics \- a quick statistical evaluation on the data is
1496 performed and based on the result either compression is performed or skipped,
1497 the NOCOMPRESS bit is not set just by the heuristic, only if the compression
1498 algorithm does not make an improvement
1499 .UNINDENT
1500 .INDENT 0.0
1501 .INDENT 3.5
1502 .sp
1503 .EX
1504 $ lsattr file
1505 \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-m file
1506 .EE
1507 .UNINDENT
1508 .UNINDENT
1509 .sp
1510 Using the forcing compression is not recommended, the heuristics are
1511 supposed to decide that and compression algorithms internally detect
1512 incompressible data too.
1513 .SH PRE-COMPRESSION HEURISTICS
1514 .sp
1515 The heuristics aim to do a few quick statistical tests on the compressed data
1516 in order to avoid probably costly compression that would turn out to be
1517 inefficient. Compression algorithms could have internal detection of
1518 incompressible data too but this leads to more overhead as the compression is
1519 done in another thread and has to write the data anyway. The heuristic is
1520 read\-only and can utilize cached memory.
1521 .sp
1522 The tests performed based on the following: data sampling, long repeated
1523 pattern detection, byte frequency, Shannon entropy.
1524 .SH COMPATIBILITY
1525 .sp
1526 Compression requires both data checksums and COW, so either \fInodatasum\fP or
1527 \fInodatasum\fP mount option/inode flag will result in no compression.
1528 .sp
1529 Direct IO reads of compressed data will always fallback to buffered reads.
1530 .sp
1531 Direct IO write behavior depends on the inode flag.
1532 For inodes with data checksum, direct IO writes always fallback to buffered
1533 writes, thus can generate compressed data if the mount option/inode flags
1534 allows that.
1535 .sp
1536 For inodes without data checksums, direct IO writes will not populate page cache,
1537 and since the inode has no data checksums, no compressed data will be generated
1538 anyway.
1539 .sp
1540 The compression algorithms have been added over time so the version
1541 compatibility should be also considered, together with other tools that may
1542 access the compressed data like bootloaders.
1543 .SH SYSFS INTERFACE
1544 .sp
1545 Btrfs has a sysfs interface to provide extra knobs.
1546 .sp
1547 The top level path is \fB/sys/fs/btrfs/\fP, and the main directory layout is the following:
1548 .TS
1549 box center;
1550 l|l|l.
1551 T{
1552 Relative Path
1553 T} T{
1554 Description
1555 T} T{
1556 Version
1557 T}
1558 _
1559 T{
1560 features/
1561 T} T{
1562 All supported features
1563 T} T{
1564 3.14
1565 T}
1566 _
1567 T{
1568 <UUID>/
1569 T} T{
1570 Mounted fs UUID
1571 T} T{
1572 3.14
1573 T}
1574 _
1575 T{
1576 <UUID>/allocation/
1577 T} T{
1578 Space allocation info
1579 T} T{
1580 3.14
1581 T}
1582 _
1583 T{
1584 <UUID>/bdi/
1585 T} T{
1586 Backing device info (writeback)
1587 T} T{
1588 5.9
1589 T}
1590 _
1591 T{
1592 <UUID>/devices/<DEVID>/
1593 T} T{
1594 Symlink to each block device sysfs
1595 T} T{
1596 5.6
1597 T}
1598 _
1599 T{
1600 <UUID>/devinfo/<DEVID>/
1601 T} T{
1602 Btrfs specific info for each device
1603 T} T{
1604 5.6
1605 T}
1606 _
1607 T{
1608 <UUID>/discard/
1609 T} T{
1610 Discard stats and tunables
1611 T} T{
1612 6.1
1613 T}
1614 _
1615 T{
1616 <UUID>/features/
1617 T} T{
1618 Features of the filesystem
1619 T} T{
1620 3.14
1621 T}
1622 _
1623 T{
1624 <UUID>/qgroups/
1625 T} T{
1626 Global qgroup info
1627 T} T{
1628 5.9
1629 T}
1630 _
1631 T{
1632 <UUID>/qgroups/<LEVEL>_<ID>/
1633 T} T{
1634 Info for each qgroup
1635 T} T{
1636 5.9
1637 T}
1638 .TE
1639 .sp
1640 For \fB/sys/fs/btrfs/features/\fP directory, each file means a supported feature
1641 of the current kernel. Most files have value 0. Otherwise it depends on the file,
1642 value \fI1\fP typically means the feature can be turned on a mounted filesystem.
1643 .sp
1644 For \fB/sys/fs/btrfs/<UUID>/features/\fP directory, each file means an enabled
1645 feature on the mounted filesystem.
1646 .sp
1647 The features share the same name in section
1648 FILESYSTEM FEATURES\&.
1649 .SS UUID
1650 .sp
1651 Files in \fB/sys/fs/btrfs/<UUID>/\fP directory are:
1652 .INDENT 0.0
1653 .TP
1654 .B bg_reclaim_threshold
1655 (RW, since: 5.19)
1656 .sp
1657 Used space percentage of total device space to start auto block group claim.
1658 Mostly for zoned devices.
1659 .TP
1660 .B checksum
1661 (RO, since: 5.5)
1662 .sp
1663 The checksum used for the mounted filesystem.
1664 This includes both the checksum type (see section
1665 CHECKSUM ALGORITHMS)
1666 and the implemented driver (mostly shows if it\(aqs hardware accelerated).
1667 .TP
1668 .B clone_alignment
1669 (RO, since: 3.16)
1670 .sp
1671 The bytes alignment for \fIclone\fP and \fIdedupe\fP ioctls.
1672 .TP
1673 .B commit_stats
1674 (RW, since: 6.0)
1675 .sp
1676 The performance statistics for btrfs transaction commit since the first
1677 mount. Mostly for debugging purposes.
1678 .sp
1679 Writing into this file will reset the maximum commit duration
1680 (\fImax_commit_ms\fP) to 0. The file looks like:
1681 .INDENT 7.0
1682 .INDENT 3.5
1683 .sp
1684 .EX
1685 commits 70649
1686 last_commit_ms 2
1687 max_commit_ms 131
1688 total_commit_ms 170840
1689 .EE
1690 .UNINDENT
1691 .UNINDENT
1692 .INDENT 7.0
1693 .IP \(bu 2
1694 \fIcommits\fP \- number of transaction commits since the first mount
1695 .IP \(bu 2
1696 \fIlast_commit_ms\fP \- duration in milliseconds of the last commit
1697 .IP \(bu 2
1698 \fImax_commit_ms\fP \- maximum time a transaction commit took since first mount or last reset
1699 .IP \(bu 2
1700 \fItotal_commit_ms\fP \- sum of all transaction commit times
1701 .UNINDENT
1702 .TP
1703 .B exclusive_operation
1704 (RO, since: 5.10)
1705 .sp
1706 Shows the running exclusive operation.
1707 Check section
1708 FILESYSTEM EXCLUSIVE OPERATIONS
1709 for details.
1710 .TP
1711 .B generation
1712 (RO, since: 5.11)
1713 .sp
1714 Show the generation of the mounted filesystem.
1715 .TP
1716 .B label
1717 (RW, since: 3.14)
1718 .sp
1719 Show the current label of the mounted filesystem.
1720 .TP
1721 .B metadata_uuid
1722 (RO, since: 5.0)
1723 .sp
1724 Shows the metadata UUID of the mounted filesystem.
1725 Check \fImetadata_uuid\fP feature for more details.
1726 .TP
1727 .B nodesize
1728 (RO, since: 3.14)
1729 .sp
1730 Show the nodesize of the mounted filesystem.
1731 .TP
1732 .B quota_override
1733 (RW, since: 4.13)
1734 .sp
1735 Shows the current quota override status.
1736 0 means no quota override.
1737 1 means quota override, quota can ignore the existing limit settings.
1738 .TP
1739 .B read_policy
1740 (RW, since: 5.11)
1741 .sp
1742 Shows the current balance policy for reads.
1743 Currently only \fBpid\fP (balance using the process id (pid) value) is
1744 supported. More balancing policies are available in experimental
1745 build, namely round\-robin.
1746 .TP
1747 .B sectorsize
1748 (RO, since: 3.14)
1749 .sp
1750 Shows the sectorsize of the mounted filesystem.
1751 .TP
1752 .B temp_fsid
1753 (RO, since 6.7)
1754 .sp
1755 Indicate that this filesystem got assigned a temporary FSID at mount time,
1756 making possible to mount devices with the same FSID.
1757 .UNINDENT
1758 .SS UUID/allocations
1759 .sp
1760 Files and directories in \fB/sys/fs/btrfs/<UUID>/allocations\fP directory are:
1761 .INDENT 0.0
1762 .TP
1763 .B global_rsv_reserved
1764 (RO, since: 3.14)
1765 .sp
1766 The used bytes of the global reservation.
1767 .TP
1768 .B global_rsv_size
1769 (RO, since: 3.14)
1770 .sp
1771 The total size of the global reservation.
1772 .TP
1773 .B \fIdata/\fP, \fImetadata/\fP and \fIsystem/\fP directories
1774 (RO, since: 5.14)
1775 .sp
1776 Space info accounting for the 3 block group types.
1777 .UNINDENT
1778 .SS UUID/allocations/{data,metadata,system}
1779 .sp
1780 Files in \fB/sys/fs/btrfs/<UUID>/allocations/\fP\fIdata,metadata,system\fP directory are:
1781 .INDENT 0.0
1782 .TP
1783 .B bg_reclaim_threshold
1784 (RW, since: 5.19)
1785 .sp
1786 Reclaimable space percentage of block group\(aqs size (excluding
1787 permanently unusable space) to reclaim the block group.
1788 Can be used on regular or zoned devices.
1789 .TP
1790 .B bytes_*
1791 (RO)
1792 .sp
1793 Values of the corresponding data structures for the given block group
1794 type and profile that are used internally and may change rapidly depending
1795 on the load.
1796 .sp
1797 Complete list: bytes_may_use, bytes_pinned, bytes_readonly,
1798 bytes_reserved, bytes_used, bytes_zone_unusable
1799 .TP
1800 .B chunk_size
1801 (RW, since: 6.0)
1802 .sp
1803 Shows the chunk size. Can be changed for data and metadata (independently)
1804 and cannot be set for system block group type.
1805 Cannot be set for zoned devices as it depends on the fixed device zone size.
1806 Upper bound is 10% of the filesystem size, the value must be multiple of 256MiB
1807 and greater than 0.
1808 .TP
1809 .B size_classes
1810 (RO, since: 6.3)
1811 .sp
1812 Numbers of block groups of a given classes based on heuristics that
1813 measure extent length, age and fragmentation.
1814 .INDENT 7.0
1815 .INDENT 3.5
1816 .sp
1817 .EX
1818 none 136
1819 small 374
1820 medium 282
1821 large 93
1822 .EE
1823 .UNINDENT
1824 .UNINDENT
1825 .UNINDENT
1826 .SS UUID/bdi
1827 .sp
1828 Symlink to the sysfs directory of the backing device info (BDI), which is
1829 related to writeback process and infrastructure.
1830 .SS UUID/devices
1831 .sp
1832 Files in \fB/sys/fs/btrfs/<UUID>/devices\fP directory are symlinks named
1833 after device nodes (e.g. sda, dm\-0) and pointing to their sysfs directory.
1834 .SS UUID/devinfo
1835 .sp
1836 The directory contains subdirectories named after device ids (numeric values). Each
1837 subdirectory has information about the device of the given \fIdevid\fP\&.
1838 .SS UUID/devinfo/DEVID
1839 .sp
1840 Files in \fB/sys/fs/btrfs/<UUID>/devinfo/<DEVID>\fP directory are:
1841 .INDENT 0.0
1842 .TP
1843 .B error_stats:
1844 (RO, since: 5.14)
1845 .sp
1846 Shows device stats of this device, same as \fBbtrfs device stats\fP (btrfs\-device(8) \%<>).
1847 .INDENT 7.0
1848 .INDENT 3.5
1849 .sp
1850 .EX
1851 write_errs 0
1852 read_errs 0
1853 flush_errs 0
1854 corruption_errs 0
1855 generation_errs 0
1856 .EE
1857 .UNINDENT
1858 .UNINDENT
1859 .TP
1860 .B fsid:
1861 (RO, since: 5.17)
1862 .sp
1863 Shows the fsid which the device belongs to.
1864 It can be different than the \fBUUID\fP if it\(aqs a seed device.
1865 .TP
1866 .B in_fs_metadata
1867 (RO, since: 5.6)
1868 .sp
1869 Shows whether we have found the device.
1870 Should always be 1, as if this turns to 0, the \fBDEVID\fP directory
1871 would get removed automatically.
1872 .TP
1873 .B missing
1874 (RO, since: 5.6)
1875 .sp
1876 Shows whether the device is considered missing by the kernel module.
1877 .TP
1878 .B replace_target
1879 (RO, since: 5.6)
1880 .sp
1881 Shows whether the device is the replace target.
1882 If no device replace is running, this value is 0.
1883 .TP
1884 .B scrub_speed_max
1885 (RW, since: 5.14)
1886 .sp
1887 Shows the scrub speed limit for this device. The unit is Bytes/s.
1888 0 means no limit. The value can be set but is not persistent.
1889 .TP
1890 .B writeable
1891 (RO, since: 5.6)
1892 .sp
1893 Show if the device is writeable.
1894 .UNINDENT
1895 .SS UUID/qgroups
1896 .sp
1897 Files in \fB/sys/fs/btrfs/<UUID>/qgroups/\fP directory are:
1898 .INDENT 0.0
1899 .TP
1900 .B enabled
1901 (RO, since: 6.1)
1902 .sp
1903 Shows if qgroup is enabled.
1904 Also, if qgroup is disabled, the \fBqgroups\fP directory will
1905 be removed automatically.
1906 .TP
1907 .B inconsistent
1908 (RO, since: 6.1)
1909 .sp
1910 Shows if the qgroup numbers are inconsistent.
1911 If 1, it\(aqs recommended to do a qgroup rescan.
1912 .TP
1913 .B drop_subtree_threshold
1914 (RW, since: 6.1)
1915 .sp
1916 Shows the subtree drop threshold to automatically mark qgroup inconsistent.
1917 .sp
1918 When dropping large subvolumes with qgroup enabled, there would be a huge
1919 load for qgroup accounting.
1920 If we have a subtree whose level is larger than or equal to this value,
1921 we will not trigger qgroup account at all, but mark qgroup inconsistent to
1922 avoid the huge workload.
1923 .sp
1924 Default value is 3, which means that trees of low height will be accounted
1925 properly as this is sufficiently fast. The value was 8 until 6.13 where
1926 no subtree drop can trigger qgroup rescan making it less useful.
1927 .sp
1928 Lower value can reduce qgroup workload, at the cost of extra qgroup rescan
1929 to re\-calculate the numbers.
1930 .UNINDENT
1931 .SS UUID/qgroups/LEVEL_ID
1932 .sp
1933 Files in each \fB/sys/fs/btrfs/<UUID>/qgroups/<LEVEL>_<ID>/\fP directory are:
1934 .INDENT 0.0
1935 .TP
1936 .B exclusive
1937 (RO, since: 5.9)
1938 .sp
1939 Shows the exclusively owned bytes of the qgroup.
1940 .TP
1941 .B limit_flags
1942 (RO, since: 5.9)
1943 .sp
1944 Shows the numeric value of the limit flags.
1945 If 0, means no limit implied.
1946 .TP
1947 .B max_exclusive
1948 (RO, since: 5.9)
1949 .sp
1950 Shows the limits on exclusively owned bytes.
1951 .TP
1952 .B max_referenced
1953 (RO, since: 5.9)
1954 .sp
1955 Shows the limits on referenced bytes.
1956 .TP
1957 .B referenced
1958 (RO, since: 5.9)
1959 .sp
1960 Shows the referenced bytes of the qgroup.
1961 .TP
1962 .B rsv_data
1963 (RO, since: 5.9)
1964 .sp
1965 Shows the reserved bytes for data.
1966 .TP
1967 .B rsv_meta_pertrans
1968 (RO, since: 5.9)
1969 .sp
1970 Shows the reserved bytes for per transaction metadata.
1971 .TP
1972 .B rsv_meta_prealloc
1973 (RO, since: 5.9)
1974 .sp
1975 Shows the reserved bytes for preallocated metadata.
1976 .UNINDENT
1977 .SS UUID/discard
1978 .sp
1979 Files in \fB/sys/fs/btrfs/<UUID>/discard/\fP directory are:
1980 .INDENT 0.0
1981 .TP
1982 .B discardable_bytes
1983 (RO, since: 6.1)
1984 .sp
1985 Shows amount of bytes that can be discarded in the async discard and
1986 nodiscard mode.
1987 .TP
1988 .B discardable_extents
1989 (RO, since: 6.1)
1990 .sp
1991 Shows number of extents to be discarded in the async discard and
1992 nodiscard mode.
1993 .TP
1994 .B discard_bitmap_bytes
1995 (RO, since: 6.1)
1996 .sp
1997 Shows amount of discarded bytes from data tracked as bitmaps.
1998 .TP
1999 .B discard_extent_bytes
2000 (RO, since: 6.1)
2001 .sp
2002 Shows amount of discarded extents from data tracked as bitmaps.
2003 .TP
2004 .B discard_bytes_saved
2005 (RO, since: 6.1)
2006 .sp
2007 Shows the amount of bytes that were reallocated without being discarded.
2008 .TP
2009 .B kbps_limit
2010 (RW, since: 6.1)
2011 .sp
2012 Tunable limit of kilobytes per second issued as discard IO in the async
2013 discard mode.
2014 .TP
2015 .B iops_limit
2016 (RW, since: 6.1)
2017 .sp
2018 Tunable limit of number of discard IO operations to be issued in the
2019 async discard mode.
2020 .TP
2021 .B max_discard_size
2022 (RW, since: 6.1)
2023 .sp
2024 Tunable limit for size of one IO discard request.
2025 .UNINDENT
2026 .SH FILESYSTEM EXCLUSIVE OPERATIONS
2027 .sp
2028 There are several operations that affect the whole filesystem and cannot be run
2029 in parallel. Attempt to start one while another is running will fail (see
2030 exceptions below).
2031 .sp
2032 Since kernel 5.10 the currently running operation can be obtained from
2033 \fB/sys/fs/UUID/exclusive_operation\fP with following values and operations:
2034 .INDENT 0.0
2035 .IP \(bu 2
2036 balance
2037 .IP \(bu 2
2038 balance paused (since 5.17)
2039 .IP \(bu 2
2040 device add
2041 .IP \(bu 2
2042 device delete
2043 .IP \(bu 2
2044 device replace
2045 .IP \(bu 2
2046 resize
2047 .IP \(bu 2
2048 swapfile activate
2049 .IP \(bu 2
2050 none
2051 .UNINDENT
2052 .sp
2053 Enqueuing is supported for several btrfs subcommands so they can be started
2054 at once and then serialized.
2055 .sp
2056 There\(aqs an exception when a paused balance allows to start a device add
2057 operation as they don\(aqt really collide and this can be used to add more space
2058 for the balance to finish.
2059 .SH FILESYSTEM LIMITS
2060 .INDENT 0.0
2061 .TP
2062 .B maximum file name length
2063 255
2064 .sp
2065 This limit is imposed by Linux VFS, the structures of BTRFS could store
2066 larger file names.
2067 .TP
2068 .B maximum symlink target length
2069 depends on the \fInodesize\fP value, for 4KiB it\(aqs 3949 bytes, for larger nodesize
2070 it\(aqs 4095 due to the system limit PATH_MAX
2071 .sp
2072 The symlink target may not be a valid path, i.e. the path name components
2073 can exceed the limits (NAME_MAX), there\(aqs no content validation at symlink(3) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man3/\:symlink\:.3\:.html>
2074 creation.
2075 .TP
2076 .B maximum number of inodes
2077 2\s-2\u64\d\s0 but depends on the available metadata space as the inodes are created
2078 dynamically
2079 .sp
2080 Each subvolume is an independent namespace of inodes and thus their
2081 numbers, so the limit is per subvolume, not for the whole filesystem.
2082 .TP
2083 .B inode numbers
2084 minimum number: 256 (for subvolumes), regular files and directories: 257,
2085 maximum number: (2\s-2\u64\d\s0 \- 256)
2086 .sp
2087 The inode numbers that can be assigned to user created files are from
2088 the whole 64bit space except first 256 and last 256 in that range that
2089 are reserved for internal b\-tree identifiers.
2090 .TP
2091 .B maximum file length
2092 inherent limit of BTRFS is 2\s-2\u64\d\s0 (16 EiB) but the practical
2093 limit of Linux VFS is 2\s-2\u63\d\s0 (8 EiB)
2094 .TP
2095 .B maximum number of subvolumes
2096 the subvolume ids can go up to 2\s-2\u48\d\s0 but the number of actual subvolumes
2097 depends on the available metadata space
2098 .sp
2099 The space consumed by all subvolume metadata includes bookkeeping of
2100 shared extents can be large (MiB, GiB). The range is not the full 64bit
2101 range because of qgroups that use the upper 16 bits for another
2102 purposes.
2103 .TP
2104 .B maximum number of hardlinks of a file in a directory
2105 65536 when the \fIextref\fP feature is turned on during mkfs (default), roughly
2106 100 otherwise and depends on file name length that fits into one metadata node
2107 .TP
2108 .B minimum filesystem size
2109 the minimal size of each device depends on the \fImixed\-bg\fP feature, without that
2110 (the default) it\(aqs about 109MiB, with mixed\-bg it\(aqs is 16MiB
2111 .UNINDENT
2112 .SH BOOTLOADER SUPPORT
2113 .sp
2114 GRUB2 (\%<https://\:www\:.gnu\:.org/\:software/\:grub>) has the most advanced support of
2115 booting from BTRFS with respect to features.
2116 .sp
2117 U\-Boot (\%<https://\:www\:.denx\:.de/\:wiki/\:U-Boot/>) has decent support for booting but
2118 not all BTRFS features are implemented, check the documentation.
2119 .sp
2120 In general, the first 1MiB on each device is unused with the exception of
2121 primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be
2122 freely used by bootloaders or for other system information. Note that booting
2123 from a filesystem on zoned device \%<> is not supported.
2124 .SH FILE ATTRIBUTES
2125 .sp
2126 The btrfs filesystem supports setting file attributes or flags. Note there are
2127 old and new interfaces, with confusing names. The following list should clarify
2128 that:
2129 .INDENT 0.0
2130 .IP \(bu 2
2131 \fIattributes\fP: chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> or lsattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:lsattr\:.1\:.html> utilities (the ioctls are
2132 FS_IOC_GETFLAGS and FS_IOC_SETFLAGS), due to the ioctl names the attributes
2133 are also called flags
2134 .IP \(bu 2
2135 \fIxflags\fP: to distinguish from the previous, it\(aqs extended flags, with tunable
2136 bits similar to the attributes but extensible and new bits will be added in
2137 the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR but they
2138 are not related to extended attributes that are also called xattrs), there\(aqs
2139 no standard tool to change the bits, there\(aqs support in xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> as
2140 command \fBxfs_io \-c chattr\fP
2141 .UNINDENT
2142 .sp
2143 Attributes have constraints associated and not all combinations can be set, the
2144 order of setting them also matters. Most attributes apply to files and
2145 directories but the semantics may differ. For directories the attribute may
2146 only mean to set this attribute to all new files (\fIinheritable\fP in the list
2147 below). Some attributes need root privileges to be set.
2148 .SS Attributes
2149 .INDENT 0.0
2150 .TP
2151 .B a
2152 (file, dir, root)
2153 \fIappend only\fP, new writes are always written at the end of the file
2154 .TP
2155 .B A
2156 (file, dir)
2157 \fIno atime updates\fP
2158 .TP
2159 .B c
2160 (file, dir, inherited)
2161 \fIcompress data\fP, all data written after this attribute is set will be compressed.
2162 Please note that compression is also affected by the mount options or the parent
2163 directory attributes.
2164 .sp
2165 When set on a directory, all newly created files will inherit this attribute.
2166 This attribute cannot be set with \(aqm\(aq at the same time.
2167 .TP
2168 .B C
2169 (file, dir, inherited)
2170 \fIno copy\-on\-write\fP, file data modifications are done in\-place
2171 .sp
2172 When set on a directory, all newly created files will inherit this attribute.
2173 .sp
2174 \fBNote:\fP
2175 .INDENT 7.0
2176 .INDENT 3.5
2177 Due to implementation limitations, this flag can be set/unset only on
2178 empty files.
2179 .UNINDENT
2180 .UNINDENT
2181 .TP
2182 .B d
2183 (file)
2184 \fIno dump\fP, makes sense with 3rd party tools like dump(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:dump\:.8\:.html>, on BTRFS the
2185 attribute can be set/unset but no other special handling is done
2186 .TP
2187 .B D
2188 (dir)
2189 \fIsynchronous directory updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP
2190 and \fIO_DSYNC\fP
2191 .TP
2192 .B i
2193 (file, dir, root)
2194 \fIimmutable\fP, no file data and metadata changes allowed even to the root user as
2195 long as this attribute is set (obviously the exception is unsetting the attribute)
2196 .TP
2197 .B m
2198 (file, dir)
2199 \fIno compression\fP, permanently turn off compression on the given file. Any
2200 compression mount options will not affect this file. (chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> support added in
2201 1.46.2)
2202 .sp
2203 When set on a directory, all newly created files will inherit this attribute.
2204 This attribute cannot be set with \fIc\fP at the same time.
2205 .TP
2206 .B S
2207 (file)
2208 \fIsynchronous updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP and
2209 \fIO_DSYNC\fP
2210 .TP
2211 .B V
2212 (file, read\-only)
2213 \fIfs\-verity enabled\fP on the file
2214 .UNINDENT
2215 .sp
2216 No other attributes are supported. For the complete list please refer to the
2217 chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> manual page.
2218 .SS XFLAGS
2219 .sp
2220 There\(aqs an overlap of letters assigned to the bits with the attributes, this list
2221 refers to what xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> provides:
2222 .INDENT 0.0
2223 .TP
2224 .B i
2225 \fIimmutable\fP, same as the attribute
2226 .TP
2227 .B a
2228 \fIappend only\fP, same as the attribute
2229 .TP
2230 .B s
2231 \fIsynchronous updates\fP, same as the attribute \fIS\fP
2232 .TP
2233 .B A
2234 \fIno atime updates\fP, same as the attribute
2235 .TP
2236 .B d
2237 \fIno dump\fP, same as the attribute
2238 .UNINDENT
2239 .SH ZONED MODE
2240 .sp
2241 Since version 5.12 btrfs supports so called \fIzoned mode\fP\&. This is a special
2242 on\-disk format and allocation/write strategy that\(aqs friendly to zoned devices.
2243 In short, a device is partitioned into fixed\-size zones and each zone can be
2244 updated by append\-only manner, or reset. As btrfs has no fixed data structures,
2245 except the super blocks, the zoned mode only requires block placement that
2246 follows the device constraints. You can learn about the whole architecture at
2247 \%<https://\:zonedstorage\:.io> .
2248 .sp
2249 The devices are also called SMR/ZBC/ZNS, in \fIhost\-managed\fP mode. Note that
2250 there are devices that appear as non\-zoned but actually are, this is
2251 \fIdrive\-managed\fP and using zoned mode won\(aqt help.
2252 .sp
2253 The zone size depends on the device, typical sizes are 256MiB or 1GiB. In
2254 general it must be a power of two. Emulated zoned devices like \fInull_blk\fP allow
2255 to set various zone sizes.
2256 .SS Requirements, limitations
2257 .INDENT 0.0
2258 .IP \(bu 2
2259 all devices must have the same zone size
2260 .IP \(bu 2
2261 maximum zone size is 8GiB
2262 .IP \(bu 2
2263 minimum zone size is 4MiB
2264 .IP \(bu 2
2265 mixing zoned and non\-zoned devices is possible, the zone writes are emulated,
2266 but this is namely for testing
2267 .IP \(bu 2
2268 the super block is handled in a special way and is at different locations than on a non\-zoned filesystem:
2269 .INDENT 2.0
2270 .IP \(bu 2
2271 primary: 0B (and the next two zones)
2272 .IP \(bu 2
2273 secondary: 512GiB (and the next two zones)
2274 .IP \(bu 2
2275 tertiary: 4TiB (4096GiB, and the next two zones)
2276 .UNINDENT
2277 .UNINDENT
2278 .SS Incompatible features
2279 .sp
2280 The main constraint of the zoned devices is lack of in\-place update of the data.
2281 This is inherently incompatible with some features:
2282 .INDENT 0.0
2283 .IP \(bu 2
2284 NODATACOW \- overwrite in\-place, cannot create such files
2285 .IP \(bu 2
2286 fallocate \- preallocating space for in\-place first write
2287 .IP \(bu 2
2288 mixed\-bg \- unordered writes to data and metadata, fixing that means using
2289 separate data and metadata block groups
2290 .IP \(bu 2
2291 booting \- the zone at offset 0 contains superblock, resetting the zone would
2292 destroy the bootloader data
2293 .UNINDENT
2294 .sp
2295 Initial support lacks some features but they\(aqre planned:
2296 .INDENT 0.0
2297 .IP \(bu 2
2298 only single (data, metadata) and DUP (metadata) profile is supported
2299 .IP \(bu 2
2300 fstrim \- due to dependency on free space cache v1
2301 .UNINDENT
2302 .SS Super block
2303 .sp
2304 As said above, super block is handled in a special way. In order to be crash
2305 safe, at least one zone in a known location must contain a valid superblock.
2306 This is implemented as a ring buffer in two consecutive zones, starting from
2307 known offsets 0B, 512GiB and 4TiB.
2308 .sp
2309 The values are different than on non\-zoned devices. Each new super block is
2310 appended to the end of the zone, once it\(aqs filled, the zone is reset and writes
2311 continue to the next one. Looking up the latest super block needs to read
2312 offsets of both zones and determine the last written version.
2313 .sp
2314 The amount of space reserved for super block depends on the zone size. The
2315 secondary and tertiary copies are at distant offsets as the capacity of the
2316 devices is expected to be large, tens of terabytes. Maximum zone size supported
2317 is 8GiB, which would mean that e.g. offset 0\-16GiB would be reserved just for
2318 the super block on a hypothetical device of that zone size. This is wasteful
2319 but required to guarantee crash safety.
2320 .SS Zone reclaim, garbage collection
2321 .sp
2322 As the zones are append\-only, overwriting data or COW changes in metadata
2323 make parts of the zones used but not connected to the filesystem structures.
2324 This makes the space unusable and grows over time. Once the ratio hits a
2325 (configurable) threshold a background reclaim process is started and relocates
2326 the remaining blocks in use to a new zone. The old one is reset and can be used
2327 again.
2328 .sp
2329 This process may take some time depending on other background work or
2330 amount of new data written. It is possible to hit an intermittent ENOSPC.
2331 Some devices also limit number of active zones.
2332 .SS Devices
2333 .SS Real hardware
2334 .sp
2335 The WD Ultrastar series 600 advertises HM\-SMR, i.e. the host\-managed zoned
2336 mode. There are two more: DA (device managed, no zoned information exported to
2337 the system), HA (host aware, can be used as regular disk but zoned writes
2338 improve performance). There are not many devices available at the moment, the
2339 information about exact zoned mode is hard to find, check data sheets or
2340 community sources gathering information from real devices.
2341 .sp
2342 Note: zoned mode won\(aqt work with DM\-SMR disks.
2343 .INDENT 0.0
2344 .IP \(bu 2
2345 Ultrastar® DC ZN540 NVMe ZNS SSD (product
2346 brief \%<https://\:documents\:.westerndigital\:.com/\:content/\:dam/\:doc-library/\:en_us/\:assets/\:public/\:western-digital/\:collateral/\:product-brief/\:product-brief-ultrastar-dc-zn540\:.pdf>)
2347 .UNINDENT
2348 .SS Emulated: null_blk
2349 .sp
2350 The driver \fInull_blk\fP provides memory backed device and is suitable for
2351 testing. There are some quirks setting up the devices. The module must be
2352 loaded with \fInr_devices=0\fP or the numbering of device nodes will be offset. The
2353 \fIconfigfs\fP must be mounted at \fI/sys/kernel/config\fP and the administration of
2354 the null_blk devices is done in \fI/sys/kernel/config/nullb\fP\&. The device nodes
2355 are named like \fB/dev/nullb0\fP and are numbered sequentially. NOTE: the device
2356 name may be different than the named directory in sysfs!
2357 .sp
2358 Setup:
2359 .INDENT 0.0
2360 .INDENT 3.5
2361 .sp
2362 .EX
2363 modprobe\ configfs
2364 modprobe\ null_blk\ nr_devices=0
2365 .EE
2366 .UNINDENT
2367 .UNINDENT
2368 .sp
2369 Create a device \fImydev\fP, assuming no other previously created devices, size is
2370 2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal
2371 example taking defaults:
2372 .INDENT 0.0
2373 .INDENT 3.5
2374 .sp
2375 .EX
2376 cd\ /sys/kernel/config/nullb/
2377 mkdir\ mydev
2378 cd\ mydev
2379 echo\ 2048\ >\ size
2380 echo\ 1\ >\ zoned
2381 echo\ 1\ >\ memory_backed
2382 echo\ 256\ >\ zone_size
2383 echo\ 1\ >\ power
2384 .EE
2385 .UNINDENT
2386 .UNINDENT
2387 .sp
2388 This will create a device \fB/dev/nullb0\fP and the value of file \fIindex\fP will
2389 match the ending number of the device node.
2390 .sp
2391 Remove the device:
2392 .INDENT 0.0
2393 .INDENT 3.5
2394 .sp
2395 .EX
2396 rmdir\ /sys/kernel/config/nullb/mydev
2397 .EE
2398 .UNINDENT
2399 .UNINDENT
2400 .sp
2401 Then continue with \fBmkfs.btrfs /dev/nullb0\fP, the zoned mode is auto\-detected.
2402 .sp
2403 For convenience, there\(aqs a script wrapping the basic null_blk management operations
2404 \%<https://\:github\:.com/\:kdave/\:nullb\:.git>, the above commands become:
2405 .INDENT 0.0
2406 .INDENT 3.5
2407 .sp
2408 .EX
2409 nullb setup
2410 nullb create \-s 2g \-z 256
2411 mkfs.btrfs /dev/nullb0
2412 \&...
2413 nullb rm nullb0
2414 .EE
2415 .UNINDENT
2416 .UNINDENT
2417 .SS Emulated: TCMU runner
2418 .sp
2419 TCMU is a framework to emulate SCSI devices in userspace, providing various
2420 backends for the storage, with zoned support as well. A file\-backed zoned
2421 device can provide more options for larger storage and zone size. Please follow
2422 the instructions at \%<https://\:zonedstorage\:.io/\:projects/\:tcmu-runner/> .
2423 .SS Compatibility, incompatibility
2424 .INDENT 0.0
2425 .IP \(bu 2
2426 the feature sets an incompat bit and requires new kernel to access the
2427 filesystem (for both read and write)
2428 .IP \(bu 2
2429 superblock needs to be handled in a special way, there are still 3 copies
2430 but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a
2431 ring buffer of the superblocks, finding the latest one needs reading it from
2432 the write pointer or do a full scan of the zones
2433 .IP \(bu 2
2434 mixing zoned and non zoned devices is possible (zones are emulated) but is
2435 recommended only for testing
2436 .IP \(bu 2
2437 mixing zoned devices with different zone sizes is not possible
2438 .IP \(bu 2
2439 zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB
2440 or 1GiB, larger size is expected, maximum zone size supported by btrfs is
2441 8GiB
2442 .UNINDENT
2443 .SS Status, stability, reporting bugs
2444 .sp
2445 The zoned mode has been released in 5.12 and there are still some rough edges
2446 and corner cases one can hit during testing. Please report bugs to
2447 \%<https://\:github\:.com/\:naota/\:linux/\:issues/> .
2448 .SS References
2449 .INDENT 0.0
2450 .IP \(bu 2
2451 \%<https://\:zonedstorage\:.io>
2452 .INDENT 2.0
2453 .IP \(bu 2
2454 \%<https://\:zonedstorage\:.io/\:projects/\:libzbc/> \-\- \fIlibzbc\fP is library and set
2455 of tools to directly manipulate devices with ZBC/ZAC support
2456 .IP \(bu 2
2457 \%<https://\:zonedstorage\:.io/\:projects/\:libzbd/> \-\- \fIlibzbd\fP uses the kernel
2458 provided zoned block device interface based on the ioctl() system calls
2459 .UNINDENT
2460 .IP \(bu 2
2461 \%<https://\:hddscan\:.com/\:blog/\:2020/\:hdd-wd-smr\:.html> \-\- some details about exact device types
2462 .IP \(bu 2
2463 \%<https://\:lwn\:.net/\:Articles/\:853308/> \-\- \fIBtrfs on zoned block devices\fP
2464 .IP \(bu 2
2465 \%<https://\:www\:.usenix\:.org/\:conference/\:vault20/\:presentation/\:bjorling> \-\- Zone
2466 Append: A New Way of Writing to Zoned Storage
2467 .UNINDENT
2468 .SH CONTROL DEVICE
2469 .sp
2470 There\(aqs a character special device \fB/dev/btrfs\-control\fP with major and minor
2471 numbers 10 and 234 (the device can be found under the \fImisc\fP category).
2472 .INDENT 0.0
2473 .INDENT 3.5
2474 .sp
2475 .EX
2476 $ ls \-l /dev/btrfs\-control
2477 crw\-\-\-\-\-\-\- 1 root root 10, 234 Jan 1 12:00 /dev/btrfs\-control
2478 .EE
2479 .UNINDENT
2480 .UNINDENT
2481 .sp
2482 The device accepts some ioctl calls that can perform following actions on the
2483 filesystem module:
2484 .INDENT 0.0
2485 .IP \(bu 2
2486 scan devices for btrfs filesystem (i.e. to let multi\-device filesystems mount
2487 automatically) and register them with the kernel module
2488 .IP \(bu 2
2489 similar to scan, but also wait until the device scanning process is finished
2490 for a given filesystem
2491 .IP \(bu 2
2492 get the supported features (can be also found under \fB/sys/fs/btrfs/features\fP)
2493 .UNINDENT
2494 .sp
2495 The device is created when btrfs is initialized, either as a module or a
2496 built\-in functionality and makes sense only in connection with that. Running
2497 e.g. mkfs without the module loaded will not register the device and will
2498 probably warn about that.
2499 .sp
2500 In rare cases when the module is loaded but the device is not present (most
2501 likely accidentally deleted), it\(aqs possible to recreate it by
2502 .INDENT 0.0
2503 .INDENT 3.5
2504 .sp
2505 .EX
2506 # mknod \-\-mode=600 /dev/btrfs\-control c 10 234
2507 .EE
2508 .UNINDENT
2509 .UNINDENT
2510 .sp
2511 or (since 5.11) by a convenience command
2512 .INDENT 0.0
2513 .INDENT 3.5
2514 .sp
2515 .EX
2516 # btrfs rescue create\-control\-device
2517 .EE
2518 .UNINDENT
2519 .UNINDENT
2520 .sp
2521 The control device is not strictly required but the device scanning will not
2522 work and a workaround would need to be used to mount a multi\-device filesystem.
2523 The mount option \fIdevice\fP can trigger the device scanning during mount, see
2524 also \fBbtrfs device scan\fP\&.
2525 .SH FILESYSTEM WITH MULTIPLE PROFILES
2526 .sp
2527 It is possible that a btrfs filesystem contains multiple block group profiles
2528 of the same type. This could happen when a profile conversion using balance
2529 filters is interrupted (see btrfs\-balance(8) \%<>). Some
2530 \fBbtrfs\fP commands perform
2531 a test to detect this kind of condition and print a warning like this:
2532 .INDENT 0.0
2533 .INDENT 3.5
2534 .sp
2535 .EX
2536 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2537 WARNING: Data: single, raid1
2538 WARNING: Metadata: single, raid1
2539 .EE
2540 .UNINDENT
2541 .UNINDENT
2542 .sp
2543 The corresponding output of \fBbtrfs filesystem df\fP might look like:
2544 .INDENT 0.0
2545 .INDENT 3.5
2546 .sp
2547 .EX
2548 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2549 WARNING: Data: single, raid1
2550 WARNING: Metadata: single, raid1
2551 Data, RAID1: total=832.00MiB, used=0.00B
2552 Data, single: total=1.63GiB, used=0.00B
2553 System, single: total=4.00MiB, used=16.00KiB
2554 Metadata, single: total=8.00MiB, used=112.00KiB
2555 Metadata, RAID1: total=64.00MiB, used=32.00KiB
2556 GlobalReserve, single: total=16.25MiB, used=0.00B
2557 .EE
2558 .UNINDENT
2559 .UNINDENT
2560 .sp
2561 There\(aqs more than one line for type \fIData\fP and \fIMetadata\fP, while the profiles
2562 are \fIsingle\fP and \fIRAID1\fP\&.
2563 .sp
2564 This state of the filesystem OK but most likely needs the user/administrator to
2565 take an action and finish the interrupted tasks. This cannot be easily done
2566 automatically, also the user knows the expected final profiles.
2567 .sp
2568 In the example above, the filesystem started as a single device and \fIsingle\fP
2569 block group profile. Then another device was added, followed by balance with
2570 \fIconvert=raid1\fP but for some reason hasn\(aqt finished. Restarting the balance
2571 with \fIconvert=raid1\fP will continue and end up with filesystem with all block
2572 group profiles \fIRAID1\fP\&.
2573 .sp
2574 \fBNote:\fP
2575 .INDENT 0.0
2576 .INDENT 3.5
2577 If you\(aqre familiar with balance filters, you can use
2578 \fIconvert=raid1,profiles=single,soft\fP, which will take only the unconverted
2579 \fIsingle\fP profiles and convert them to \fIraid1\fP\&. This may speed up the conversion
2580 as it would not try to rewrite the already convert \fIraid1\fP profiles.
2581 .UNINDENT
2582 .UNINDENT
2583 .sp
2584 Having just one profile is desired as this also clearly defines the profile of
2585 newly allocated block groups, otherwise this depends on internal allocation
2586 policy. When there are multiple profiles present, the order of selection is
2587 RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are
2588 satisfied.
2589 .sp
2590 Commands that print the warning were chosen so they\(aqre brought to user
2591 attention when the filesystem state is being changed in that regard. This is:
2592 \fBdevice add\fP, \fBdevice delete\fP, \fBbalance cancel\fP, \fBbalance pause\fP\&. Commands
2593 that report space usage: \fBfilesystem df\fP, \fBdevice usage\fP\&. The command
2594 \fBfilesystem usage\fP provides a line in the overall summary:
2595 .INDENT 0.0
2596 .INDENT 3.5
2597 .sp
2598 .EX
2599 Multiple profiles: yes (data, metadata)
2600 .EE
2601 .UNINDENT
2602 .UNINDENT
2603 .SH SEEDING DEVICE
2604 .sp
2605 The COW mechanism and multiple devices under one hood enable an interesting
2606 concept, called a seeding device: extending a read\-only filesystem on a
2607 device with another device that captures all writes. For example
2608 imagine an immutable golden image of an operating system enhanced with another
2609 device that allows to use the data from the golden image and normal operation.
2610 This idea originated on CD\-ROMs with base OS and allowing to use them for live
2611 systems, but this became obsolete. There are technologies providing similar
2612 functionality, like unionmount \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Union_mount>,
2613 overlayfs \%<https://\:en\:.wikipedia\:.org/\:wiki/\:OverlayFS> or
2614 qcow2 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Qcow#\:qcow2> image snapshot.
2615 .sp
2616 The seeding device starts as a normal filesystem, once the contents is ready,
2617 \fBbtrfstune \-S 1\fP is used to flag it as a seeding device. Mounting such device
2618 will not allow any writes, except adding a new device by \fBbtrfs device add\fP\&.
2619 Then the filesystem can be remounted as read\-write.
2620 .sp
2621 Given that the filesystem on the seeding device is always recognized as
2622 read\-only, it can be used to seed multiple filesystems from one device at the
2623 same time. The UUID that is normally attached to a device is automatically
2624 changed to a random UUID on each mount.
2625 .sp
2626 \fBNote:\fP
2627 .INDENT 0.0
2628 .INDENT 3.5
2629 Before v6.17 kernel, a seed device could have been mounted
2630 independently along with sprouted filesystems.
2631 But since 6.17 kernel, a seed device can only be mounted either through
2632 a sprouted filesystem, or the seed device itself, not both at the same time.
2633 .sp
2634 This is to ensure a block device to have only a single filesystem bound
2635 to it, so that runtime device missing events can be properly handled.
2636 .UNINDENT
2637 .UNINDENT
2638 .sp
2639 Once the seeding device is mounted, it needs the writable device. After adding
2640 it, unmounting and mounting with \fBumount /path; mount /dev/writable
2641 /path\fP or remounting read\-write with \fBremount \-o remount,rw\fP makes the
2642 filesystem at \fB/path\fP ready for use.
2643 .sp
2644 \fBNote:\fP
2645 .INDENT 0.0
2646 .INDENT 3.5
2647 There was a known bug with using remount to make the mount writeable:
2648 remount will leave the filesystem in a state where it is unable to
2649 clean deleted snapshots, so it will leak space until it is unmounted
2650 and mounted properly.
2651 .sp
2652 That bug has been fixed in 5.11 and newer kernels.
2653 .UNINDENT
2654 .UNINDENT
2655 .sp
2656 Furthermore, deleting the seeding device from the filesystem can turn it into
2657 a normal filesystem, provided that the writable device can also contain all the
2658 data from the seeding device.
2659 .sp
2660 The seeding device flag can be cleared again by \fBbtrfstune \-f \-S 0\fP, e.g.
2661 allowing to update with newer data but please note that this will invalidate
2662 all existing filesystems that use this particular seeding device. This works
2663 for some use cases, not for others, and the forcing flag to the command is
2664 mandatory to avoid accidental mistakes.
2665 .sp
2666 Example how to create and use one seeding device:
2667 .INDENT 0.0
2668 .INDENT 3.5
2669 .sp
2670 .EX
2671 # mkfs.btrfs /dev/sda
2672 # mount /dev/sda /mnt/mnt1
2673 \&... fill mnt1 with data
2674 # umount /mnt/mnt1
2675
2676 # btrfstune \-S 1 /dev/sda
2677
2678 # mount /dev/sda /mnt/mnt1
2679 # btrfs device add /dev/sdb /mnt/mnt1
2680 # umount /mnt/mnt1
2681 # mount /dev/sdb /mnt/mnt1
2682 \&... /mnt/mnt1 is now writable
2683 .EE
2684 .UNINDENT
2685 .UNINDENT
2686 .sp
2687 Now \fB/mnt/mnt1\fP can be used normally. The device \fB/dev/sda\fP can be mounted
2688 again with a another writable device:
2689 .INDENT 0.0
2690 .INDENT 3.5
2691 .sp
2692 .EX
2693 # mount /dev/sda /mnt/mnt2
2694 # btrfs device add /dev/sdc /mnt/mnt2
2695 # umount /mnt/mnt2
2696 # mount /dev/sdc /mnt/mnt2
2697 \&... /mnt/mnt2 is now writable
2698 .EE
2699 .UNINDENT
2700 .UNINDENT
2701 .sp
2702 The writable device (file:\fI/dev/sdb\fP) can be decoupled from the seeding device and
2703 used independently:
2704 .INDENT 0.0
2705 .INDENT 3.5
2706 .sp
2707 .EX
2708 # btrfs device delete /dev/sda /mnt/mnt1
2709 .EE
2710 .UNINDENT
2711 .UNINDENT
2712 .sp
2713 As the contents originated in the seeding device, it\(aqs possible to turn
2714 \fB/dev/sdb\fP to a seeding device again and repeat the whole process.
2715 .sp
2716 A few things to note:
2717 .INDENT 0.0
2718 .IP \(bu 2
2719 it\(aqs recommended to use only single device for the seeding device, it works
2720 for multiple devices but the \fIsingle\fP profile must be used in order to make
2721 the seeding device deletion work
2722 .IP \(bu 2
2723 block group profiles \fIsingle\fP and \fIdup\fP support the use cases above
2724 .IP \(bu 2
2725 the label is copied from the seeding device and can be changed by \fBbtrfs filesystem label\fP
2726 .IP \(bu 2
2727 each new mount of the seeding device gets a new random UUID
2728 .IP \(bu 2
2729 \fBumount /path; mount /dev/writable /path\fP can be replaced with
2730 \fBmount \-o remount,rw /path\fP
2731 but it won\(aqt reclaim space of deleted subvolumes until the seeding device
2732 is mounted read\-write again before making it seeding again
2733 .UNINDENT
2734 .SS Chained seeding devices
2735 .sp
2736 Though it\(aqs not recommended and is rather an obscure and untested use case,
2737 chaining seeding devices is possible. In the first example, the writable device
2738 \fB/dev/sdb\fP can be turned onto another seeding device again, depending on the
2739 unchanged seeding device \fB/dev/sda\fP\&. Then using \fB/dev/sdb\fP as the primary
2740 seeding device it can be extended with another writable device, say \fB/dev/sdd\fP,
2741 and it continues as before as a simple tree structure on devices.
2742 .INDENT 0.0
2743 .INDENT 3.5
2744 .sp
2745 .EX
2746 # mkfs.btrfs /dev/sda
2747 # mount /dev/sda /mnt/mnt1
2748 \&... fill mnt1 with data
2749 # umount /mnt/mnt1
2750
2751 # btrfstune \-S 1 /dev/sda
2752
2753 # mount /dev/sda /mnt/mnt1
2754 # btrfs device add /dev/sdb /mnt/mnt1
2755 # mount \-o remount,rw /mnt/mnt1
2756 \&... /mnt/mnt1 is now writable
2757 # umount /mnt/mnt1
2758
2759 # btrfstune \-S 1 /dev/sdb
2760
2761 # mount /dev/sdb /mnt/mnt1
2762 # btrfs device add /dev/sdc /mnt
2763 # mount \-o remount,rw /mnt/mnt1
2764 \&... /mnt/mnt1 is now writable
2765 # umount /mnt/mnt1
2766 .EE
2767 .UNINDENT
2768 .UNINDENT
2769 .sp
2770 As a result we have:
2771 .INDENT 0.0
2772 .IP \(bu 2
2773 \fIsda\fP is a single seeding device, with its initial contents
2774 .IP \(bu 2
2775 \fIsdb\fP is a seeding device but requires \fIsda\fP, the contents are from the time
2776 when \fIsdb\fP is made seeding, i.e. contents of \fIsda\fP with any later changes
2777 .IP \(bu 2
2778 \fIsdc\fP last writable, can be made a seeding one the same way as was \fIsdb\fP,
2779 preserving its contents and depending on \fIsda\fP and \fIsdb\fP
2780 .UNINDENT
2781 .sp
2782 As long as the seeding devices are unmodified and available, they can be used
2783 to start another branch.
2784 .SH RAID56 STATUS AND RECOMMENDED PRACTICES
2785 .sp
2786 The RAID56 feature provides striping and parity over several devices, same as
2787 the traditional RAID5/6. There are some implementation and design deficiencies
2788 that make it unreliable for some corner cases and the feature \fBshould not be
2789 used in production, only for evaluation or testing\fP\&. The power failure safety
2790 for metadata with RAID56 is not 100%.
2791 .SS Metadata
2792 .sp
2793 Do not use \fIraid5\fP nor \fIraid6\fP for metadata. Use \fIraid1\fP or \fIraid1c3\fP
2794 respectively.
2795 .sp
2796 The substitute profiles provide the same guarantees against loss of 1 or 2
2797 devices, and in some respect can be an improvement. Recovering from one
2798 missing device will only need to access the remaining 1st or 2nd copy, that in
2799 general may be stored on some other devices due to the way RAID1 works on
2800 btrfs, unlike on a striped profile (similar to \fIraid0\fP) that would need all
2801 devices all the time.
2802 .sp
2803 The space allocation pattern and consumption is different (e.g. on N devices):
2804 for \fIraid5\fP as an example, a 1GiB chunk is reserved on each device, while with
2805 \fIraid1\fP there\(aqs each 1GiB chunk stored on 2 devices. The consumption of each
2806 1GiB of used metadata is then \fIN * 1GiB\fP for vs \fI2 * 1GiB\fP\&. Using \fIraid1\fP
2807 is also more convenient for balancing/converting to other profile due to lower
2808 requirement on the available chunk space.
2809 .SS Missing/incomplete support
2810 .sp
2811 When RAID56 is on the same filesystem with different raid profiles, the space
2812 reporting is inaccurate, e.g. \fBdf\fP, \fBbtrfs filesystem df\fP or
2813 \fBbtrfs filesystem usage\fP\&. When there\(aqs only a one profile per block
2814 group type (e.g. RAID5 for data) the reporting is accurate.
2815 .sp
2816 When scrub is started on a RAID56 filesystem, it\(aqs started on all devices that
2817 degrade the performance. The workaround is to start it on each device
2818 separately. Due to that the device stats may not match the actual state and
2819 some errors might get reported multiple times.
2820 .sp
2821 The \fIwrite hole\fP problem. An unclean shutdown could leave a partially written
2822 stripe in a state where the some stripe ranges and the parity are from the old
2823 writes and some are new. The information which is which is not tracked. Write
2824 journal is not implemented. Alternatively a full read\-modify\-write would make
2825 sure that a full stripe is always written, avoiding the write hole completely,
2826 but performance in that case turned out to be too bad for use.
2827 .sp
2828 The striping happens on all available devices (at the time the chunks were
2829 allocated), so in case a new device is added it may not be utilized
2830 immediately and would require a rebalance. A fixed configured stripe width is
2831 not implemented.
2832 .SH GLOSSARY
2833 .sp
2834 Terms in \fIitalics\fP also appear in this glossary.
2835 .INDENT 0.0
2836 .TP
2837 .B allocator
2838 Usually \fIallocator\fP means the \fIblock\fP allocator, i.e. the logic
2839 inside the filesystem which decides where to place newly allocated blocks
2840 in order to maintain several constraints (like data locality, low
2841 fragmentation).
2842 .sp
2843 In btrfs, allocator may also refer to \fIchunk\fP allocator, i.e. the
2844 logic behind placing chunks on devices.
2845 .TP
2846 .B balance
2847 An operation that can be done to a btrfs filesystem, for example
2848 through \fBbtrfs balance /path\fP\&. A
2849 balance passes all data in the filesystem through the \fIallocator\fP
2850 again. It is primarily intended to rebalance the data in the filesystem
2851 across the \fIdevices\fP when a device is added or removed. A balance
2852 will regenerate missing copies for the redundant \fIRAID\fP levels, if a
2853 device has failed. As of Linux kernel 3.3, a balance operation can be
2854 made selective about which parts of the filesystem are rewritten
2855 using filters \%<#\:man-balance-filters>\&.
2856 .TP
2857 .B barrier
2858 An instruction to the underlying hardware to ensure that everything before
2859 the barrier is physically written to permanent storage before anything
2860 after it. Used in btrfs\(aqs \fIcopy on write\fP approach to ensure
2861 filesystem consistency.
2862 .TP
2863 .B block
2864 A single physically and logically contiguous piece of storage on a
2865 device, of size e.g. 4K. In some contexts also referred to as \fIsector\fP,
2866 though the term \fIblock\fP is preferred.
2867 .TP
2868 .B block group
2869 The unit of allocation of space in btrfs. A block group is laid out on
2870 the disk by the btrfs \fIallocator\fP, and will consist of one or more
2871 \fIchunks\fP, each stored on a different \fIdevice\fP\&. The number of chunks
2872 used in a block group will depend on its \fIRAID\fP level.
2873 .TP
2874 .B B\-tree
2875 The fundamental storage data structure used in btrfs. Except for the
2876 \fIsuperblocks\fP, all of btrfs \fImetadata\fP is stored in one of several
2877 B\-trees on disk. B\-trees store key/item pairs. While the same code is
2878 used to implement all of the B\-trees, there are a few different
2879 categories of B\-tree. The name \fIbtrfs\fP refers to its use of B\-trees.
2880 .TP
2881 .B btrfsck, fsck, btrfs\-check
2882 Tool in \fIbtrfs\-progs\fP that checks an unmounted filesystem (\fIoffline\fP)
2883 and reports on any errors in the filesystem structures it finds. By
2884 default the tool runs in read\-only mode as fixing errors is potentially
2885 dangerous. See also \fIscrub\fP\&.
2886 .TP
2887 .B btrfs\-progs
2888 User mode tools to manage btrfs\-specific features. Maintained at
2889 \%<http://\:github\:.com/\:kdave/\:btrfs-progs\:.git> . The main frontend to btrfs
2890 features is the standalone tool \fIbtrfs\fP, although
2891 other tools such as \fImkfs.btrfs\fP and \fIbtrfstune\fP are also part of
2892 btrfs\-progs.
2893 .TP
2894 .B chunk
2895 A part of a \fIblock group\fP\&. Chunks are either 1 GiB in size (for data)
2896 or 256 MiB (for \fImetadata\fP), depending on the overall filesystem size.
2897 .TP
2898 .B chunk tree
2899 A layer that keeps information about mapping between physical and
2900 logical block addresses. It\(aqs stored within the \fIsystem\fP group.
2901 .TP
2902 .B cleaner
2903 Usually referred to in context of deleted subvolumes. It\(aqs a background
2904 process that removes the actual data once a subvolume has been deleted.
2905 Cleaning can involve lots of IO and CPU activity depending on the
2906 fragmentation and amount of shared data with other subvolumes.
2907 .sp
2908 The cleaner kernel thread also processes defragmentation triggered by the
2909 \fIautodefrag\fP mount option, removing of empty blocks groups and some
2910 other finalization tasks.
2911 .TP
2912 .B copy\-on\-write, COW
2913 Also known as \fICOW\fP\&. The method that btrfs uses for modifying data.
2914 Instead of directly overwriting data in place, btrfs takes a copy of
2915 the data, alters it, and then writes the modified data back to a
2916 different (unused) location on the disk. It then updates the \fImetadata\fP
2917 to reflect the new location of the data. In order to update the
2918 metadata, the affected metadata blocks are also treated in the same
2919 way. In COW filesystems, files tend to fragment as they are modified.
2920 Copy\-on\-write is also used in the implementation of \fIsnapshots\fP and
2921 \fIreflink copies\fP\&. A copy\-on\-write filesystem is, in theory,
2922 \fIalways\fP consistent, provided the underlying hardware supports
2923 \fIbarriers\fP\&.
2924 .TP
2925 .B default subvolume
2926 The \fIsubvolume\fP in a btrfs filesystem which is mounted when mounting
2927 the filesystem without using the \fBsubvol=\fP mount option.
2928 .TP
2929 .B device
2930 A Linux block device, e.g. a whole disk, partition, LVM logical volume,
2931 loopback device, or network block device. A btrfs filesystem can reside
2932 on one or more devices.
2933 .TP
2934 .B df
2935 A standard Unix tool for reporting the amount of space used and free in
2936 a filesystem. The standard tool does not give accurate results, but the
2937 \fIbtrfs\fP command from \fIbtrfs\-progs\fP has
2938 an implementation of \fIdf\fP which shows space available in more detail. See
2939 the
2940 [[FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F|FAQ]]
2941 for a more detailed explanation of btrfs free space accounting.
2942 .TP
2943 .B DUP
2944 A form of \(dq\fIRAID\fP\(dq which stores two copies of each piece of data on
2945 the same \fIdevice\fP\&. This is similar to \fIRAID1\fP, and protects
2946 against \fIblock\fP\-level errors on the device, but does not provide any
2947 guarantees if the entire device fails. By default, btrfs uses \fIDUP\fP
2948 profile for metadata on single device filesystem.s
2949 .TP
2950 .B ENOSPC
2951 Error code returned by the OS to a user program when the filesystem
2952 cannot allocate enough data to fulfill the user request. In most
2953 filesystems, it indicates there is no free space available in the
2954 filesystem. Due to the additional space requirements from btrfs\(aqs
2955 \fICOW\fP behaviour, btrfs can sometimes return ENOSPC when there is
2956 apparently (in terms of \fIdf\fP) a large amount of space free. This is
2957 effectively a bug in btrfs, and (if it is repeatable), using the mount
2958 option \fBenospc_debug\fP may give a report
2959 that will help the btrfs developers. See the
2960 [[FAQ#if_your_device_is_large_.28.3E16GiB.29|FAQ entry]] on free space.
2961 .TP
2962 .B extent
2963 Contiguous sequence of bytes on disk that holds file data. It\(aqs a compact
2964 representation that tracks the start and length of the byte range, so the
2965 logic behind allocating blocks (\fIdelayed allocation\fP) strives for
2966 maximizing the length before writing the extents to the devices.
2967 .TP
2968 .B extent buffer
2969 An abstraction of a \fIb\-tree\fP metadata block storing item keys and item
2970 data. The underlying related structures are physical device block and a
2971 CPU memory page.
2972 .TP
2973 .B fallocate
2974 Command line tool in util\-linux, and a syscall, that reserves space in
2975 the filesystem for a file, without actually writing any file data to
2976 the filesystem. First data write will turn the preallocated extents
2977 into regular ones. See fallocate(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:fallocate\:.1\:.html> and fallocate(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fallocate\:.2\:.html> manual pages
2978 for more details.
2979 .TP
2980 .B filefrag
2981 A tool to show the number of extents in a file, and hence the amount of
2982 fragmentation in the file. It is usually part of the e2fsprogs package
2983 on most Linux distributions. While initially developed for the ext2
2984 filesystem, it works on Btrfs as well. It uses the \fIFIEMAP\fP ioctl.
2985 .TP
2986 .B free space cache
2987 Also known as \(dqspace cache v1\(dq. A separate cache tracking free space as
2988 btrfs only tracks the allocated space. The free space is by definition
2989 any hole between allocated ranges. Finding the free ranges can be
2990 I/O intensive so the cache stores a condensed representation of it.
2991 It is updated every \fItransaction\fP commit.
2992 .sp
2993 The v1 free space cache has been superseded by free space tree.
2994 .TP
2995 .B free space tree
2996 Successor of \fIfree space cache\fP, also known as \(dqspace cache v2\(dq and now
2997 default. The free space is tracked in a better way and using COW
2998 unlike a custom mechanism of v1.
2999 .TP
3000 .B fsync
3001 On Unix and Unix\-like operating systems (of which Linux is the latter),
3002 the fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> system call causes all buffered file
3003 descriptor related data changes to be flushed to the underlying block
3004 device. When a file is modified on a modern operating system the
3005 changes are generally not written to the disk immediately but rather
3006 those changes are buffered in memory for performance reasons,
3007 calling fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> causes any in\-memory changes to be written
3008 to disk.
3009 .TP
3010 .B generation
3011 An internal counter which updates for each \fItransaction\fP\&. When a
3012 \fImetadata\fP block is written (using \fIcopy on write\fP), current
3013 generation is stored in the block, so that blocks which are too new
3014 (and hence possibly inconsistent) can be identified.
3015 .TP
3016 .B key
3017 A fixed sized tuple used to identify and sort items in a \fIB\-tree\fP\&.
3018 The key is broken up into 3 parts: \fIobjectid\fP, \fItype\fP, and
3019 \fIoffset\fP\&. The \fItype\fP field indicates how each of the other two
3020 fields should be used, and what to expect to find in the item.
3021 .TP
3022 .B item
3023 A variable sized structure stored in B\-tree leaves. Items hold
3024 different types of data depending on key type.
3025 .TP
3026 .B log tree
3027 A b\-tree that temporarily tracks ongoing metadata updates until a full
3028 transaction commit is done. It\(aqs a performance optimization of
3029 \fIfsync\fP\&. The log tracked in the tree are replayed if the filesystem
3030 is not unmounted cleanly.
3031 .TP
3032 .B metadata
3033 Data about data. In btrfs, this includes all of the internal data
3034 structures of the filesystem, including directory structures,
3035 filenames, file permissions, checksums, and the location of each file\(aqs
3036 \fIextents\fP\&. All btrfs metadata is stored in \fIB\-trees\fP\&.
3037 .TP
3038 .B mkfs.btrfs
3039 The tool (from \fIbtrfs\-progs\fP) to create a btrfs filesystem.
3040 .TP
3041 .B offline
3042 A filesystem which is not mounted is offline. Some tools (e.g.
3043 \fIbtrfsck\fP) will only work on offline filesystems. Compare \fIonline\fP\&.
3044 .TP
3045 .B online
3046 A filesystem which is mounted is online. Most btrfs tools will only
3047 work on online filesystems. Compare \fIoffline\fP\&.
3048 .TP
3049 .B orphan
3050 A file that\(aqs still in use (opened by a running process) but all
3051 directory entries of that file have been removed.
3052 .TP
3053 .B RAID
3054 A class of different methods for writing some additional redundant data
3055 across multiple \fIdevices\fP so that if one device fails, the missing
3056 data can be reconstructed from the remaining ones. See \fIRAID0\fP,
3057 \fIRAID1\fP, \fIRAID5\fP, \fIRAID6\fP, \fIRAID10\fP, \fIDUP\fP and
3058 \fIsingle\fP\&. Traditional RAID methods operate across multiple devices of
3059 equal size, whereas btrfs\(aq RAID implementation works inside \fIblock
3060 groups\fP\&.
3061 .TP
3062 .B RAID0
3063 A form of \fIRAID\fP which provides no guarantees of error recovery, but
3064 stripes a single copy of data across multiple devices for performance
3065 purposes. The stripe size is fixed to 64KB for now.
3066 .TP
3067 .B RAID1, RAID1C3, RAID1C4
3068 A form of \fIRAID\fP which stores two/three/four complete copies of each
3069 piece of data. Each copy is stored on a different \fIdevice\fP\&. btrfs
3070 requires a minimum of two devices to use RAID\-1 or three/four respectively.
3071 This is the default block group profile for btrfs\(aqs \fImetadata\fP on more
3072 than one device.
3073 .TP
3074 .B RAID5
3075 A form of \fIRAID\fP which stripes a single copy of data across multiple
3076 \fIdevices\fP, including one device\(aqs worth of additional parity data.
3077 Can be used to recover from a single device failure.
3078 .TP
3079 .B RAID6
3080 A form of \fIRAID\fP which stripes a single copy of data across multiple
3081 \fIdevices\fP, including two device\(aqs worth of additional parity data. Can
3082 be used to recover from the failure of two devices.
3083 .TP
3084 .B RAID10
3085 A form of \fIRAID\fP which stores two complete copies of each piece of
3086 data, and also stripes each copy across multiple devices for
3087 performance.
3088 .TP
3089 .B reflink
3090 Commonly used as a reference to a shallow copy of file extents that share
3091 the extents until the first change. Reflinked files (e.g. by the \fBcp\fP)
3092 are different files but point to the same extents, any change will be
3093 detected and new copy of the data created, keeping the files independent.
3094 Related to that is extent range cloning, that works on a range of a file.
3095 .TP
3096 .B relocation
3097 The process of moving block groups within the filesystem while
3098 maintaining full filesystem integrity and consistency. This
3099 functionality is underlying \fIbalance\fP and \fIdevice\fP removing features.
3100 .TP
3101 .B scrub \%<>
3102 An \fIonline\fP filesystem checking tool. Reads all the data and metadata
3103 on the filesystem, verifies \fIchecksums\fP and eventually uses redundant
3104 copies from \fIRAID\fP or \fIDUP\fP repair any corrupt data/metadata.
3105 .TP
3106 .B seed device \%<>
3107 A readonly device can be used as a filesystem seed or template (e.g. a
3108 CD\-ROM containing an OS image). Read/write devices can be added to
3109 store modifications (using \fIcopy on write\fP), changes to the writable
3110 devices are persistent across reboots. The original device remains
3111 unchanged and can be removed at any time (after Btrfs has been
3112 instructed to copy over all missing blocks). Multiple read/write file
3113 systems can be built from the same seed.
3114 .TP
3115 .B single
3116 A block group profile storing a single copy of each piece of data.
3117 .TP
3118 .B snapshot \%<>
3119 A \fIsubvolume\fP which is a \fIcopy on write\fP copy of another subvolume. The
3120 two subvolumes share all of their common (unmodified) data, which means
3121 that snapshots can be used to keep the historical state of a filesystem
3122 very cheaply. After the snapshot is made, the original subvolume and
3123 the snapshot are of equal status: the original does not \(dqown\(dq the
3124 snapshot, and either one can be deleted without affecting the other
3125 one.
3126 .TP
3127 .B subvolume \%<>
3128 A tree of files and directories inside a btrfs that can be mounted as
3129 if it were an independent filesystem. A subvolume is created by taking
3130 a reference on the root of another subvolume. Each btrfs filesystem has
3131 at least one subvolume, the \fItop\-level subvolume\fP, which contains
3132 everything else in the filesystem. Additional subvolumes can be created
3133 and deleted with the \fIbtrfs<\fP tool. All subvolumes share the same pool
3134 of free space in the filesystem. See also \fIdefault subvolume\fP\&.
3135 .TP
3136 .B super block
3137 A special metadata block that is a main access point of the filesystem
3138 structures. It\(aqs size is fixed and there are fixed locations on the devices
3139 used for detecting and opening the filesystem. Updating the superblock
3140 defines one \fItransaction\fP\&. The super blocks contains filesystem
3141 identification (UUID), checksum type, block pointers to fundamental
3142 trees, features and creation parameters.
3143 .TP
3144 .B system array
3145 A technical term for \fIsuper block\fP metadata describing how to assemble a
3146 filesystem from multiple device, storing information about chunks and devices that are
3147 required to be scanned/registered at the time the mount happens.
3148 Scanning is done by command \fBbtrfs device scan\fP, alternatively
3149 all the required devices can be specified by a mount option \fIdevice=/path\fP\&.
3150 .TP
3151 .B top\-level subvolume
3152 The \fIsubvolume\fP at the very top of the filesystem. This is the only
3153 subvolume present in a newly\-created btrfs filesystem, and internally has ID 5,
3154 otherwise could be referenced as 0 (e.g. within the \fIset\-default\fP subcommand of
3155 \fIbtrfs\fP).
3156 .TP
3157 .B transaction
3158 A consistent set of changes. To avoid generating very large amounts of
3159 disk activity, btrfs caches changes in RAM for up to 30 seconds
3160 (sometimes more often if the filesystem is running short on space or
3161 doing a lot of \fIfsync*s), and then writes (commits) these changes out
3162 to disk in one go (using *copy on write\fP behaviour). This period of
3163 caching is called a transaction. Only one transaction is active on the
3164 filesystem at any one time.
3165 .TP
3166 .B transid
3167 An alternative term for \fIgeneration\fP\&.
3168 .TP
3169 .B writeback
3170 \fIWriteback\fP in the context of the Linux kernel can be defined as the
3171 process of writing \(dqdirty\(dq memory from the page cache to the disk,
3172 when certain conditions are met (timeout, number of dirty pages over a
3173 ratio).
3174 .UNINDENT
3175 .SH STORAGE MODEL, HARDWARE CONSIDERATIONS
3176 .SS Storage model
3177 .sp
3178 \fIA storage model is a model that captures key physical aspects of data
3179 structure in a data store. A filesystem is the logical structure organizing
3180 data on top of the storage device.\fP
3181 .sp
3182 The filesystem assumes several features or limitations of the storage device
3183 and utilizes them or applies measures to guarantee reliability. BTRFS in
3184 particular is based on a COW (copy on write) mode of writing, i.e. not updating
3185 data in place but rather writing a new copy to a different location and then
3186 atomically switching the pointers.
3187 .sp
3188 In an ideal world, the device does what it promises. The filesystem assumes
3189 that this may not be true so additional mechanisms are applied to either detect
3190 misbehaving hardware or get valid data by other means. The devices may (and do)
3191 apply their own detection and repair mechanisms but we won\(aqt assume any.
3192 .sp
3193 The following assumptions about storage devices are considered (sorted by
3194 importance, numbers are for further reference):
3195 .INDENT 0.0
3196 .IP 1. 3
3197 atomicity of reads and writes of blocks/sectors (the smallest unit of data
3198 the device presents to the upper layers)
3199 .IP 2. 3
3200 there\(aqs a flush command that instructs the device to forcibly order writes
3201 before and after the command; alternatively there\(aqs a barrier command that
3202 facilitates the ordering but may not flush the data
3203 .IP 3. 3
3204 data sent to write to a given device offset will be written without further
3205 changes to the data and to the offset
3206 .IP 4. 3
3207 writes can be reordered by the device, unless explicitly serialized by the
3208 flush command
3209 .IP 5. 3
3210 reads and writes can be freely reordered and interleaved
3211 .UNINDENT
3212 .sp
3213 The consistency model of BTRFS builds on these assumptions. The logical data
3214 updates are grouped, into a generation, written on the device, serialized by
3215 the flush command and then the super block is written ending the generation.
3216 All logical links among metadata comprising a consistent view of the data may
3217 not cross the generation boundary.
3218 .SS When things go wrong
3219 .sp
3220 \fBNo or partial atomicity of block reads/writes (1)\fP
3221 .INDENT 0.0
3222 .IP \(bu 2
3223 \fIProblem\fP: a partial block contents is written (\fItorn write\fP), e.g. due to a
3224 power glitch or other electronics failure during the read/write
3225 .IP \(bu 2
3226 \fIDetection\fP: checksum mismatch on read
3227 .IP \(bu 2
3228 \fIRepair\fP: use another copy or rebuild from multiple blocks using some encoding
3229 scheme
3230 .UNINDENT
3231 .sp
3232 \fBThe flush command does not flush (2)\fP
3233 .sp
3234 This is perhaps the most serious problem and impossible to mitigate by
3235 filesystem without limitations and design restrictions. What could happen in
3236 the worst case is that writes from one generation bleed to another one, while
3237 still letting the filesystem consider the generations isolated. Crash at any
3238 point would leave data on the device in an inconsistent state without any hint
3239 what exactly got written, what is missing and leading to stale metadata link
3240 information.
3241 .sp
3242 Devices usually honor the flush command, but for performance reasons may do
3243 internal caching, where the flushed data are not yet persistently stored. A
3244 power failure could lead to a similar scenario as above, although it\(aqs less
3245 likely that later writes would be written before the cached ones. This is
3246 beyond what a filesystem can take into account. Devices or controllers are
3247 usually equipped with batteries or capacitors to write the cache contents even
3248 after power is cut. (\fIBattery backed write cache\fP)
3249 .sp
3250 \fBData get silently changed on write (3)\fP
3251 .sp
3252 Such thing should not happen frequently, but still can happen spuriously due
3253 the complex internal workings of devices or physical effects of the storage
3254 media itself.
3255 .INDENT 0.0
3256 .IP \(bu 2
3257 \fIProblem\fP: while the data are written atomically, the contents get changed
3258 .IP \(bu 2
3259 \fIDetection\fP: checksum mismatch on read
3260 .IP \(bu 2
3261 \fIRepair\fP: use another copy or rebuild from multiple blocks using some
3262 encoding scheme
3263 .UNINDENT
3264 .sp
3265 \fBData get silently written to another offset (3)\fP
3266 .sp
3267 This would be another serious problem as the filesystem has no information
3268 when it happens. For that reason the measures have to be done ahead of time.
3269 This problem is also commonly called \fIghost write\fP\&.
3270 .sp
3271 The metadata blocks have the checksum embedded in the blocks, so a correct
3272 atomic write would not corrupt the checksum. It\(aqs likely that after reading
3273 such block the data inside would not be consistent with the rest. To rule that
3274 out there\(aqs embedded block number in the metadata block. It\(aqs the logical
3275 block number because this is what the logical structure expects and verifies.
3276 .sp
3277 The following is based on information publicly available, user feedback,
3278 community discussions or bug report analyses. It\(aqs not complete and further
3279 research is encouraged when in doubt.
3280 .SS Main memory
3281 .sp
3282 The data structures and raw data blocks are temporarily stored in computer
3283 memory before they get written to the device. It is critical that memory is
3284 reliable because even simple bit flips can have vast consequences and lead to
3285 damaged structures, not only in the filesystem but in the whole operating
3286 system.
3287 .sp
3288 Based on experience in the community, memory bit flips are more common than one
3289 would think. When it happens, it\(aqs reported by the tree\-checker or by a checksum
3290 mismatch after reading blocks. There are some very obvious instances of bit
3291 flips that happen, e.g. in an ordered sequence of keys in metadata blocks. We can
3292 easily infer from the other data what values get damaged and how. However, fixing
3293 that is not straightforward and would require cross\-referencing data from the
3294 entire filesystem to see the scope.
3295 .sp
3296 If available, ECC memory should lower the chances of bit flips, but this
3297 type of memory is not available in all cases. A memory test should be performed
3298 in case there\(aqs a visible bit flip pattern, though this may not detect a faulty
3299 memory module because the actual load of the system could be the factor making
3300 the problems appear. In recent years attacks on how the memory modules operate
3301 have been demonstrated (\fIrowhammer\fP) achieving specific bits to be flipped.
3302 While these were targeted, this shows that a series of reads or writes can
3303 affect unrelated parts of memory.
3304 .sp
3305 Block group profiles with redundancy (like RAID1) will not protect against
3306 memory errors as the blocks are first stored in memory before they are written
3307 to the devices from the same source.
3308 .sp
3309 A filesystem mounted read\-only will not affect the underlying block device in
3310 almost 100% (with highly unlikely exceptions). The exception is a tree\-log that
3311 needs to be replayed during mount (and before the read\-only mount takes place),
3312 working memory is needed for that and that can be affected by bit flips.
3313 There\(aqs a theoretical case where bit flip changes the filesystem status from
3314 read\-only to read\-write.
3315 .sp
3316 Further reading:
3317 .INDENT 0.0
3318 .IP \(bu 2
3319 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Row_hammer>
3320 .IP \(bu 2
3321 memory overclocking, XMP, potential risks
3322 .UNINDENT
3323 .sp
3324 What to do:
3325 .INDENT 0.0
3326 .IP \(bu 2
3327 run \fImemtest\fP, note that sometimes memory errors happen only when the system
3328 is under heavy load that the default memtest cannot trigger
3329 .IP \(bu 2
3330 memory errors may appear as filesystem going read\-only due to \(dqpre write\(dq
3331 check, that verify meta data before they get written but fail some basic
3332 consistency checks
3333 .IP \(bu 2
3334 newly built systems should be tested before being put to production use,
3335 ideally start a IO/CPU load that will be run on such system later; namely
3336 systems that will utilize overclocking or special performance features
3337 .UNINDENT
3338 .SS Direct memory access (DMA)
3339 .sp
3340 Another class of errors is related to DMA (direct memory access) performed
3341 by device drivers. While this could be considered a software error, the
3342 data transfers that happen without CPU assistance may accidentally corrupt
3343 other pages. Storage devices utilize DMA for performance reasons, the
3344 filesystem structures and data pages are passed back and forth, making
3345 errors possible in case page life time is not properly tracked.
3346 .sp
3347 There are lots of quirks (device\-specific workarounds) in Linux kernel
3348 drivers (regarding not only DMA) that are added when found. The quirks
3349 may avoid specific errors or disable some features to avoid worse problems.
3350 .sp
3351 What to do:
3352 .INDENT 0.0
3353 .IP \(bu 2
3354 use up\-to\-date kernel (recent releases or maintained long term support versions)
3355 .IP \(bu 2
3356 as this may be caused by faulty drivers, keep the systems up\-to\-date
3357 .UNINDENT
3358 .SS Rotational disks (HDD)
3359 .sp
3360 Rotational HDDs typically fail at the level of individual sectors or small clusters.
3361 Read failures are caught on the levels below the filesystem and are returned to
3362 the user as \fIEIO \- Input/output error\fP\&. Reading the blocks repeatedly may
3363 return the data eventually, but this is better done by specialized tools and
3364 filesystem takes the result of the lower layers. Rewriting the sectors may
3365 trigger internal remapping but this inevitably leads to data loss.
3366 .sp
3367 Disk firmware is technically software but from the filesystem perspective is
3368 part of the hardware. IO requests are processed, and caching or various
3369 other optimizations are performed, which may lead to bugs under high load or
3370 unexpected physical conditions or unsupported use cases.
3371 .sp
3372 Disks are connected by cables with two ends, both of which can cause problems
3373 when not attached properly. Data transfers are protected by checksums and the
3374 lower layers try hard to transfer the data correctly or not at all. The errors
3375 from badly\-connecting cables may manifest as large amount of failed read or
3376 write requests, or as short error bursts depending on physical conditions.
3377 .sp
3378 What to do:
3379 .INDENT 0.0
3380 .IP \(bu 2
3381 check \fBsmartctl\fP for potential issues
3382 .UNINDENT
3383 .SS Solid state drives (SSD)
3384 .sp
3385 The mechanism of information storage is different from HDDs and this affects
3386 the failure mode as well. The data are stored in cells grouped in large blocks
3387 with limited number of resets and other write constraints. The firmware tries
3388 to avoid unnecessary resets and performs optimizations to maximize the storage
3389 media lifetime. The known techniques are deduplication (blocks with same
3390 fingerprint/hash are mapped to same physical block), compression or internal
3391 remapping and garbage collection of used memory cells. Due to the additional
3392 processing there are measures to verify the data e.g. by ECC codes.
3393 .sp
3394 The observations of failing SSDs show that the whole electronic fails at once
3395 or affects a lot of data (e.g. stored on one chip). Recovering such data
3396 may need specialized equipment and reading data repeatedly does not help as
3397 it\(aqs possible with HDDs.
3398 .sp
3399 There are several technologies of the memory cells with different
3400 characteristics and price. The lifetime is directly affected by the type and
3401 frequency of data written. Writing \(dqtoo much\(dq distinct data (e.g. encrypted)
3402 may render the internal deduplication ineffective and lead to a lot of rewrites
3403 and increased wear of the memory cells.
3404 .sp
3405 There are several technologies and manufacturers so it\(aqs hard to describe them
3406 but there are some that exhibit similar behaviour:
3407 .INDENT 0.0
3408 .IP \(bu 2
3409 expensive SSD will use more durable memory cells and is optimized for
3410 reliability and high load
3411 .IP \(bu 2
3412 cheap SSD is projected for a lower load (\(dqdesktop user\(dq) and is optimized for
3413 cost, it may employ the optimizations and/or extended error reporting
3414 partially or not at all
3415 .UNINDENT
3416 .sp
3417 It\(aqs not possible to reliably determine the expected lifetime of an SSD due to
3418 lack of information about how it works or due to lack of reliable stats provided
3419 by the device.
3420 .sp
3421 Metadata writes tend to be the biggest component of lifetime writes to a SSD,
3422 so there is some value in reducing them. Depending on the device class (high
3423 end/low end) the features like DUP block group profiles may affect the
3424 reliability in both ways:
3425 .INDENT 0.0
3426 .IP \(bu 2
3427 \fIhigh end\fP are typically more reliable and using \fIsingle\fP for data and
3428 metadata could be suitable to reduce device wear
3429 .IP \(bu 2
3430 \fIlow end\fP could lack ability to identify errors so an additional redundancy
3431 at the filesystem level (checksums, \fIDUP\fP) could help
3432 .UNINDENT
3433 .sp
3434 Only users who consume 50 to 100% of the SSD\(aqs actual lifetime writes need to be
3435 concerned by the write amplification of btrfs DUP metadata. Most users will be
3436 far below 50% of the actual lifetime, or will write the drive to death and
3437 discover how many writes 100% of the actual lifetime was. SSD firmware often
3438 adds its own write multipliers that can be arbitrary and unpredictable and
3439 dependent on application behavior, and these will typically have far greater
3440 effect on SSD lifespan than DUP metadata. It\(aqs more or less impossible to
3441 predict when a SSD will run out of lifetime writes to within a factor of two, so
3442 it\(aqs hard to justify wear reduction as a benefit.
3443 .sp
3444 Further reading:
3445 .INDENT 0.0
3446 .IP \(bu 2
3447 \%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-and-deduplication-end-spinning-disk-2012>
3448 .IP \(bu 2
3449 \%<https://\:www\:.snia\:.org/\:educational-library/\:realities-solid-state-storage-2013-2013>
3450 .IP \(bu 2
3451 \%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-performance-primer-2013>
3452 .IP \(bu 2
3453 \%<https://\:www\:.snia\:.org/\:educational-library/\:how-controllers-maximize-ssd-life-2013>
3454 .UNINDENT
3455 .sp
3456 What to do:
3457 .INDENT 0.0
3458 .IP \(bu 2
3459 run \fBsmartctl\fP or self\-tests to look for potential issues
3460 .IP \(bu 2
3461 keep the firmware up\-to\-date
3462 .UNINDENT
3463 .SS NVM express, non\-volatile memory (NVMe)
3464 .sp
3465 NVMe is a type of persistent memory usually connected over a system bus (PCIe)
3466 or similar interface and the speeds are an order of magnitude faster than SSD.
3467 It is also a non\-rotating type of storage, and is not typically connected by a
3468 cable. It\(aqs not a SCSI type device either but rather a complete specification
3469 for logical device interface.
3470 .sp
3471 In a way the errors could be compared to a combination of SSD class and regular
3472 memory. Errors may exhibit as random bit flips or IO failures. There are tools
3473 to access the internal log (\fBnvme log\fP and \fBnvme\-cli\fP) for a more detailed
3474 analysis.
3475 .sp
3476 There are separate error detection and correction steps performed e.g. on the
3477 bus level and in most cases never making in to the filesystem level. Once this
3478 happens it could mean there\(aqs some systematic error like overheating or bad
3479 physical connection of the device. You may want to run self\-tests (using
3480 \fBsmartctl\fP).
3481 .INDENT 0.0
3482 .IP \(bu 2
3483 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:NVM_Express>
3484 .IP \(bu 2
3485 \%<https://\:www\:.smartmontools\:.org/\:wiki/\:NVMe_Support>
3486 .UNINDENT
3487 .SS Drive firmware
3488 .sp
3489 Firmware is technically still software but embedded into the hardware. As all
3490 software has bugs, so does firmware. Storage devices can update the firmware
3491 and fix known bugs. In some cases the it\(aqs possible to avoid certain bugs by
3492 quirks (device\-specific workarounds) in Linux kernel.
3493 .sp
3494 A faulty firmware can cause wide range of corruptions from small and localized
3495 to large affecting lots of data. Self\-repair capabilities may not be sufficient.
3496 .sp
3497 What to do:
3498 .INDENT 0.0
3499 .IP \(bu 2
3500 check for firmware updates in case there are known problems, note that
3501 updating firmware can be risky on itself
3502 .IP \(bu 2
3503 use up\-to\-date kernel (recent releases or maintained long term support versions)
3504 .UNINDENT
3505 .SS SD flash cards
3506 .sp
3507 There are a lot of devices with low power consumption and thus using storage
3508 media based on low power consumption too, typically flash memory stored on
3509 a chip enclosed in a detachable card package. An improperly inserted card may be
3510 damaged by electrical spikes when the device is turned on or off. The chips
3511 storing data in turn may be damaged permanently. All types of flash memory
3512 have a limited number of rewrites, so the data are internally translated by FTL
3513 (flash translation layer). This is implemented in firmware (technically a
3514 software) and prone to bugs that manifest as hardware errors.
3515 .sp
3516 Adding redundancy like using DUP profiles for both data and metadata can help
3517 in some cases but a full backup might be the best option once problems appear
3518 and replacing the card could be required as well.
3519 .SS Hardware as the main source of filesystem corruptions
3520 .sp
3521 \fBIf you use unreliable hardware and don\(aqt know about that, don\(aqt blame the
3522 filesystem when it tells you.\fP
3523 .SH SEE ALSO
3524 .sp
3525 acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html>,
3526 btrfs(8) \%<>,
3527 chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>,
3528 fstrim(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:fstrim\:.8\:.html>,
3529 ioctl(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:ioctl\:.2\:.html>,
3530 btrfs\-ioctl(2) \%<>,
3531 mkfs.btrfs(8) \%<>,
3532 mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html>,
3533 swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html>
3534 .\" End of generated man page.