Debian Codegraph @

upstream/debian-unstable/man5/btrfs.5

raw
1 '\" t
2 .\" Man page generated from reStructuredText
3 .\" by the Docutils 0.22.4 manpage writer.
4 .
5 .
6 .nr rst2man-indent-level 0
7 .
8 .de1 rstReportMargin
9 \\$1 \\n[an-margin]
10 level \\n[rst2man-indent-level]
11 level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
12 -
13 \\n[rst2man-indent0]
14 \\n[rst2man-indent1]
15 \\n[rst2man-indent2]
16 ..
17 .de1 INDENT
18 .\" .rstReportMargin pre:
19 . RS \\$1
20 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
21 . nr rst2man-indent-level +1
22 .\" .rstReportMargin post:
23 ..
24 .de UNINDENT
25 . RE
26 .\" indent \\n[an-margin]
27 .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
28 .nr rst2man-indent-level -1
29 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
30 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
31 ..
32 .TH "BTRFS" "5" "May 10, 2026" "7.0" "BTRFS"
33 .SH NAME
34 btrfs \- topics about the BTRFS filesystem (mount options, supported file attributes and other)
35 .SH DESCRIPTION
36 .sp
37 This document describes topics related to BTRFS that are not specific to the
38 tools. Currently covers:
39 .INDENT 0.0
40 .IP 1. 4
41 mount options
42 .IP 2. 4
43 filesystem features
44 .IP 3. 4
45 swapfile support
46 .IP 4. 4
47 checksum algorithms
48 .IP 5. 4
49 compression
50 .IP 6. 4
51 sysfs interface
52 .IP 7. 4
53 filesystem exclusive operations
54 .IP 8. 4
55 filesystem limits
56 .IP 9. 4
57 bootloader support
58 .IP 10. 4
59 file attributes
60 .IP 11. 4
61 zoned mode
62 .IP 12. 4
63 control device
64 .IP 13. 4
65 filesystems with multiple block group profiles
66 .IP 14. 4
67 seeding device
68 .IP 15. 4
69 RAID56 status and recommended practices
70 .IP 16. 4
71 glossary
72 .IP 17. 4
73 storage model, hardware considerations
74 .UNINDENT
75 .SH MOUNT OPTIONS
76 .SS BTRFS SPECIFIC MOUNT OPTIONS
77 .sp
78 This section describes mount options specific to BTRFS. For the generic mount
79 options please refer to mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> manual page and also see the section
80 with BTRFS specifics below \%<btrfs-man5//#\:mount-options-generic>\&. The options are
81 sorted alphabetically (discarding the \fIno\fP prefix).
82 .sp
83 \fBNote:\fP
84 .INDENT 0.0
85 .INDENT 3.5
86 Most mount options apply to the whole filesystem and only options in the
87 first mounted subvolume will take effect. This is due to lack of implementation
88 and may change in the future. This means that (for example) you can\(aqt set
89 per\-subvolume \fInodatacow\fP, \fInodatasum\fP, or \fIcompress\fP using mount options. This
90 should eventually be fixed, but it has proved to be difficult to implement
91 correctly within the Linux VFS framework.
92 .UNINDENT
93 .UNINDENT
94 .sp
95 Mount options are processed in order, only the last occurrence of an option
96 takes effect and may disable other options due to constraints (see e.g.
97 \fInodatacow\fP and \fIcompress\fP). The output of \fBmount\fP command shows which options
98 have been applied.
99 .INDENT 0.0
100 .TP
101 .B acl, noacl
102 (default: on)
103 .sp
104 Enable/disable support for POSIX Access Control Lists (ACLs). See the
105 acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html> manual page for more information about ACLs.
106 .sp
107 The support for ACL is build\-time configurable (BTRFS_FS_POSIX_ACL) and
108 mount fails if \fIacl\fP is requested but the feature is not compiled in.
109 .UNINDENT
110 .INDENT 0.0
111 .TP
112 .B autodefrag, noautodefrag
113 (since: 3.0, default: off)
114 .sp
115 Enable automatic file defragmentation.
116 When enabled, small random writes into files (in a range of tens of kilobytes,
117 currently it\(aqs 64KiB) are detected and queued up for the defragmentation process.
118 May not be well suited for large database workloads.
119 .sp
120 The read latency may increase due to reading the adjacent blocks that make up the
121 range for defragmentation, successive write will merge the blocks in the new
122 location.
123 .sp
124 \fBWarning:\fP
125 .INDENT 7.0
126 .INDENT 3.5
127 Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14\-rc2 as
128 well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or
129 ≥ 3.13.4 will break up the reflinks of COW data (for example files
130 copied with \fBcp \-\-reflink\fP, snapshots or de\-duplicated data).
131 This may cause considerable increase of space usage depending on the
132 broken up reflinks.
133 .UNINDENT
134 .UNINDENT
135 .TP
136 .B barrier, nobarrier
137 (default: on)
138 .sp
139 Ensure that all IO write operations make it through the device cache and are stored
140 permanently when the filesystem is at its consistency checkpoint. This
141 typically means that a flush command is sent to the device that will
142 synchronize all pending data and ordinary metadata blocks, then writes the
143 superblock and issues another flush.
144 .sp
145 The write flushes incur a slight hit and also prevent the IO block
146 scheduler to reorder requests in a more effective way. Disabling barriers gets
147 rid of that penalty but will most certainly lead to a corrupted filesystem in
148 case of a crash or power loss. The ordinary metadata blocks could be yet
149 unwritten at the time the new superblock is stored permanently, expecting that
150 the block pointers to metadata were stored permanently before.
151 .sp
152 On a device with a volatile battery\-backed write\-back cache, the \fInobarrier\fP
153 option will not lead to filesystem corruption as the pending blocks are
154 supposed to make it to the permanent storage.
155 .TP
156 .B clear_cache
157 Force clearing and rebuilding of the free space cache if something
158 has gone wrong.
159 This option only takes effect on the first read\-write mount.
160 .sp
161 For free space cache \fIv1\fP, this only clears (and, unless \fInospace_cache\fP is
162 used, rebuilds) the free space cache for block groups that are modified while
163 the filesystem is mounted with that option. To actually clear an entire free
164 space cache \fIv1\fP, see \fBbtrfs check \-\-clear\-space\-cache v1\fP\&.
165 .sp
166 For free space cache \fIv2\fP, this clears the entire free space cache.
167 To do so without requiring to mounting the filesystem, see
168 \fBbtrfs check \-\-clear\-space\-cache v2\fP\&.
169 .sp
170 See also: \fIspace_cache\fP\&.
171 .TP
172 .B commit=<seconds>
173 (since: 3.12, default: 30)
174 .sp
175 Set the interval of periodic transaction commit when data are synchronized
176 to permanent storage. Higher interval values lead to larger amount of unwritten
177 data to accumulate in memory, which has obvious consequences when the
178 system crashes. The upper bound is not forced, but a warning is
179 printed if it\(aqs more than 300 seconds (5 minutes). Use with care.
180 .sp
181 The periodic commit is not the only mechanism to do the transaction commit,
182 this can also happen by explicit \fBsync\fP or indirectly by other
183 commands that affect the global filesystem state or internal kernel
184 mechanisms that flush based on various thresholds or policies (e.g. cgroups).
185 .TP
186 .B compress, compress=<type[:level]>, compress\-force, compress\-force=<type[:level]>
187 (default: off, level support since: 5.1)
188 .sp
189 Control BTRFS file data compression. Type may be specified as \fIzlib\fP,
190 \fIlzo\fP, \fIzstd\fP or \fIno\fP (for no compression, used for remounting). If no type
191 is specified, \fIzlib\fP is used. If \fIcompress\-force\fP is specified,
192 then compression will always be attempted, but the data may end up uncompressed
193 if the compression would make them larger.
194 .sp
195 Both \fIzlib\fP and \fIzstd\fP (since version 5.1) expose the compression level as a
196 tunable knob with higher levels trading speed and memory (\fIzstd\fP) for higher
197 compression ratios. This can be set by appending a colon and the desired level.
198 ZLIB accepts the range [1, 9] and ZSTD accepts [1, 15]. If no level is set,
199 both currently use a default level of 3. The value 0 is an alias for the
200 default level.
201 .sp
202 Otherwise some simple heuristics are applied to detect an incompressible file.
203 If the first blocks written to a file are not compressible, the whole file is
204 permanently marked to skip compression. As this is too simple, the
205 \fIcompress\-force\fP is a workaround that will compress most of the files at the
206 cost of some wasted CPU cycles on failed attempts.
207 Since kernel 4.15, a set of heuristic algorithms have been improved by using
208 frequency sampling, repeated pattern detection and Shannon entropy calculation
209 to avoid that.
210 .sp
211 \fBNote:\fP
212 .INDENT 7.0
213 .INDENT 3.5
214 If compression is enabled, \fInodatacow\fP and \fInodatasum\fP are disabled.
215 .UNINDENT
216 .UNINDENT
217 .TP
218 .B datacow, nodatacow
219 (default: on)
220 .sp
221 Enable data copy\-on\-write for newly created files.
222 \fINodatacow\fP implies \fInodatasum\fP, and disables \fIcompression\fP\&. All files created
223 under \fInodatacow\fP are also set the NOCOW file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>).
224 .sp
225 \fBNote:\fP
226 .INDENT 7.0
227 .INDENT 3.5
228 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
229 .UNINDENT
230 .UNINDENT
231 .sp
232 Updates in\-place improve performance for workloads that do frequent overwrites,
233 at the cost of potential partial writes, in case the write is interrupted
234 (system crash, device failure).
235 .TP
236 .B datasum, nodatasum
237 (default: on)
238 .sp
239 Enable data checksumming for newly created files.
240 \fIDatasum\fP implies \fIdatacow\fP, i.e. the normal mode of operation. All files created
241 under \fInodatasum\fP inherit the \(dqno checksums\(dq property, however there\(aqs no
242 corresponding file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>).
243 .sp
244 \fBNote:\fP
245 .INDENT 7.0
246 .INDENT 3.5
247 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
248 .UNINDENT
249 .UNINDENT
250 .sp
251 There is a slight performance gain when checksums are turned off, the
252 corresponding metadata blocks holding the checksums do not need to updated.
253 The cost of checksumming of the blocks in memory is much lower than the IO,
254 modern CPUs feature hardware support of the checksumming algorithm.
255 .UNINDENT
256 .INDENT 0.0
257 .TP
258 .B degraded
259 (default: off)
260 .sp
261 Allow mounts with fewer devices than the RAID profile constraints
262 require. A read\-write mount (or remount) may fail when there are too many devices
263 missing, for example if a stripe member is completely missing from RAID0.
264 .sp
265 Since 4.14, the constraint checks have been improved and are verified on the
266 chunk level, not at the device level. This allows degraded mounts of
267 filesystems with mixed RAID profiles for data and metadata, even if the
268 device number constraints would not be satisfied for some of the profiles.
269 .sp
270 Example: metadata \-\- raid1, data \-\- single, devices \-\- \fB/dev/sda\fP, \fB/dev/sdb\fP
271 .sp
272 Suppose the data are completely stored on \fIsda\fP, then missing \fIsdb\fP will not
273 prevent the mount, even if 1 missing device would normally prevent (any)
274 \fIsingle\fP profile to mount. In case some of the data chunks are stored on \fIsdb\fP,
275 then the constraint of single/data is not satisfied and the filesystem
276 cannot be mounted.
277 .UNINDENT
278 .INDENT 0.0
279 .TP
280 .B device=<devicepath>
281 Specify a path to a device that will be scanned for BTRFS filesystem during
282 mount. This is usually done automatically by a device manager (like udev) or
283 using the \fBbtrfs device scan\fP command (e.g. run from the initial ramdisk). In
284 cases where this is not possible the \fIdevice\fP mount option can help.
285 .sp
286 \fBNote:\fP
287 .INDENT 7.0
288 .INDENT 3.5
289 Booting e.g. a RAID1 system may fail even if all filesystem\(aqs \fIdevice\fP
290 paths are provided as the actual device nodes may not be discovered by the
291 system at that point.
292 .UNINDENT
293 .UNINDENT
294 .TP
295 .B discard, discard=sync, discard=async, nodiscard
296 (default: async when devices support it since 6.2, async support since: 5.6)
297 .sp
298 Enable discarding of freed file blocks. This is useful for SSD/NVMe
299 devices, thinly provisioned LUNs, or virtual machine images; however,
300 every storage layer must support discard for it to work.
301 .sp
302 In the synchronous mode (\fIsync\fP or without option value), lack of asynchronous
303 queued TRIM on the backing device TRIM can severely degrade performance,
304 because a synchronous TRIM operation will be attempted instead. Queued TRIM
305 requires SATA devices with chipsets revision newer than 3.1 and devices.
306 .sp
307 The asynchronous mode (\fIasync\fP) gathers extents in larger chunks before sending
308 them to the devices for TRIM. The overhead and performance impact should be
309 negligible compared to the previous mode and it\(aqs supposed to be the preferred
310 mode if needed.
311 .sp
312 If it is not necessary to immediately discard freed blocks, then the \fBfstrim\fP
313 tool can be used to discard all free blocks in a batch. Scheduling a TRIM
314 during a period of low system activity will prevent latent interference with
315 the performance of other operations. Also, a device may ignore the TRIM command
316 if the range is too small, so running a batch discard has a greater probability
317 of actually discarding the blocks.
318 .TP
319 .B enospc_debug, noenospc_debug
320 (default: off)
321 .sp
322 Enable verbose output for some ENOSPC conditions. It\(aqs safe to use but can
323 be noisy if the system reaches near\-full state.
324 .TP
325 .B fatal_errors=<action>
326 (since: 3.4, default: bug)
327 .sp
328 Action to take when encountering a fatal error.
329 .INDENT 7.0
330 .TP
331 .B bug
332 \fIBUG()\fP on a fatal error, the system will stay in the crashed state and may be
333 still partially usable, but reboot is required for full operation
334 .TP
335 .B panic
336 \fIpanic()\fP on a fatal error, depending on other system configuration, this may
337 be followed by a reboot. Please refer to the documentation of kernel boot
338 parameters, e.g. \fIpanic\fP, \fIoops\fP or \fIcrashkernel\fP\&.
339 .UNINDENT
340 .TP
341 .B flushoncommit, noflushoncommit
342 (default: off)
343 .sp
344 This option forces any data dirtied by a write in a prior transaction to commit
345 as part of the current commit, effectively a full filesystem sync.
346 .sp
347 This makes the committed state a fully consistent view of the file system from
348 the application\(aqs perspective (i.e. it includes all completed file system
349 operations). This was previously the behavior only when a snapshot was
350 created.
351 .sp
352 When off, the filesystem is consistent but buffered writes may last more than
353 one transaction commit.
354 .TP
355 .B fragment=<type>
356 (depends on compile\-time option CONFIG_BTRFS_DEBUG, since: 4.4, default: off)
357 .sp
358 A debugging helper to intentionally fragment given \fItype\fP of block groups. The
359 type can be \fIdata\fP, \fImetadata\fP or \fIall\fP\&. This mount option should not be used
360 outside of debugging environments and is not recognized if the kernel config
361 option \fICONFIG_BTRFS_DEBUG\fP is not enabled.
362 .TP
363 .B nologreplay
364 (default: off, even read\-only)
365 .sp
366 The tree\-log contains pending updates to the filesystem until the full commit.
367 The log is replayed on next mount, this can be disabled by this option. See
368 also \fItreelog\fP\&. Note that \fInologreplay\fP is the same as \fInorecovery\fP\&.
369 .sp
370 \fBWarning:\fP
371 .INDENT 7.0
372 .INDENT 3.5
373 Currently, the tree log is replayed even with a read\-only mount! To
374 disable that behaviour, mount also with \fInologreplay\fP\&.
375 .UNINDENT
376 .UNINDENT
377 .TP
378 .B max_inline=<bytes>
379 (default: min(2048, page size) )
380 .sp
381 Specify the maximum amount of space, that can be inlined in
382 a metadata b\-tree leaf. The value is specified in bytes, optionally
383 with a K suffix (case insensitive). In practice, this value
384 is limited by the filesystem block size (named \fIsectorsize\fP at mkfs time),
385 and memory page size of the system. In case of sectorsize limit, there\(aqs
386 some space unavailable due to b\-tree leaf headers. For example, a 4KiB
387 sectorsize, maximum size of inline data is about 3900 bytes.
388 .sp
389 Inlining can be completely turned off by specifying 0. This will increase data
390 block slack if file sizes are much smaller than block size but will reduce
391 metadata consumption in return.
392 .sp
393 \fBNote:\fP
394 .INDENT 7.0
395 .INDENT 3.5
396 The default value has changed to 2048 in kernel 4.6.
397 .UNINDENT
398 .UNINDENT
399 .TP
400 .B metadata_ratio=<value>
401 (default: 0, internal logic)
402 .sp
403 Specifies that 1 metadata chunk should be allocated after every \fIvalue\fP data
404 chunks. Default behaviour depends on internal logic, some percent of unused
405 metadata space is attempted to be maintained but is not always possible if
406 there\(aqs not enough space left for chunk allocation. The option could be useful to
407 override the internal logic in favor of the metadata allocation if the expected
408 workload is supposed to be metadata intense (snapshots, reflinks, xattrs,
409 inlined files).
410 .TP
411 .B norecovery
412 (since: 4.5, default: off)
413 .sp
414 Do not attempt any data recovery at mount time. This will disable \fIlogreplay\fP
415 and avoids other write operations. Note that this option is the same as
416 \fInologreplay\fP\&.
417 .sp
418 \fBNote:\fP
419 .INDENT 7.0
420 .INDENT 3.5
421 The opposite option \fIrecovery\fP used to have different meaning but was
422 changed for consistency with other filesystems, where \fInorecovery\fP is used for
423 skipping log replay. BTRFS does the same and in general will try to avoid any
424 write operations.
425 .UNINDENT
426 .UNINDENT
427 .UNINDENT
428 .INDENT 0.0
429 .TP
430 .B rescan_uuid_tree
431 (since: 3.12, default: off)
432 .sp
433 Force check and rebuild procedure of the UUID tree. This should not
434 normally be needed. Alternatively the tree can be cleared from
435 userspace by command btrfs rescue clear\-uuid\-tree \%<#\:man-rescue-clear-uuid-tree>
436 and then it will be automatically rebuilt in kernel (the mount option
437 is not needed in that case).
438 .TP
439 .B rescue
440 (since: 5.9)
441 .sp
442 Modes allowing mount with damaged filesystem structures, all requires
443 the filesystem to be mounted read\-only and doesn\(aqt allow remount to read\-write.
444 This is supposed to provide unified and more fine grained tuning of
445 errors that affect filesystem operation.
446 .INDENT 7.0
447 .IP \(bu 2
448 \fIusebackuproot\fP (since 5.9)
449 .sp
450 Try to use backup root slots inside super block.
451 Replaces standalone option \fIusebackuproot\fP
452 .IP \(bu 2
453 \fInologreplay\fP (since 5.9)
454 .sp
455 Do not replay any dirty logs.
456 Replaces standalone option \fInologreplay\fP
457 .IP \(bu 2
458 \fIignorebadroots\fP, \fIibadroots\fP (since: 5.11)
459 .sp
460 Ignore bad tree roots, greatly improve the chance for data salvage.
461 .IP \(bu 2
462 \fIignoredatacsums\fP, \fIidatacsums\fP (since: 5.11)
463 .sp
464 Ignore data checksum verification.
465 .IP \(bu 2
466 \fIignoremetacsums\fP, \fIimetacsums\fP (since 6.12)
467 .sp
468 Ignore metadata checksum verification, useful for interrupted checksum conversion.
469 .IP \(bu 2
470 \fIall\fP (since: 5.9)
471 .sp
472 Enable all supported rescue options.
473 .UNINDENT
474 .TP
475 .B skip_balance
476 (since: 3.3, default: off)
477 .sp
478 Skip automatic resume of an interrupted balance operation. The operation can
479 later be resumed with \fBbtrfs balance resume\fP, or the paused state can be
480 removed with \fBbtrfs balance cancel\fP\&. The default behaviour is to resume an
481 interrupted balance immediately after the filesystem is mounted.
482 .TP
483 .B space_cache, space_cache=<version>, nospace_cache
484 (\fInospace_cache\fP since: 3.2, \fIspace_cache=v1\fP and \fIspace_cache=v2\fP since 4.5, default: \fIspace_cache=v2\fP)
485 .sp
486 Options to control the free space cache. The free space cache greatly improves
487 performance when reading block group free space into memory. However, managing
488 the space cache consumes some resources, including a small amount of disk
489 space.
490 .sp
491 There are two implementations of the free space cache. The original
492 one, referred to as \fIv1\fP, used to be a safe default but has been
493 superseded by \fIv2\fP\&. The \fIv1\fP space cache can be disabled at mount time
494 with \fInospace_cache\fP without clearing.
495 .sp
496 On very large filesystems (many terabytes) and certain workloads, the
497 performance of the \fIv1\fP space cache may degrade drastically. The \fIv2\fP
498 implementation, which adds a new b\-tree called the free space tree, addresses
499 this issue. Once enabled, the \fIv2\fP space cache will always be used and cannot
500 be disabled unless it is cleared. Use \fIclear_cache,space_cache=v1\fP or
501 \fIclear_cache,nospace_cache\fP to do so. If \fIv2\fP is enabled, and \fIv1\fP space
502 cache will be cleared (at the first mount) and kernels without \fIv2\fP
503 support will only be able to mount the filesystem in read\-only mode.
504 On an unmounted filesystem the caches (both versions) can be cleared by
505 \(dqbtrfs check \-\-clear\-space\-cache\(dq.
506 .sp
507 The btrfs\-check(8) \%<> and \fI:doc:\(gamkfs.btrfs\fP commands have full \fIv2\fP free space
508 cache support since v4.19.
509 .sp
510 If a version is not explicitly specified, the default implementation will be
511 chosen, which is \fIv2\fP\&.
512 .TP
513 .B ssd, ssd_spread, nossd, nossd_spread
514 (default: SSD autodetected)
515 .sp
516 Options to control SSD allocation schemes. By default, BTRFS will
517 enable or disable SSD optimizations depending on status of a device with
518 respect to rotational or non\-rotational type. This is determined by the
519 contents of \fB/sys/block/DEV/queue/rotational\fP). If it is 0, the
520 \fIssd\fP option is turned on. The option \fInossd\fP will disable the
521 autodetection.
522 .sp
523 The optimizations make use of the absence of the seek penalty that\(aqs inherent
524 for the rotational devices. The blocks can be typically written faster and
525 are not offloaded to separate threads.
526 .sp
527 \fBNote:\fP
528 .INDENT 7.0
529 .INDENT 3.5
530 Since 4.14, the block layout optimizations have been dropped. This used
531 to help with first generations of SSD devices. Their FTL (flash translation
532 layer) was not effective and the optimization was supposed to improve the wear
533 by better aligning blocks. This is no longer true with modern SSD devices and
534 the optimization had no real benefit. Furthermore it caused increased
535 fragmentation. The layout tuning has been kept intact for the option
536 \fIssd_spread\fP\&.
537 .UNINDENT
538 .UNINDENT
539 .sp
540 The \fIssd_spread\fP mount option attempts to allocate into bigger and aligned
541 chunks of unused space, and may perform better on low\-end SSDs. \fIssd_spread\fP
542 implies \fIssd\fP, enabling all other SSD heuristics as well. The option \fInossd\fP
543 will disable all SSD options while \fInossd_spread\fP only disables \fIssd_spread\fP\&.
544 .TP
545 .B subvol=<path>
546 Mount subvolume from \fIpath\fP rather than the toplevel subvolume. The
547 \fIpath\fP is always treated as relative to the toplevel subvolume.
548 This mount option overrides the default subvolume set for the given filesystem.
549 .TP
550 .B subvolid=<subvolid>
551 Mount subvolume specified by a \fIsubvolid\fP number rather than the toplevel
552 subvolume. You can use \fBbtrfs subvolume list\fP of \fBbtrfs subvolume show\fP to see
553 subvolume ID numbers.
554 This mount option overrides the default subvolume set for the given filesystem.
555 .sp
556 \fBNote:\fP
557 .INDENT 7.0
558 .INDENT 3.5
559 If both \fIsubvolid\fP and \fIsubvol\fP are specified, they must point at the
560 same subvolume, otherwise the mount will fail.
561 .UNINDENT
562 .UNINDENT
563 .TP
564 .B thread_pool=<number>
565 (default: min(NRCPUS + 2, 8) )
566 .sp
567 The number of worker threads to start. NRCPUS is number of on\-line CPUs
568 detected at the time of mount. Small number leads to less parallelism in
569 processing data and metadata, higher numbers could lead to a performance hit
570 due to increased locking contention, process scheduling, cache\-line bouncing or
571 costly data transfers between local CPU memories.
572 .TP
573 .B treelog, notreelog
574 (default: on)
575 .sp
576 Enable the tree logging used for \fIfsync\fP and \fIO_SYNC\fP writes. The tree log
577 stores changes without the need of a full filesystem sync. The log operations
578 are flushed at sync and transaction commit. If the system crashes between two
579 such syncs, the pending tree log operations are replayed during mount.
580 .sp
581 \fBWarning:\fP
582 .INDENT 7.0
583 .INDENT 3.5
584 Currently, the tree log is replayed even with a read\-only mount! To
585 disable that behaviour, also mount with \fInologreplay\fP\&.
586 .UNINDENT
587 .UNINDENT
588 .sp
589 The tree log could contain new files/directories, these would not exist on
590 a mounted filesystem if the log is not replayed.
591 .TP
592 .B usebackuproot
593 (since: 4.6, default: off)
594 .sp
595 Enable autorecovery attempts if a bad tree root is found at mount time.
596 Currently this scans a backup list of several previous tree roots and tries to
597 use the first readable. This can be used with read\-only mounts as well.
598 .sp
599 \fBNote:\fP
600 .INDENT 7.0
601 .INDENT 3.5
602 This option has replaced \fIrecovery\fP which has been deprecated.
603 .UNINDENT
604 .UNINDENT
605 .TP
606 .B user_subvol_rm_allowed
607 (default: off)
608 .sp
609 Allow subvolumes to be deleted by their respective owner. Otherwise, only the
610 root user can do that.
611 .sp
612 \fBNote:\fP
613 .INDENT 7.0
614 .INDENT 3.5
615 Historically, any user could create a snapshot even if he was not owner
616 of the source subvolume, the subvolume deletion has been restricted for that
617 reason. The subvolume creation has been restricted but this mount option is
618 still required. This is a usability issue.
619 Since 4.18, the rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an
620 ordinary directory. Whether this is possible can be detected at runtime, see
621 \fIrmdir_subvol\fP feature in \fIFILESYSTEM FEATURES\fP\&.
622 .UNINDENT
623 .UNINDENT
624 .UNINDENT
625 .SS DEPRECATED MOUNT OPTIONS
626 .sp
627 List of mount options that have been removed, kept for backward compatibility.
628 .INDENT 0.0
629 .TP
630 .B recovery
631 (since: 3.2, default: off, deprecated since: 4.5)
632 .sp
633 \fBNote:\fP
634 .INDENT 7.0
635 .INDENT 3.5
636 This option has been replaced by \fIusebackuproot\fP and should not be used
637 but will work on 4.5+ kernels.
638 .UNINDENT
639 .UNINDENT
640 .TP
641 .B inode_cache, noinode_cache
642 (removed in: 5.11, since: 3.0, default: off)
643 .sp
644 \fBNote:\fP
645 .INDENT 7.0
646 .INDENT 3.5
647 The functionality has been removed in 5.11, any stale data created by
648 previous use of the \fIinode_cache\fP option can be removed by
649 btrfs rescue clear\-ino\-cache \%<#\:man-rescue-clear-ino-cache>\&.
650 .UNINDENT
651 .UNINDENT
652 .TP
653 .B check_int, check_int_data, check_int_print_mask=<value>
654 (removed in: 6.7, since: 3.0, default: off)
655 .sp
656 These debugging options control the behavior of the integrity checking
657 module (the BTRFS_FS_CHECK_INTEGRITY config option required). The main goal is
658 to verify that all blocks from a given transaction period are properly linked.
659 .sp
660 \fIcheck_int\fP enables the integrity checker module, which examines all
661 block write requests to ensure on\-disk consistency, at a large
662 memory and CPU cost.
663 .sp
664 \fIcheck_int_data\fP includes extent data in the integrity checks, and
665 implies the \fIcheck_int\fP option.
666 .sp
667 \fIcheck_int_print_mask\fP takes a bit mask of BTRFSIC_PRINT_MASK_* values
668 as defined in \fIfs/btrfs/check\-integrity.c\fP, to control the integrity
669 checker module behavior.
670 .sp
671 See comments at the top of \fIfs/btrfs/check\-integrity.c\fP
672 for more information.
673 .UNINDENT
674 .SS NOTES ON GENERIC MOUNT OPTIONS
675 .sp
676 Some of the general mount options from mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> that affect BTRFS and are
677 worth mentioning.
678 .INDENT 0.0
679 .TP
680 .B context
681 The context refers to the SELinux contexts and policy definitions passed
682 as mount options. This works properly since version v6.8 (because the
683 mount option parser of BTRFS was ported to new API that also understood
684 the options).
685 .TP
686 .B noatime
687 under read intensive work\-loads, specifying \fInoatime\fP significantly improves
688 performance because no new access time information needs to be written. Without
689 this option, the default is \fIrelatime\fP, which only reduces the number of
690 inode atime updates in comparison to the traditional \fIstrictatime\fP\&. The worst
691 case for atime updates under \fIrelatime\fP occurs when many files are read whose
692 atime is older than 24 h and which are freshly snapshotted. In that case the
693 atime is updated and COW happens \- for each file \- in bulk. See also
694 \%<https://\:lwn\:.net/\:Articles/\:499293/> \- \fIAtime and btrfs: a bad combination? (LWN, 2012\-05\-31)\fP\&.
695 .sp
696 Note that \fInoatime\fP may break applications that rely on atime uptimes like
697 the venerable Mutt (unless you use \fImaildir\fP mailboxes).
698 .UNINDENT
699 .SH FILESYSTEM FEATURES
700 .sp
701 The basic set of filesystem features gets extended over time. The backward
702 compatibility is maintained and the features are optional, need to be
703 explicitly asked for so accidental use will not create incompatibilities.
704 .sp
705 There are several classes and the respective tools to manage the features:
706 .INDENT 0.0
707 .TP
708 .B at mkfs time only
709 This is namely for core structures, like the b\-tree nodesize or checksum
710 algorithm, see mkfs.btrfs(8) \%<> for more details.
711 .TP
712 .B after mkfs, on an unmounted filesystem
713 Features that may optimize internal structures or add new structures to support
714 new functionality, see btrfstune(8) \%<>\&. The command
715 \fBbtrfs inspect\-internal dump\-super /dev/sdx\fP
716 will dump a superblock, you can map the value of
717 \fIincompat_flags\fP to the features listed below
718 .TP
719 .B after mkfs, on a mounted filesystem
720 The features of a filesystem (with a given UUID) are listed in
721 \fB/sys/fs/btrfs/UUID/features/\fP, one file per feature. The status is stored
722 inside the file. The value \fI1\fP is for enabled and active, while \fI0\fP means the
723 feature was enabled at mount time but turned off afterwards.
724 .sp
725 Whether a particular feature can be turned on a mounted filesystem can be found
726 in the directory \fB/sys/fs/btrfs/features/\fP, one file per feature. The value \fI1\fP
727 means the feature can be enabled.
728 .UNINDENT
729 .sp
730 List of features (see also mkfs.btrfs(8) \%<> section
731 FILESYSTEM FEATURES \%<#\:man-mkfs-filesystem-features>):
732 .INDENT 0.0
733 .TP
734 .B big_metadata
735 (since: 3.4)
736 .sp
737 the filesystem uses \fInodesize\fP for metadata blocks, this can be bigger than the
738 page size
739 .TP
740 .B block_group_tree
741 (since: 6.1)
742 .sp
743 block group item representation using a dedicated b\-tree, this can greatly
744 reduce mount time for large filesystems
745 .TP
746 .B compress_lzo
747 (since: 2.6.38)
748 .sp
749 the \fIlzo\fP compression has been used on the filesystem, either as a mount option
750 or via \fBbtrfs filesystem defrag\fP\&.
751 .TP
752 .B compress_zstd
753 (since: 4.14)
754 .sp
755 the \fIzstd\fP compression has been used on the filesystem, either as a mount option
756 or via \fBbtrfs filesystem defrag\fP\&.
757 .TP
758 .B default_subvol
759 (since: 2.6.34)
760 .sp
761 the default subvolume has been set on the filesystem
762 .TP
763 .B extended_iref
764 (since: 3.7)
765 .sp
766 increased hardlink limit per file in a directory to 65536, older kernels
767 supported a varying number of hardlinks depending on the sum of all file name
768 sizes that can be stored into one metadata block
769 .TP
770 .B free_space_tree
771 (since: 4.5)
772 .sp
773 free space representation using a dedicated b\-tree, successor of v1 space cache
774 .TP
775 .B metadata_uuid
776 (since: 5.0)
777 .sp
778 the main filesystem UUID is the metadata_uuid, which stores the new UUID only
779 in the superblock while all metadata blocks still have the UUID set at mkfs
780 time, see btrfstune(8) \%<> for more
781 .TP
782 .B mixed_backref
783 (since: 2.6.31)
784 .sp
785 the last major disk format change, improved backreferences, now default
786 .TP
787 .B mixed_groups
788 (since: 2.6.37)
789 .sp
790 mixed data and metadata block groups, i.e. the data and metadata are not
791 separated and occupy the same block groups, this mode is suitable for small
792 volumes as there are no constraints how the remaining space should be used
793 (compared to the split mode, where empty metadata space cannot be used for data
794 and vice versa)
795 .sp
796 on the other hand, the final layout is quite unpredictable and possibly highly
797 fragmented, which means worse performance
798 .TP
799 .B no_holes
800 (since: 3.14)
801 .sp
802 improved representation of file extents where holes are not explicitly
803 stored as an extent, saves a few percent of metadata if sparse files are used
804 .TP
805 .B raid1c34
806 (since: 5.5)
807 .sp
808 extended RAID1 mode with copies on 3 or 4 devices respectively
809 .TP
810 .B raid_stripe_tree
811 (since: 6.7)
812 .sp
813 a separate tree for tracking file extents on RAID profiles
814 .TP
815 .B RAID56
816 (since: 3.9)
817 .sp
818 the filesystem contains or contained a RAID56 profile of block groups
819 .TP
820 .B rmdir_subvol
821 (since: 4.18)
822 .sp
823 indicate that rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an
824 ordinary directory. Note that this feature only depends on the kernel version.
825 .TP
826 .B skinny_metadata
827 (since: 3.10)
828 .sp
829 reduced\-size metadata for extent references, saves a few percent of metadata
830 .TP
831 .B send_stream_version
832 (since: 5.10)
833 .sp
834 number of the highest supported send stream version
835 .TP
836 .B simple_quota
837 (since: 6.7)
838 .sp
839 simplified quota accounting
840 .TP
841 .B supported_checksums
842 (since: 5.5)
843 .sp
844 list of checksum algorithms supported by the kernel module, the respective
845 modules or built\-in implementing the algorithms need to be present to mount
846 the filesystem, see section CHECKSUM ALGORITHMS\&.
847 .TP
848 .B supported_sectorsizes
849 (since: 5.13)
850 .sp
851 list of values that are accepted as sector sizes (\fBmkfs.btrfs \-\-sectorsize\fP) by
852 the running kernel
853 .TP
854 .B supported_rescue_options
855 (since: 5.11)
856 .sp
857 list of values for the mount option \fIrescue\fP that are supported by the running
858 kernel, see btrfs(5) \%<>
859 .TP
860 .B zoned
861 (since: 5.12)
862 .sp
863 zoned mode is allocation/write friendly to host\-managed zoned devices,
864 allocation space is partitioned into fixed\-size zones that must be updated
865 sequentially, see section ZONED MODE
866 .UNINDENT
867 .SH SWAPFILE SUPPORT
868 .sp
869 A swapfile, when active, is a file\-backed swap area. It is supported since kernel 5.0.
870 Use swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html> to activate it, until then (respectively again after deactivating it
871 with swapoff(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapoff\:.8\:.html>) it\(aqs just a normal file (with NODATACOW set), for which the special
872 restrictions for active swapfiles don\(aqt apply.
873 .sp
874 There are some limitations of the implementation in BTRFS and Linux swap
875 subsystem:
876 .INDENT 0.0
877 .IP \(bu 2
878 filesystem \- must be only single device
879 .IP \(bu 2
880 filesystem \- must have only \fIsingle\fP data profile
881 .IP \(bu 2
882 subvolume \- cannot be snapshotted if it contains any active swapfiles
883 .IP \(bu 2
884 swapfile \- must be preallocated (i.e. no holes)
885 .IP \(bu 2
886 swapfile \- must be NODATACOW (i.e. also NODATASUM, no compression)
887 .UNINDENT
888 .sp
889 The limitations come namely from the COW\-based design and mapping layer of
890 blocks that allows the advanced features like relocation and multi\-device
891 filesystems. However, the swap subsystem expects simpler mapping and no
892 background changes of the file block location once they\(aqve been assigned to
893 swap. The constraints mentioned above (single device and single profile) are
894 related to the swapfile itself, i.e. the extents and their placement. It is
895 possible to create swapfile on multi\-device filesystem as long as the extents
896 are on one device but this cannot be affected by user and depends on free space
897 fragmentation and available unused space for new chunks.
898 .sp
899 With active swapfiles, the following whole\-filesystem operations will skip
900 some block groups or may fail:
901 .INDENT 0.0
902 .IP \(bu 2
903 balance \- block groups with extents of any active swapfiles are skipped and
904 reported through dmesg, the rest will be processed normally
905 .IP \(bu 2
906 scrub \- ditto
907 .IP \(bu 2
908 resize grow \- unaffected
909 .IP \(bu 2
910 resize shrink \- works as long as the extents of any active swapfiles are
911 outside of the shrunk range
912 .IP \(bu 2
913 device add \- if the new devices do not interfere with any already active swapfiles
914 this operation will work, though no new swapfile can be activated
915 afterwards
916 .IP \(bu 2
917 device delete \- if the device has been added as above, it can be also deleted
918 .IP \(bu 2
919 device replace \- ditto
920 .UNINDENT
921 .sp
922 The above limitations will significantly affect btrfs maintenance routines,
923 namely balance and scrub. Balance and scrub will skip those block groups which
924 have extents of an active swapfile on them, and those block groups may also
925 contain extents belonging to other files. This means swapfiles will have an
926 amplified impact, much larger than the size of the swapfile itself.
927 .sp
928 For balance, it means data block groups may not be properly balanced, causing
929 unbalanced data/metadata usage, which can lead to early ENOSPC errors.
930 .sp
931 For scrub, it means data block groups may not be verified, degrading the
932 integrity of data.
933 .sp
934 Users should be aware of the impact when using swapfiles alongside other data.
935 This is especially discouraged for root filesystem.
936 .sp
937 When there are no active swapfiles and a whole\-filesystem exclusive operation
938 is running (e.g. balance, device delete, shrink), the swapfiles cannot be
939 temporarily activated. The operation must finish first.
940 .sp
941 To create and activate a swapfile run the following commands:
942 .INDENT 0.0
943 .INDENT 3.5
944 .sp
945 .EX
946 # truncate \-s 0 swapfile
947 # chattr +C swapfile
948 # fallocate \-l 2G swapfile
949 # chmod 0600 swapfile
950 # mkswap swapfile
951 # swapon swapfile
952 .EE
953 .UNINDENT
954 .UNINDENT
955 .sp
956 Since version 6.1 it\(aqs possible to create the swapfile in a single command
957 (except the activation):
958 .INDENT 0.0
959 .INDENT 3.5
960 .sp
961 .EX
962 # btrfs filesystem mkswapfile \-\-size 2G swapfile
963 # swapon swapfile
964 .EE
965 .UNINDENT
966 .UNINDENT
967 .sp
968 Please note that the UUID returned by the \fImkswap\fP utility identifies the swap
969 \(dqfilesystem\(dq and because it\(aqs stored in a file, it\(aqs not generally visible and
970 usable as an identifier unlike if it was on a block device.
971 .sp
972 Once activated the file will appear in \fB/proc/swaps\fP:
973 .INDENT 0.0
974 .INDENT 3.5
975 .sp
976 .EX
977 # cat /proc/swaps
978 Filename Type Size Used Priority
979 /path/swapfile file 2097152 0 \-2
980 .EE
981 .UNINDENT
982 .UNINDENT
983 .sp
984 The swapfile can be created as one\-time operation or, once properly created,
985 activated on each boot by the \fBswapon \-a\fP command (usually started by the
986 service manager). Add the following entry to \fI/etc/fstab\fP, assuming the
987 filesystem that provides the \fI/path\fP has been already mounted at this point.
988 Additional mount options relevant for the swapfile can be set too (like
989 priority, not the BTRFS mount options).
990 .INDENT 0.0
991 .INDENT 3.5
992 .sp
993 .EX
994 /path/swapfile none swap defaults 0 0
995 .EE
996 .UNINDENT
997 .UNINDENT
998 .sp
999 From now on the subvolume with the active swapfile cannot be snapshotted until
1000 the swapfile is deactivated again by \fBswapoff\fP\&. Then the swapfile is a
1001 regular file and the subvolume can be snapshotted again, though this would prevent
1002 another activation any swapfile that has been snapshotted. New swapfiles (not
1003 snapshotted) can be created and activated.
1004 .sp
1005 Otherwise, an inactive swapfile does not affect the containing subvolume. Activation
1006 creates a temporary in\-memory status and prevents some file operations, but is
1007 not stored permanently.
1008 .SH HIBERNATION
1009 .sp
1010 A swapfile can be used for hibernation but it\(aqs not straightforward. Before
1011 hibernation a resume offset must be written to file \fI/sys/power/resume_offset\fP
1012 or the kernel command line parameter \fIresume_offset\fP must be set.
1013 .sp
1014 The value is the physical offset on the device. Note that \fBthis is not the same
1015 value that\fP \fBfilefrag\fP \fBprints as physical offset!\fP
1016 .sp
1017 Btrfs filesystem uses mapping between logical and physical addresses but here
1018 the physical can still map to one or more device\-specific physical block
1019 addresses. It\(aqs the device\-specific physical offset that is suitable as resume
1020 offset.
1021 .sp
1022 Since version 6.1 there\(aqs a command btrfs inspect\-internal map\-swapfile \%<#\:man-inspect-map-swapfile>
1023 that will print the device physical offset and the adjusted value for
1024 \fB/sys/power/resume_offset\fP\&. Note that the value is divided by page size, i.e.
1025 it\(aqs not the offset itself.
1026 .INDENT 0.0
1027 .INDENT 3.5
1028 .sp
1029 .EX
1030 # btrfs filesystem mkswapfile swapfile
1031 # btrfs inspect\-internal map\-swapfile swapfile
1032 Physical start: 811511726080
1033 Resume offset: 198122980
1034 .EE
1035 .UNINDENT
1036 .UNINDENT
1037 .sp
1038 For scripting and convenience the option \fI\-r\fP will print just the offset:
1039 .INDENT 0.0
1040 .INDENT 3.5
1041 .sp
1042 .EX
1043 # btrfs inspect\-internal map\-swapfile \-r swapfile
1044 198122980
1045 .EE
1046 .UNINDENT
1047 .UNINDENT
1048 .sp
1049 The command \fBmap\-swapfile\fP also verifies all the requirements, i.e. no holes,
1050 single device, etc.
1051 .SH TROUBLESHOOTING
1052 .sp
1053 If the swapfile activation fails please verify that you followed all the steps
1054 above or check the system log (e.g. \fBdmesg\fP or \fBjournalctl\fP) for more
1055 information.
1056 .sp
1057 Notably, the \fBswapon\fP utility exits with a message that does not say what
1058 failed:
1059 .INDENT 0.0
1060 .INDENT 3.5
1061 .sp
1062 .EX
1063 # swapon /path/swapfile
1064 swapon: /path/swapfile: swapon failed: Invalid argument
1065 .EE
1066 .UNINDENT
1067 .UNINDENT
1068 .sp
1069 The specific reason is likely to be printed to the system log by the btrfs
1070 module:
1071 .INDENT 0.0
1072 .INDENT 3.5
1073 .sp
1074 .EX
1075 # journalctl \-t kernel | grep swapfile
1076 kernel: BTRFS warning (device sda): swapfile must have single data profile
1077 .EE
1078 .UNINDENT
1079 .UNINDENT
1080 .SH CHECKSUM ALGORITHMS
1081 .sp
1082 Data and metadata are checksummed by default. The checksum is calculated before
1083 writing and verified after reading the blocks from devices. The whole metadata
1084 block has an inline checksum stored in the b\-tree node header. Each data block
1085 has a detached checksum stored in the checksum tree.
1086 .sp
1087 \fBNote:\fP
1088 .INDENT 0.0
1089 .INDENT 3.5
1090 Since a data checksum is calculated just before submitting to the block
1091 device, btrfs has a strong requirement that the corresponding data block must
1092 not be modified until the writeback is finished.
1093 .sp
1094 This requirement is met for a buffered write as btrfs has the full control on
1095 its page cache, but a direct write (\fBO_DIRECT\fP) bypasses page cache, and
1096 btrfs can not control the direct IO buffer (as it can be in user space memory).
1097 Thus it\(aqs possible that a user space program modifies its direct write buffer
1098 before the buffer is fully written back, and this can lead to a data
1099 checksum mismatch.
1100 .sp
1101 To avoid this, kernel starting with version 6.14 will force a direct
1102 write to fall back to buffered, if the inode requires a data checksum.
1103 This will bring a small performance penalty. If you require true zero\-copy
1104 direct writes, then set the \fBNODATASUM\fP flag for the inode and make
1105 sure the direct IO buffer is fully aligned to block size.
1106 .UNINDENT
1107 .UNINDENT
1108 .sp
1109 There are several checksum algorithms supported. The default and backward
1110 compatible algorithm is \fIcrc32c\fP\&. Since kernel 5.5 there are three more with different
1111 characteristics and trade\-offs regarding speed and strength. The following list
1112 may help you to decide which one to select.
1113 .INDENT 0.0
1114 .TP
1115 .B CRC32C (32 bits digest)
1116 Default, best backward compatibility. Very fast, modern CPUs have
1117 instruction\-level support, not collision\-resistant but still good error
1118 detection capabilities.
1119 .TP
1120 .B XXHASH (64 bits digest)
1121 Can be used as CRC32C successor. Very fast, optimized for modern CPUs utilizing
1122 instruction pipelining, good collision resistance and error detection.
1123 .TP
1124 .B SHA256 (256 bits digest)
1125 Cryptographic\-strength hash. Relatively slow but with possible CPU
1126 instruction acceleration or specialized hardware cards. FIPS certified and
1127 in wide use.
1128 .TP
1129 .B BLAKE2b (256 bits digest)
1130 Cryptographic\-strength hash. Relatively fast, with possible CPU acceleration
1131 using SIMD extensions. Not standardized but based on BLAKE which was a SHA3
1132 finalist, in wide use. The algorithm used is BLAKE2b\-256 that\(aqs optimized for
1133 64\-bit platforms.
1134 .UNINDENT
1135 .sp
1136 The \fIdigest size\fP affects overall size of data block checksums stored in the
1137 filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so
1138 there\(aqs no increase. Each data block has a separate checksum stored, with
1139 additional overhead of the b\-tree leaves.
1140 .sp
1141 Approximate relative performance of the algorithms, measured against CRC32C
1142 using implementations on a 11th gen 3.6GHz intel CPU:
1143 .TS
1144 box center;
1145 l|l|l|l.
1146 T{
1147 Digest
1148 T} T{
1149 Cycles/4KiB
1150 T} T{
1151 Ratio
1152 T} T{
1153 Implementation
1154 T}
1155 _
1156 T{
1157 CRC32C
1158 T} T{
1159 470
1160 T} T{
1161 1.00
1162 T} T{
1163 CPU instruction, PCL combination
1164 T}
1165 _
1166 T{
1167 XXHASH
1168 T} T{
1169 870
1170 T} T{
1171 1.9
1172 T} T{
1173 reference impl.
1174 T}
1175 _
1176 T{
1177 SHA256
1178 T} T{
1179 7600
1180 T} T{
1181 16
1182 T} T{
1183 libgcrypt
1184 T}
1185 _
1186 T{
1187 SHA256
1188 T} T{
1189 8500
1190 T} T{
1191 18
1192 T} T{
1193 openssl
1194 T}
1195 _
1196 T{
1197 SHA256
1198 T} T{
1199 8700
1200 T} T{
1201 18
1202 T} T{
1203 botan
1204 T}
1205 _
1206 T{
1207 SHA256
1208 T} T{
1209 32000
1210 T} T{
1211 68
1212 T} T{
1213 builtin, CPU instruction
1214 T}
1215 _
1216 T{
1217 SHA256
1218 T} T{
1219 37000
1220 T} T{
1221 78
1222 T} T{
1223 libsodium
1224 T}
1225 _
1226 T{
1227 SHA256
1228 T} T{
1229 78000
1230 T} T{
1231 166
1232 T} T{
1233 builtin, reference impl.
1234 T}
1235 _
1236 T{
1237 BLAKE2b
1238 T} T{
1239 10000
1240 T} T{
1241 21
1242 T} T{
1243 builtin/AVX2
1244 T}
1245 _
1246 T{
1247 BLAKE2b
1248 T} T{
1249 10900
1250 T} T{
1251 23
1252 T} T{
1253 libgcrypt
1254 T}
1255 _
1256 T{
1257 BLAKE2b
1258 T} T{
1259 13500
1260 T} T{
1261 29
1262 T} T{
1263 builtin/SSE41
1264 T}
1265 _
1266 T{
1267 BLAKE2b
1268 T} T{
1269 13700
1270 T} T{
1271 29
1272 T} T{
1273 libsodium
1274 T}
1275 _
1276 T{
1277 BLAKE2b
1278 T} T{
1279 14100
1280 T} T{
1281 30
1282 T} T{
1283 openssl
1284 T}
1285 _
1286 T{
1287 BLAKE2b
1288 T} T{
1289 14500
1290 T} T{
1291 31
1292 T} T{
1293 kcapi
1294 T}
1295 _
1296 T{
1297 BLAKE2b
1298 T} T{
1299 14500
1300 T} T{
1301 34
1302 T} T{
1303 builtin, reference impl.
1304 T}
1305 .TE
1306 .sp
1307 Many kernels are configured with SHA256 as built\-in and not as a module.
1308 Up to kernel v6.15 the accelerated versions are however provided by the
1309 modules and must be loaded
1310 explicitly (\fBmodprobe sha256\fP) before mounting the filesystem to make use of
1311 them. You can check in \fB/sys/fs/btrfs/FSID/checksum\fP which one is used. If you
1312 see \fIsha256\-generic\fP, then you may want to unmount and mount the filesystem
1313 again. Changing that on a mounted filesystem is not possible.
1314 .sp
1315 Since kernel v6.16 the accelerated implementation is always used if available.
1316 .sp
1317 Check the file \fB/proc/crypto\fP, when the implementation is built\-in, you\(aqd find:
1318 .INDENT 0.0
1319 .INDENT 3.5
1320 .sp
1321 .EX
1322 name : sha256
1323 driver : sha256\-generic
1324 module : kernel
1325 priority : 100
1326 \&...
1327 .EE
1328 .UNINDENT
1329 .UNINDENT
1330 .sp
1331 While accelerated implementation is e.g.:
1332 .INDENT 0.0
1333 .INDENT 3.5
1334 .sp
1335 .EX
1336 name : sha256
1337 driver : sha256\-avx2
1338 module : sha256_ssse3
1339 priority : 170
1340 \&...
1341 .EE
1342 .UNINDENT
1343 .UNINDENT
1344 .SH COMPRESSION
1345 .sp
1346 Btrfs supports transparent file compression. There are three algorithms
1347 available: ZLIB, LZO and ZSTD (since v4.14), with various levels.
1348 The compression happens on the level of file extents and the algorithm is
1349 selected by file property, mount option or by a defrag command.
1350 You can have a single btrfs mount point that has some files that are
1351 uncompressed, some that are compressed with LZO, some with ZLIB, for instance
1352 (though you may not want it that way, it is supported).
1353 .sp
1354 Once the compression is set, all newly written data will be compressed, i.e.
1355 existing data are untouched. Data are split into smaller chunks (128KiB) before
1356 compression to make random rewrites possible without a high performance hit. Due
1357 to the increased number of extents the metadata consumption is higher. The
1358 chunks are compressed in parallel.
1359 .sp
1360 The algorithms can be characterized as follows regarding the speed/ratio
1361 trade\-offs:
1362 .INDENT 0.0
1363 .TP
1364 .B ZLIB
1365 .INDENT 7.0
1366 .IP \(bu 2
1367 slower, higher compression ratio
1368 .IP \(bu 2
1369 levels: 1 to 9, mapped directly, default level is 3
1370 .IP \(bu 2
1371 good backward compatibility
1372 .UNINDENT
1373 .TP
1374 .B LZO
1375 .INDENT 7.0
1376 .IP \(bu 2
1377 faster compression and decompression than ZLIB, worse compression ratio, designed to be fast
1378 .IP \(bu 2
1379 no levels
1380 .IP \(bu 2
1381 good backward compatibility
1382 .UNINDENT
1383 .TP
1384 .B ZSTD
1385 .INDENT 7.0
1386 .IP \(bu 2
1387 compression comparable to ZLIB with higher compression/decompression speeds and different ratio
1388 .IP \(bu 2
1389 levels: \-15..15, mapped directly, default is 3
1390 .IP \(bu 2
1391 support since 4.14
1392 .IP \(bu 2
1393 levels 1..15 supported since 5.1
1394 .IP \(bu 2
1395 levels \-15..\-1 supported since 6.15
1396 .UNINDENT
1397 .UNINDENT
1398 .sp
1399 The differences depend on the actual data set and cannot be expressed by a
1400 single number or recommendation. Higher levels consume more CPU time and may
1401 not bring a significant improvement, lower levels are close to real time.
1402 .SH HOW TO ENABLE COMPRESSION
1403 .sp
1404 Typically the compression can be enabled on the whole filesystem, specified for
1405 the mount point. Note that the compression mount options are shared among all
1406 mounts of the same filesystem, either bind mounts or subvolume mounts.
1407 Please refer to btrfs(5) \%<> section
1408 MOUNT OPTIONS\&.
1409 .INDENT 0.0
1410 .INDENT 3.5
1411 .sp
1412 .EX
1413 $ mount \-o compress=zstd /dev/sdx /mnt
1414 .EE
1415 .UNINDENT
1416 .UNINDENT
1417 .sp
1418 This will enable the \fBzstd\fP algorithm on the default level (which is 3).
1419 The level can be specified manually too like \fBzstd:3\fP\&. Higher levels compress
1420 better at the cost of time. This in turn may cause increased write latency, low
1421 levels are suitable for real\-time compression and on reasonably fast CPU don\(aqt
1422 cause noticeable performance drops.
1423 .INDENT 0.0
1424 .INDENT 3.5
1425 .sp
1426 .EX
1427 $ btrfs filesystem defrag \-czstd file
1428 .EE
1429 .UNINDENT
1430 .UNINDENT
1431 .sp
1432 The command above will start defragmentation of the whole \fIfile\fP and apply
1433 the compression, regardless of the mount option. The compression level can be
1434 also specified with the \fB\-\-level\fP or \fB\-L\fP argument as of version \fB6.14\fP\&.
1435 The compression algorithm is not persistent and applies only
1436 to the defragmentation command, for any other writes other compression settings
1437 apply.
1438 .sp
1439 Persistent settings on a per\-file basis can be set in two ways:
1440 .INDENT 0.0
1441 .INDENT 3.5
1442 .sp
1443 .EX
1444 $ chattr +c file
1445 $ btrfs property set file compression zstd
1446 .EE
1447 .UNINDENT
1448 .UNINDENT
1449 .sp
1450 The first command is using legacy interface of file attributes inherited from
1451 ext2 filesystem and is not flexible, so by default the \fIzlib\fP compression is
1452 set. The other command sets a property on the file with the given algorithm.
1453 (Note: setting level that way is not yet implemented.)
1454 .SH COMPRESSION LEVELS
1455 .sp
1456 The level support of ZLIB has been added in v4.14, LZO does not support levels
1457 (the kernel implementation provides only one), ZSTD level support has been added
1458 in v5.1 and the negative levels in v6.15.
1459 .sp
1460 There are 9 levels of ZLIB supported (1 to 9), mapping 1:1 from the mount option
1461 to the algorithm defined level. The default is level 3, which provides the
1462 reasonably good compression ratio and is still reasonably fast. The difference
1463 in compression gain of levels 7, 8 and 9 is comparable but the higher levels
1464 take longer.
1465 .sp
1466 The ZSTD support includes levels \-15..15, a subset of full range of what ZSTD
1467 provides. Levels \-15..\-1 are real\-time with worse compression ratio, levels
1468 1..3 are near real\-time with good compression, 4..8 are slower with improved
1469 compression and 9..15 try even harder though the resulting size may not be
1470 significantly improved. Higher levels also require more memory and as they need
1471 more CPU the system performance is affected.
1472 .sp
1473 Level 0 always maps to the default. The compression level does not affect
1474 compatibility.
1475 .SH EXCEPTIONS
1476 .sp
1477 Any file that has been touched by the \fIfallocate\fP system call will always be
1478 excepted from compression even if \fIforce\-compress\fP mount option is used.
1479 .sp
1480 The reason for this is that a successful \fIfallocate\fP call must guarantee that
1481 future writes to the allocated range will not fail because of lack of space.
1482 This is difficult to guarantee in a COW filesystem. To reduce the chances of
1483 it happening, btrfs preallocates space and disables compression for the file.
1484 .sp
1485 As a workaround, one can trigger a compressed rewrite for such a file using the
1486 \fIbtrfs defrag\fP command. Be aware that if the file is touched again by the
1487 \fIfallocate\fP system call, it will be excepted again from compression for all the
1488 new data written to it.
1489 .SH INCOMPRESSIBLE DATA
1490 .sp
1491 Files with already compressed data or with data that won\(aqt compress well with
1492 the CPU and memory constraints of the kernel implementations are using a simple
1493 decision logic. If the first portion of data being compressed is not smaller
1494 than the original, the compression of the whole file is disabled. Unless the
1495 filesystem is mounted with \fIcompress\-force\fP in which case btrfs will try
1496 compressing every block, falling back to storing the uncompressed version for
1497 each block that ends up larger after compression. This is not optimal and
1498 subject to optimizations and further development.
1499 .sp
1500 If a file is identified as incompressible, a flag is set (\fINOCOMPRESS\fP) and it\(aqs
1501 sticky. On that file compression won\(aqt be performed unless forced. The flag
1502 can be also set by \fBchattr +m\fP (since e2fsprogs 1.46.2) or by properties with
1503 value \fIno\fP or \fInone\fP\&. Empty value will reset it to the default that\(aqs currently
1504 applicable on the mounted filesystem.
1505 .sp
1506 There are two ways to detect incompressible data:
1507 .INDENT 0.0
1508 .IP \(bu 2
1509 actual compression attempt \- data are compressed, if the result is not smaller,
1510 it\(aqs discarded, so this depends on the algorithm and level
1511 .IP \(bu 2
1512 pre\-compression heuristics \- a quick statistical evaluation on the data is
1513 performed and based on the result either compression is performed or skipped,
1514 the NOCOMPRESS bit is not set just by the heuristic, only if the compression
1515 algorithm does not make an improvement
1516 .UNINDENT
1517 .INDENT 0.0
1518 .INDENT 3.5
1519 .sp
1520 .EX
1521 $ lsattr file
1522 \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-m file
1523 .EE
1524 .UNINDENT
1525 .UNINDENT
1526 .sp
1527 Using the forcing compression is not recommended, the heuristics are
1528 supposed to decide that and compression algorithms internally detect
1529 incompressible data too.
1530 .SH PRE-COMPRESSION HEURISTICS
1531 .sp
1532 The heuristics aim to do a few quick statistical tests on the compressed data
1533 in order to avoid probably costly compression that would turn out to be
1534 inefficient. Compression algorithms could have internal detection of
1535 incompressible data too but this leads to more overhead as the compression is
1536 done in another thread and has to write the data anyway. The heuristic is
1537 read\-only and can utilize cached memory.
1538 .sp
1539 The tests performed based on the following: data sampling, long repeated
1540 pattern detection, byte frequency, Shannon entropy.
1541 .SH COMPATIBILITY
1542 .sp
1543 Compression requires both data checksums and COW, so either \fInodatasum\fP or
1544 \fInodatasum\fP mount option/inode flag will result in no compression.
1545 .sp
1546 Direct IO reads of compressed data will always fallback to buffered reads.
1547 .sp
1548 Direct IO write behavior depends on the inode flag.
1549 For inodes with data checksum, direct IO writes always fallback to buffered
1550 writes, thus can generate compressed data if the mount option/inode flags
1551 allows that.
1552 .sp
1553 For inodes without data checksums, direct IO writes will not populate page cache,
1554 and since the inode has no data checksums, no compressed data will be generated
1555 anyway.
1556 .sp
1557 The compression algorithms have been added over time so the version
1558 compatibility should be also considered, together with other tools that may
1559 access the compressed data like bootloaders.
1560 .SH SYSFS INTERFACE
1561 .sp
1562 Btrfs has a sysfs interface to provide extra knobs.
1563 .sp
1564 The top level path is \fB/sys/fs/btrfs/\fP, and the main directory layout is the following:
1565 .TS
1566 box center;
1567 l|l|l.
1568 T{
1569 Relative Path
1570 T} T{
1571 Description
1572 T} T{
1573 Version
1574 T}
1575 _
1576 T{
1577 features/
1578 T} T{
1579 All supported features
1580 T} T{
1581 3.14
1582 T}
1583 _
1584 T{
1585 <UUID>/
1586 T} T{
1587 Mounted fs UUID
1588 T} T{
1589 3.14
1590 T}
1591 _
1592 T{
1593 <UUID>/allocation/
1594 T} T{
1595 Space allocation info
1596 T} T{
1597 3.14
1598 T}
1599 _
1600 T{
1601 <UUID>/bdi/
1602 T} T{
1603 Backing device info (writeback)
1604 T} T{
1605 5.9
1606 T}
1607 _
1608 T{
1609 <UUID>/devices/<DEVID>/
1610 T} T{
1611 Symlink to each block device sysfs
1612 T} T{
1613 5.6
1614 T}
1615 _
1616 T{
1617 <UUID>/devinfo/<DEVID>/
1618 T} T{
1619 Btrfs specific info for each device
1620 T} T{
1621 5.6
1622 T}
1623 _
1624 T{
1625 <UUID>/discard/
1626 T} T{
1627 Discard stats and tunables
1628 T} T{
1629 6.1
1630 T}
1631 _
1632 T{
1633 <UUID>/features/
1634 T} T{
1635 Features of the filesystem
1636 T} T{
1637 3.14
1638 T}
1639 _
1640 T{
1641 <UUID>/qgroups/
1642 T} T{
1643 Global qgroup info
1644 T} T{
1645 5.9
1646 T}
1647 _
1648 T{
1649 <UUID>/qgroups/<LEVEL>_<ID>/
1650 T} T{
1651 Info for each qgroup
1652 T} T{
1653 5.9
1654 T}
1655 .TE
1656 .sp
1657 For \fB/sys/fs/btrfs/features/\fP directory, each file means a supported feature
1658 of the current kernel. Most files have value 0. Otherwise it depends on the file,
1659 value \fI1\fP typically means the feature can be turned on a mounted filesystem.
1660 .sp
1661 For \fB/sys/fs/btrfs/<UUID>/features/\fP directory, each file means an enabled
1662 feature on the mounted filesystem.
1663 .sp
1664 The features share the same name in section
1665 FILESYSTEM FEATURES\&.
1666 .SS UUID
1667 .sp
1668 Files in \fB/sys/fs/btrfs/<UUID>/\fP directory are:
1669 .INDENT 0.0
1670 .TP
1671 .B bg_reclaim_threshold
1672 (RW, since: 5.19)
1673 .sp
1674 Used space percentage of total device space to start auto block group claim.
1675 Mostly for zoned devices.
1676 .TP
1677 .B checksum
1678 (RO, since: 5.5)
1679 .sp
1680 The checksum used for the mounted filesystem.
1681 This includes both the checksum type (see section
1682 CHECKSUM ALGORITHMS)
1683 and the implemented driver (mostly shows if it\(aqs hardware accelerated).
1684 .TP
1685 .B clone_alignment
1686 (RO, since: 3.16)
1687 .sp
1688 The bytes alignment for \fIclone\fP and \fIdedupe\fP ioctls.
1689 .TP
1690 .B commit_stats
1691 (RW, since: 6.0)
1692 .sp
1693 The performance statistics for btrfs transaction commit since the first
1694 mount. Mostly for debugging purposes.
1695 .sp
1696 Writing into this file will reset the maximum commit duration
1697 (\fImax_commit_ms\fP) to 0. The file looks like:
1698 .INDENT 7.0
1699 .INDENT 3.5
1700 .sp
1701 .EX
1702 commits 70649
1703 last_commit_ms 2
1704 max_commit_ms 131
1705 total_commit_ms 170840
1706 .EE
1707 .UNINDENT
1708 .UNINDENT
1709 .INDENT 7.0
1710 .IP \(bu 2
1711 \fIcommits\fP \- number of transaction commits since the first mount
1712 .IP \(bu 2
1713 \fIlast_commit_ms\fP \- duration in milliseconds of the last commit
1714 .IP \(bu 2
1715 \fImax_commit_ms\fP \- maximum time a transaction commit took since first mount or last reset
1716 .IP \(bu 2
1717 \fItotal_commit_ms\fP \- sum of all transaction commit times
1718 .UNINDENT
1719 .TP
1720 .B exclusive_operation
1721 (RO, since: 5.10)
1722 .sp
1723 Shows the running exclusive operation.
1724 Check section
1725 FILESYSTEM EXCLUSIVE OPERATIONS
1726 for details.
1727 .TP
1728 .B generation
1729 (RO, since: 5.11)
1730 .sp
1731 Show the generation of the mounted filesystem.
1732 .TP
1733 .B label
1734 (RW, since: 3.14)
1735 .sp
1736 Show the current label of the mounted filesystem.
1737 .TP
1738 .B metadata_uuid
1739 (RO, since: 5.0)
1740 .sp
1741 Shows the metadata UUID of the mounted filesystem.
1742 Check \fImetadata_uuid\fP feature for more details.
1743 .TP
1744 .B nodesize
1745 (RO, since: 3.14)
1746 .sp
1747 Show the nodesize of the mounted filesystem.
1748 .TP
1749 .B quota_override
1750 (RW, since: 4.13)
1751 .sp
1752 Shows the current quota override status.
1753 0 means no quota override.
1754 1 means quota override, quota can ignore the existing limit settings.
1755 .TP
1756 .B read_policy
1757 (RW, since: 5.11)
1758 .sp
1759 Shows the current balance policy for reads.
1760 Currently only \fBpid\fP (balance using the process id (pid) value) is
1761 supported. More balancing policies are available in experimental
1762 build, namely round\-robin.
1763 .TP
1764 .B sectorsize
1765 (RO, since: 3.14)
1766 .sp
1767 Shows the sectorsize of the mounted filesystem.
1768 .TP
1769 .B temp_fsid
1770 (RO, since 6.7)
1771 .sp
1772 Indicate that this filesystem got assigned a temporary FSID at mount time,
1773 making possible to mount devices with the same FSID.
1774 .UNINDENT
1775 .SS UUID/allocations
1776 .sp
1777 Files and directories in \fB/sys/fs/btrfs/<UUID>/allocations\fP directory are:
1778 .INDENT 0.0
1779 .TP
1780 .B global_rsv_reserved
1781 (RO, since: 3.14)
1782 .sp
1783 The used bytes of the global reservation.
1784 .TP
1785 .B global_rsv_size
1786 (RO, since: 3.14)
1787 .sp
1788 The total size of the global reservation.
1789 .TP
1790 .B \fIdata/\fP, \fImetadata/\fP and \fIsystem/\fP directories
1791 (RO, since: 5.14)
1792 .sp
1793 Space info accounting for the 3 block group types.
1794 .UNINDENT
1795 .SS UUID/allocations/{data,metadata,system}
1796 .sp
1797 Files in \fB/sys/fs/btrfs/<UUID>/allocations/\fP\fIdata,metadata,system\fP directory are:
1798 .INDENT 0.0
1799 .TP
1800 .B bg_reclaim_threshold
1801 (RW, since: 5.19)
1802 .sp
1803 Reclaimable space percentage of block group\(aqs size (excluding
1804 permanently unusable space) to reclaim the block group.
1805 Can be used on regular or zoned devices.
1806 .TP
1807 .B bytes_*
1808 (RO)
1809 .sp
1810 Values of the corresponding data structures for the given block group
1811 type and profile that are used internally and may change rapidly depending
1812 on the load.
1813 .sp
1814 Complete list: bytes_may_use, bytes_pinned, bytes_readonly,
1815 bytes_reserved, bytes_used, bytes_zone_unusable
1816 .TP
1817 .B chunk_size
1818 (RW, since: 6.0)
1819 .sp
1820 Shows the chunk size. Can be changed for data and metadata (independently)
1821 and cannot be set for system block group type.
1822 Cannot be set for zoned devices as it depends on the fixed device zone size.
1823 Upper bound is 10% of the filesystem size, the value must be multiple of 256MiB
1824 and greater than 0.
1825 .TP
1826 .B size_classes
1827 (RO, since: 6.3)
1828 .sp
1829 Numbers of block groups of a given classes based on heuristics that
1830 measure extent length, age and fragmentation.
1831 .INDENT 7.0
1832 .INDENT 3.5
1833 .sp
1834 .EX
1835 none 136
1836 small 374
1837 medium 282
1838 large 93
1839 .EE
1840 .UNINDENT
1841 .UNINDENT
1842 .UNINDENT
1843 .SS UUID/bdi
1844 .sp
1845 Symlink to the sysfs directory of the backing device info (BDI), which is
1846 related to writeback process and infrastructure.
1847 .SS UUID/devices
1848 .sp
1849 Files in \fB/sys/fs/btrfs/<UUID>/devices\fP directory are symlinks named
1850 after device nodes (e.g. sda, dm\-0) and pointing to their sysfs directory.
1851 .SS UUID/devinfo
1852 .sp
1853 The directory contains subdirectories named after device ids (numeric values). Each
1854 subdirectory has information about the device of the given \fIdevid\fP\&.
1855 .SS UUID/devinfo/DEVID
1856 .sp
1857 Files in \fB/sys/fs/btrfs/<UUID>/devinfo/<DEVID>\fP directory are:
1858 .INDENT 0.0
1859 .TP
1860 .B error_stats:
1861 (RO, since: 5.14)
1862 .sp
1863 Shows device stats of this device, same as \fBbtrfs device stats\fP (btrfs\-device(8) \%<>).
1864 .INDENT 7.0
1865 .INDENT 3.5
1866 .sp
1867 .EX
1868 write_errs 0
1869 read_errs 0
1870 flush_errs 0
1871 corruption_errs 0
1872 generation_errs 0
1873 .EE
1874 .UNINDENT
1875 .UNINDENT
1876 .TP
1877 .B fsid:
1878 (RO, since: 5.17)
1879 .sp
1880 Shows the fsid which the device belongs to.
1881 It can be different than the \fBUUID\fP if it\(aqs a seed device.
1882 .TP
1883 .B in_fs_metadata
1884 (RO, since: 5.6)
1885 .sp
1886 Shows whether we have found the device.
1887 Should always be 1, as if this turns to 0, the \fBDEVID\fP directory
1888 would get removed automatically.
1889 .TP
1890 .B missing
1891 (RO, since: 5.6)
1892 .sp
1893 Shows whether the device is considered missing by the kernel module.
1894 .TP
1895 .B replace_target
1896 (RO, since: 5.6)
1897 .sp
1898 Shows whether the device is the replace target.
1899 If no device replace is running, this value is 0.
1900 .TP
1901 .B scrub_speed_max
1902 (RW, since: 5.14)
1903 .sp
1904 Shows the scrub speed limit for this device. The unit is Bytes/s.
1905 0 means no limit. The value can be set but is not persistent.
1906 .TP
1907 .B writeable
1908 (RO, since: 5.6)
1909 .sp
1910 Show if the device is writeable.
1911 .UNINDENT
1912 .SS UUID/qgroups
1913 .sp
1914 Files in \fB/sys/fs/btrfs/<UUID>/qgroups/\fP directory are:
1915 .INDENT 0.0
1916 .TP
1917 .B enabled
1918 (RO, since: 6.1)
1919 .sp
1920 Shows if qgroup is enabled.
1921 Also, if qgroup is disabled, the \fBqgroups\fP directory will
1922 be removed automatically.
1923 .TP
1924 .B inconsistent
1925 (RO, since: 6.1)
1926 .sp
1927 Shows if the qgroup numbers are inconsistent.
1928 If 1, it\(aqs recommended to do a qgroup rescan.
1929 .TP
1930 .B drop_subtree_threshold
1931 (RW, since: 6.1)
1932 .sp
1933 Shows the subtree drop threshold to automatically mark qgroup inconsistent.
1934 .sp
1935 When dropping large subvolumes with qgroup enabled, there would be a huge
1936 load for qgroup accounting.
1937 If we have a subtree whose level is larger than or equal to this value,
1938 we will not trigger qgroup account at all, but mark qgroup inconsistent to
1939 avoid the huge workload.
1940 .sp
1941 Default value is 3, which means that trees of low height will be accounted
1942 properly as this is sufficiently fast. The value was 8 until 6.13 where
1943 no subtree drop can trigger qgroup rescan making it less useful.
1944 .sp
1945 Lower value can reduce qgroup workload, at the cost of extra qgroup rescan
1946 to re\-calculate the numbers.
1947 .UNINDENT
1948 .SS UUID/qgroups/LEVEL_ID
1949 .sp
1950 Files in each \fB/sys/fs/btrfs/<UUID>/qgroups/<LEVEL>_<ID>/\fP directory are:
1951 .INDENT 0.0
1952 .TP
1953 .B exclusive
1954 (RO, since: 5.9)
1955 .sp
1956 Shows the exclusively owned bytes of the qgroup.
1957 .TP
1958 .B limit_flags
1959 (RO, since: 5.9)
1960 .sp
1961 Shows the numeric value of the limit flags.
1962 If 0, means no limit implied.
1963 .TP
1964 .B max_exclusive
1965 (RO, since: 5.9)
1966 .sp
1967 Shows the limits on exclusively owned bytes.
1968 .TP
1969 .B max_referenced
1970 (RO, since: 5.9)
1971 .sp
1972 Shows the limits on referenced bytes.
1973 .TP
1974 .B referenced
1975 (RO, since: 5.9)
1976 .sp
1977 Shows the referenced bytes of the qgroup.
1978 .TP
1979 .B rsv_data
1980 (RO, since: 5.9)
1981 .sp
1982 Shows the reserved bytes for data.
1983 .TP
1984 .B rsv_meta_pertrans
1985 (RO, since: 5.9)
1986 .sp
1987 Shows the reserved bytes for per transaction metadata.
1988 .TP
1989 .B rsv_meta_prealloc
1990 (RO, since: 5.9)
1991 .sp
1992 Shows the reserved bytes for preallocated metadata.
1993 .UNINDENT
1994 .SS UUID/discard
1995 .sp
1996 Files in \fB/sys/fs/btrfs/<UUID>/discard/\fP directory are:
1997 .INDENT 0.0
1998 .TP
1999 .B discardable_bytes
2000 (RO, since: 6.1)
2001 .sp
2002 Shows amount of bytes that can be discarded in the async discard and
2003 nodiscard mode.
2004 .TP
2005 .B discardable_extents
2006 (RO, since: 6.1)
2007 .sp
2008 Shows number of extents to be discarded in the async discard and
2009 nodiscard mode.
2010 .TP
2011 .B discard_bitmap_bytes
2012 (RO, since: 6.1)
2013 .sp
2014 Shows amount of discarded bytes from data tracked as bitmaps.
2015 .TP
2016 .B discard_extent_bytes
2017 (RO, since: 6.1)
2018 .sp
2019 Shows amount of discarded extents from data tracked as bitmaps.
2020 .TP
2021 .B discard_bytes_saved
2022 (RO, since: 6.1)
2023 .sp
2024 Shows the amount of bytes that were reallocated without being discarded.
2025 .TP
2026 .B kbps_limit
2027 (RW, since: 6.1)
2028 .sp
2029 Tunable limit of kilobytes per second issued as discard IO in the async
2030 discard mode.
2031 .TP
2032 .B iops_limit
2033 (RW, since: 6.1)
2034 .sp
2035 Tunable limit of number of discard IO operations to be issued in the
2036 async discard mode.
2037 .TP
2038 .B max_discard_size
2039 (RW, since: 6.1)
2040 .sp
2041 Tunable limit for size of one IO discard request.
2042 .UNINDENT
2043 .SH FILESYSTEM EXCLUSIVE OPERATIONS
2044 .sp
2045 There are several operations that affect the whole filesystem and cannot be run
2046 in parallel. Attempt to start one while another is running will fail (see
2047 exceptions below).
2048 .sp
2049 Since kernel 5.10 the currently running operation can be obtained from
2050 \fB/sys/fs/UUID/exclusive_operation\fP with following values and operations:
2051 .INDENT 0.0
2052 .IP \(bu 2
2053 balance
2054 .IP \(bu 2
2055 balance paused (since 5.17)
2056 .IP \(bu 2
2057 device add
2058 .IP \(bu 2
2059 device delete
2060 .IP \(bu 2
2061 device replace
2062 .IP \(bu 2
2063 resize
2064 .IP \(bu 2
2065 swapfile activate
2066 .IP \(bu 2
2067 none
2068 .UNINDENT
2069 .sp
2070 Enqueuing is supported for several btrfs subcommands so they can be started
2071 at once and then serialized.
2072 .sp
2073 There\(aqs an exception when a paused balance allows to start a device add
2074 operation as they don\(aqt really collide and this can be used to add more space
2075 for the balance to finish.
2076 .SH FILESYSTEM LIMITS
2077 .INDENT 0.0
2078 .TP
2079 .B maximum file name length
2080 255
2081 .sp
2082 This limit is imposed by Linux VFS, the structures of BTRFS could store
2083 larger file names.
2084 .TP
2085 .B maximum symlink target length
2086 depends on the \fInodesize\fP value, for 4KiB it\(aqs 3949 bytes, for larger nodesize
2087 it\(aqs 4095 due to the system limit PATH_MAX
2088 .sp
2089 The symlink target may not be a valid path, i.e. the path name components
2090 can exceed the limits (NAME_MAX), there\(aqs no content validation at symlink(3) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man3/\:symlink\:.3\:.html>
2091 creation.
2092 .TP
2093 .B maximum number of inodes
2094 2\s-2\u64\d\s0 but depends on the available metadata space as the inodes are created
2095 dynamically
2096 .sp
2097 Each subvolume is an independent namespace of inodes and thus their
2098 numbers, so the limit is per subvolume, not for the whole filesystem.
2099 .TP
2100 .B inode numbers
2101 minimum number: 256 (for subvolumes), regular files and directories: 257,
2102 maximum number: (2\s-2\u64\d\s0 \- 256)
2103 .sp
2104 The inode numbers that can be assigned to user created files are from
2105 the whole 64bit space except first 256 and last 256 in that range that
2106 are reserved for internal b\-tree identifiers.
2107 .TP
2108 .B maximum file length
2109 inherent limit of BTRFS is 2\s-2\u64\d\s0 (16 EiB) but the practical
2110 limit of Linux VFS is 2\s-2\u63\d\s0 (8 EiB)
2111 .TP
2112 .B maximum number of subvolumes
2113 the subvolume ids can go up to 2\s-2\u48\d\s0 but the number of actual subvolumes
2114 depends on the available metadata space
2115 .sp
2116 The space consumed by all subvolume metadata includes bookkeeping of
2117 shared extents can be large (MiB, GiB). The range is not the full 64bit
2118 range because of qgroups that use the upper 16 bits for another
2119 purposes.
2120 .TP
2121 .B maximum number of hardlinks of a file in a directory
2122 65536 when the \fIextref\fP feature is turned on during mkfs (default), roughly
2123 100 otherwise and depends on file name length that fits into one metadata node
2124 .TP
2125 .B minimum filesystem size
2126 the minimal size of each device depends on the \fImixed\-bg\fP feature, without that
2127 (the default) it\(aqs about 109MiB, with mixed\-bg it\(aqs is 16MiB
2128 .UNINDENT
2129 .SH BOOTLOADER SUPPORT
2130 .sp
2131 GRUB2 (\%<https://\:www\:.gnu\:.org/\:software/\:grub>) has the most advanced support of
2132 booting from BTRFS with respect to features.
2133 .sp
2134 U\-Boot (\%<https://\:www\:.denx\:.de/\:wiki/\:U-Boot/>) has decent support for booting but
2135 not all BTRFS features are implemented, check the documentation.
2136 .sp
2137 In general, the first 1MiB on each device is unused with the exception of
2138 primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be
2139 freely used by bootloaders or for other system information. Note that booting
2140 from a filesystem on zoned device \%<> is not supported.
2141 .SH FILE ATTRIBUTES
2142 .sp
2143 The btrfs filesystem supports setting file attributes or flags. Note there are
2144 old and new interfaces, with confusing names. The following list should clarify
2145 that:
2146 .INDENT 0.0
2147 .IP \(bu 2
2148 \fIattributes\fP: chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> or lsattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:lsattr\:.1\:.html> utilities (the ioctls are
2149 FS_IOC_GETFLAGS and FS_IOC_SETFLAGS), due to the ioctl names the attributes
2150 are also called flags
2151 .IP \(bu 2
2152 \fIxflags\fP: to distinguish from the previous, it\(aqs extended flags, with tunable
2153 bits similar to the attributes but extensible and new bits will be added in
2154 the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR but they
2155 are not related to extended attributes that are also called xattrs), there\(aqs
2156 no standard tool to change the bits, there\(aqs support in xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> as
2157 command \fBxfs_io \-c chattr\fP
2158 .UNINDENT
2159 .sp
2160 Attributes have constraints associated and not all combinations can be set, the
2161 order of setting them also matters. Most attributes apply to files and
2162 directories but the semantics may differ. For directories the attribute may
2163 only mean to set this attribute to all new files (\fIinheritable\fP in the list
2164 below). Some attributes need root privileges to be set.
2165 .SS Attributes
2166 .INDENT 0.0
2167 .TP
2168 .B a
2169 (file, dir, root)
2170 \fIappend only\fP, new writes are always written at the end of the file
2171 .TP
2172 .B A
2173 (file, dir)
2174 \fIno atime updates\fP
2175 .TP
2176 .B c
2177 (file, dir, inherited)
2178 \fIcompress data\fP, all data written after this attribute is set will be compressed.
2179 Please note that compression is also affected by the mount options or the parent
2180 directory attributes.
2181 .sp
2182 When set on a directory, all newly created files will inherit this attribute.
2183 This attribute cannot be set with \(aqm\(aq at the same time.
2184 .TP
2185 .B C
2186 (file, dir, inherited)
2187 \fIno copy\-on\-write\fP, file data modifications are done in\-place
2188 .sp
2189 When set on a directory, all newly created files will inherit this attribute.
2190 .sp
2191 \fBNote:\fP
2192 .INDENT 7.0
2193 .INDENT 3.5
2194 Due to implementation limitations, this flag can be set/unset only on
2195 empty files.
2196 .UNINDENT
2197 .UNINDENT
2198 .TP
2199 .B d
2200 (file)
2201 \fIno dump\fP, makes sense with 3rd party tools like dump(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:dump\:.8\:.html>, on BTRFS the
2202 attribute can be set/unset but no other special handling is done
2203 .TP
2204 .B D
2205 (dir)
2206 \fIsynchronous directory updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP
2207 and \fIO_DSYNC\fP
2208 .TP
2209 .B i
2210 (file, dir, root)
2211 \fIimmutable\fP, no file data and metadata changes allowed even to the root user as
2212 long as this attribute is set (obviously the exception is unsetting the attribute)
2213 .TP
2214 .B m
2215 (file, dir)
2216 \fIno compression\fP, permanently turn off compression on the given file. Any
2217 compression mount options will not affect this file. (chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> support added in
2218 1.46.2)
2219 .sp
2220 When set on a directory, all newly created files will inherit this attribute.
2221 This attribute cannot be set with \fIc\fP at the same time.
2222 .TP
2223 .B S
2224 (file)
2225 \fIsynchronous updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP and
2226 \fIO_DSYNC\fP
2227 .TP
2228 .B V
2229 (file, read\-only)
2230 \fIfs\-verity enabled\fP on the file
2231 .UNINDENT
2232 .sp
2233 No other attributes are supported. For the complete list please refer to the
2234 chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> manual page.
2235 .SS XFLAGS
2236 .sp
2237 There\(aqs an overlap of letters assigned to the bits with the attributes, this list
2238 refers to what xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> provides:
2239 .INDENT 0.0
2240 .TP
2241 .B i
2242 \fIimmutable\fP, same as the attribute
2243 .TP
2244 .B a
2245 \fIappend only\fP, same as the attribute
2246 .TP
2247 .B s
2248 \fIsynchronous updates\fP, same as the attribute \fIS\fP
2249 .TP
2250 .B A
2251 \fIno atime updates\fP, same as the attribute
2252 .TP
2253 .B d
2254 \fIno dump\fP, same as the attribute
2255 .UNINDENT
2256 .SH ZONED MODE
2257 .sp
2258 Since version 5.12 btrfs supports so called \fIzoned mode\fP\&. This is a special
2259 on\-disk format and allocation/write strategy that\(aqs friendly to zoned devices.
2260 In short, a device is partitioned into fixed\-size zones and each zone can be
2261 updated by append\-only manner, or reset. As btrfs has no fixed data structures,
2262 except the super blocks, the zoned mode only requires block placement that
2263 follows the device constraints. You can learn about the whole architecture at
2264 \%<https://\:zonedstorage\:.io> .
2265 .sp
2266 The devices are also called SMR/ZBC/ZNS, in \fIhost\-managed\fP mode. Note that
2267 there are devices that appear as non\-zoned but actually are, this is
2268 \fIdrive\-managed\fP and using zoned mode won\(aqt help.
2269 .sp
2270 The zone size depends on the device, typical sizes are 256MiB or 1GiB. In
2271 general it must be a power of two. Emulated zoned devices like \fInull_blk\fP allow
2272 to set various zone sizes.
2273 .SS Requirements, limitations
2274 .INDENT 0.0
2275 .IP \(bu 2
2276 all devices must have the same zone size
2277 .IP \(bu 2
2278 maximum zone size is 8GiB
2279 .IP \(bu 2
2280 minimum zone size is 4MiB
2281 .IP \(bu 2
2282 mixing zoned and non\-zoned devices is possible, the zone writes are emulated,
2283 but this is namely for testing
2284 .IP \(bu 2
2285 the super block is handled in a special way and is at different locations than on a non\-zoned filesystem:
2286 .INDENT 2.0
2287 .IP \(bu 2
2288 primary: 0B (and the next two zones)
2289 .IP \(bu 2
2290 secondary: 512GiB (and the next two zones)
2291 .IP \(bu 2
2292 tertiary: 4TiB (4096GiB, and the next two zones)
2293 .UNINDENT
2294 .UNINDENT
2295 .SS Incompatible features
2296 .sp
2297 The main constraint of the zoned devices is lack of in\-place update of the data.
2298 This is inherently incompatible with some features:
2299 .INDENT 0.0
2300 .IP \(bu 2
2301 NODATACOW \- overwrite in\-place, cannot create such files
2302 .IP \(bu 2
2303 fallocate \- preallocating space for in\-place first write
2304 .IP \(bu 2
2305 mixed\-bg \- unordered writes to data and metadata, fixing that means using
2306 separate data and metadata block groups
2307 .IP \(bu 2
2308 booting \- the zone at offset 0 contains superblock, resetting the zone would
2309 destroy the bootloader data
2310 .UNINDENT
2311 .sp
2312 Initial support lacks some features but they\(aqre planned:
2313 .INDENT 0.0
2314 .IP \(bu 2
2315 only single (data, metadata) and DUP (metadata) profile is supported
2316 .IP \(bu 2
2317 fstrim \- due to dependency on free space cache v1
2318 .UNINDENT
2319 .SS Super block
2320 .sp
2321 As said above, super block is handled in a special way. In order to be crash
2322 safe, at least one zone in a known location must contain a valid superblock.
2323 This is implemented as a ring buffer in two consecutive zones, starting from
2324 known offsets 0B, 512GiB and 4TiB.
2325 .sp
2326 The values are different than on non\-zoned devices. Each new super block is
2327 appended to the end of the zone, once it\(aqs filled, the zone is reset and writes
2328 continue to the next one. Looking up the latest super block needs to read
2329 offsets of both zones and determine the last written version.
2330 .sp
2331 The amount of space reserved for super block depends on the zone size. The
2332 secondary and tertiary copies are at distant offsets as the capacity of the
2333 devices is expected to be large, tens of terabytes. Maximum zone size supported
2334 is 8GiB, which would mean that e.g. offset 0\-16GiB would be reserved just for
2335 the super block on a hypothetical device of that zone size. This is wasteful
2336 but required to guarantee crash safety.
2337 .SS Zone reclaim, garbage collection
2338 .sp
2339 As the zones are append\-only, overwriting data or COW changes in metadata
2340 make parts of the zones used but not connected to the filesystem structures.
2341 This makes the space unusable and grows over time. Once the ratio hits a
2342 (configurable) threshold a background reclaim process is started and relocates
2343 the remaining blocks in use to a new zone. The old one is reset and can be used
2344 again.
2345 .sp
2346 This process may take some time depending on other background work or
2347 amount of new data written. It is possible to hit an intermittent ENOSPC.
2348 Some devices also limit number of active zones.
2349 .SS Devices
2350 .SS Real hardware
2351 .sp
2352 The WD Ultrastar series 600 advertises HM\-SMR, i.e. the host\-managed zoned
2353 mode. There are two more: DA (device managed, no zoned information exported to
2354 the system), HA (host aware, can be used as regular disk but zoned writes
2355 improve performance). There are not many devices available at the moment, the
2356 information about exact zoned mode is hard to find, check data sheets or
2357 community sources gathering information from real devices.
2358 .sp
2359 Note: zoned mode won\(aqt work with DM\-SMR disks.
2360 .INDENT 0.0
2361 .IP \(bu 2
2362 Ultrastar® DC ZN540 NVMe ZNS SSD (product
2363 brief \%<https://\:documents\:.westerndigital\:.com/\:content/\:dam/\:doc-library/\:en_us/\:assets/\:public/\:western-digital/\:collateral/\:product-brief/\:product-brief-ultrastar-dc-zn540\:.pdf>)
2364 .UNINDENT
2365 .SS Emulated: null_blk
2366 .sp
2367 The driver \fInull_blk\fP provides memory backed device and is suitable for
2368 testing. There are some quirks setting up the devices. The module must be
2369 loaded with \fInr_devices=0\fP or the numbering of device nodes will be offset. The
2370 \fIconfigfs\fP must be mounted at \fI/sys/kernel/config\fP and the administration of
2371 the null_blk devices is done in \fI/sys/kernel/config/nullb\fP\&. The device nodes
2372 are named like \fB/dev/nullb0\fP and are numbered sequentially. NOTE: the device
2373 name may be different than the named directory in sysfs!
2374 .sp
2375 Setup:
2376 .INDENT 0.0
2377 .INDENT 3.5
2378 .sp
2379 .EX
2380 modprobe\ configfs
2381 modprobe\ null_blk\ nr_devices=0
2382 .EE
2383 .UNINDENT
2384 .UNINDENT
2385 .sp
2386 Create a device \fImydev\fP, assuming no other previously created devices, size is
2387 2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal
2388 example taking defaults:
2389 .INDENT 0.0
2390 .INDENT 3.5
2391 .sp
2392 .EX
2393 cd\ /sys/kernel/config/nullb/
2394 mkdir\ mydev
2395 cd\ mydev
2396 echo\ 2048\ >\ size
2397 echo\ 1\ >\ zoned
2398 echo\ 1\ >\ memory_backed
2399 echo\ 256\ >\ zone_size
2400 echo\ 1\ >\ power
2401 .EE
2402 .UNINDENT
2403 .UNINDENT
2404 .sp
2405 This will create a device \fB/dev/nullb0\fP and the value of file \fIindex\fP will
2406 match the ending number of the device node.
2407 .sp
2408 Remove the device:
2409 .INDENT 0.0
2410 .INDENT 3.5
2411 .sp
2412 .EX
2413 rmdir\ /sys/kernel/config/nullb/mydev
2414 .EE
2415 .UNINDENT
2416 .UNINDENT
2417 .sp
2418 Then continue with \fBmkfs.btrfs /dev/nullb0\fP, the zoned mode is auto\-detected.
2419 .sp
2420 For convenience, there\(aqs a script wrapping the basic null_blk management operations
2421 \%<https://\:github\:.com/\:kdave/\:nullb\:.git>, the above commands become:
2422 .INDENT 0.0
2423 .INDENT 3.5
2424 .sp
2425 .EX
2426 nullb setup
2427 nullb create \-s 2g \-z 256
2428 mkfs.btrfs /dev/nullb0
2429 \&...
2430 nullb rm nullb0
2431 .EE
2432 .UNINDENT
2433 .UNINDENT
2434 .SS Emulated: TCMU runner
2435 .sp
2436 TCMU is a framework to emulate SCSI devices in userspace, providing various
2437 backends for the storage, with zoned support as well. A file\-backed zoned
2438 device can provide more options for larger storage and zone size. Please follow
2439 the instructions at \%<https://\:zonedstorage\:.io/\:projects/\:tcmu-runner/> .
2440 .SS Compatibility, incompatibility
2441 .INDENT 0.0
2442 .IP \(bu 2
2443 the feature sets an incompat bit and requires new kernel to access the
2444 filesystem (for both read and write)
2445 .IP \(bu 2
2446 superblock needs to be handled in a special way, there are still 3 copies
2447 but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a
2448 ring buffer of the superblocks, finding the latest one needs reading it from
2449 the write pointer or do a full scan of the zones
2450 .IP \(bu 2
2451 mixing zoned and non zoned devices is possible (zones are emulated) but is
2452 recommended only for testing
2453 .IP \(bu 2
2454 mixing zoned devices with different zone sizes is not possible
2455 .IP \(bu 2
2456 zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB
2457 or 1GiB, larger size is expected, maximum zone size supported by btrfs is
2458 8GiB
2459 .UNINDENT
2460 .SS Status, stability, reporting bugs
2461 .sp
2462 The zoned mode has been released in 5.12 and there are still some rough edges
2463 and corner cases one can hit during testing. Please report bugs to
2464 \%<https://\:github\:.com/\:naota/\:linux/\:issues/> .
2465 .SS References
2466 .INDENT 0.0
2467 .IP \(bu 2
2468 \%<https://\:zonedstorage\:.io>
2469 .INDENT 2.0
2470 .IP \(bu 2
2471 \%<https://\:zonedstorage\:.io/\:projects/\:libzbc/> \-\- \fIlibzbc\fP is library and set
2472 of tools to directly manipulate devices with ZBC/ZAC support
2473 .IP \(bu 2
2474 \%<https://\:zonedstorage\:.io/\:projects/\:libzbd/> \-\- \fIlibzbd\fP uses the kernel
2475 provided zoned block device interface based on the ioctl() system calls
2476 .UNINDENT
2477 .IP \(bu 2
2478 \%<https://\:hddscan\:.com/\:blog/\:2020/\:hdd-wd-smr\:.html> \-\- some details about exact device types
2479 .IP \(bu 2
2480 \%<https://\:lwn\:.net/\:Articles/\:853308/> \-\- \fIBtrfs on zoned block devices\fP
2481 .IP \(bu 2
2482 \%<https://\:www\:.usenix\:.org/\:conference/\:vault20/\:presentation/\:bjorling> \-\- Zone
2483 Append: A New Way of Writing to Zoned Storage
2484 .UNINDENT
2485 .SH CONTROL DEVICE
2486 .sp
2487 There\(aqs a character special device \fB/dev/btrfs\-control\fP with major and minor
2488 numbers 10 and 234 (the device can be found under the \fImisc\fP category).
2489 .INDENT 0.0
2490 .INDENT 3.5
2491 .sp
2492 .EX
2493 $ ls \-l /dev/btrfs\-control
2494 crw\-\-\-\-\-\-\- 1 root root 10, 234 Jan 1 12:00 /dev/btrfs\-control
2495 .EE
2496 .UNINDENT
2497 .UNINDENT
2498 .sp
2499 The device accepts some ioctl calls that can perform following actions on the
2500 filesystem module:
2501 .INDENT 0.0
2502 .IP \(bu 2
2503 scan devices for btrfs filesystem (i.e. to let multi\-device filesystems mount
2504 automatically) and register them with the kernel module
2505 .IP \(bu 2
2506 similar to scan, but also wait until the device scanning process is finished
2507 for a given filesystem
2508 .IP \(bu 2
2509 get the supported features (can be also found under \fB/sys/fs/btrfs/features\fP)
2510 .UNINDENT
2511 .sp
2512 The device is created when btrfs is initialized, either as a module or a
2513 built\-in functionality and makes sense only in connection with that. Running
2514 e.g. mkfs without the module loaded will not register the device and will
2515 probably warn about that.
2516 .sp
2517 In rare cases when the module is loaded but the device is not present (most
2518 likely accidentally deleted), it\(aqs possible to recreate it by
2519 .INDENT 0.0
2520 .INDENT 3.5
2521 .sp
2522 .EX
2523 # mknod \-\-mode=600 /dev/btrfs\-control c 10 234
2524 .EE
2525 .UNINDENT
2526 .UNINDENT
2527 .sp
2528 or (since 5.11) by a convenience command
2529 .INDENT 0.0
2530 .INDENT 3.5
2531 .sp
2532 .EX
2533 # btrfs rescue create\-control\-device
2534 .EE
2535 .UNINDENT
2536 .UNINDENT
2537 .sp
2538 The control device is not strictly required but the device scanning will not
2539 work and a workaround would need to be used to mount a multi\-device filesystem.
2540 The mount option \fIdevice\fP can trigger the device scanning during mount, see
2541 also \fBbtrfs device scan\fP\&.
2542 .SH FILESYSTEM WITH MULTIPLE PROFILES
2543 .sp
2544 It is possible that a btrfs filesystem contains multiple block group profiles
2545 of the same type. This could happen when a profile conversion using balance
2546 filters is interrupted (see btrfs\-balance(8) \%<>). Some
2547 \fBbtrfs\fP commands perform
2548 a test to detect this kind of condition and print a warning like this:
2549 .INDENT 0.0
2550 .INDENT 3.5
2551 .sp
2552 .EX
2553 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2554 WARNING: Data: single, raid1
2555 WARNING: Metadata: single, raid1
2556 .EE
2557 .UNINDENT
2558 .UNINDENT
2559 .sp
2560 The corresponding output of \fBbtrfs filesystem df\fP might look like:
2561 .INDENT 0.0
2562 .INDENT 3.5
2563 .sp
2564 .EX
2565 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2566 WARNING: Data: single, raid1
2567 WARNING: Metadata: single, raid1
2568 Data, RAID1: total=832.00MiB, used=0.00B
2569 Data, single: total=1.63GiB, used=0.00B
2570 System, single: total=4.00MiB, used=16.00KiB
2571 Metadata, single: total=8.00MiB, used=112.00KiB
2572 Metadata, RAID1: total=64.00MiB, used=32.00KiB
2573 GlobalReserve, single: total=16.25MiB, used=0.00B
2574 .EE
2575 .UNINDENT
2576 .UNINDENT
2577 .sp
2578 There\(aqs more than one line for type \fIData\fP and \fIMetadata\fP, while the profiles
2579 are \fIsingle\fP and \fIRAID1\fP\&.
2580 .sp
2581 This state of the filesystem OK but most likely needs the user/administrator to
2582 take an action and finish the interrupted tasks. This cannot be easily done
2583 automatically, also the user knows the expected final profiles.
2584 .sp
2585 In the example above, the filesystem started as a single device and \fIsingle\fP
2586 block group profile. Then another device was added, followed by balance with
2587 \fIconvert=raid1\fP but for some reason hasn\(aqt finished. Restarting the balance
2588 with \fIconvert=raid1\fP will continue and end up with filesystem with all block
2589 group profiles \fIRAID1\fP\&.
2590 .sp
2591 \fBNote:\fP
2592 .INDENT 0.0
2593 .INDENT 3.5
2594 If you\(aqre familiar with balance filters, you can use
2595 \fIconvert=raid1,profiles=single,soft\fP, which will take only the unconverted
2596 \fIsingle\fP profiles and convert them to \fIraid1\fP\&. This may speed up the conversion
2597 as it would not try to rewrite the already convert \fIraid1\fP profiles.
2598 .UNINDENT
2599 .UNINDENT
2600 .sp
2601 Having just one profile is desired as this also clearly defines the profile of
2602 newly allocated block groups, otherwise this depends on internal allocation
2603 policy. When there are multiple profiles present, the order of selection is
2604 RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are
2605 satisfied.
2606 .sp
2607 Commands that print the warning were chosen so they\(aqre brought to user
2608 attention when the filesystem state is being changed in that regard. This is:
2609 \fBdevice add\fP, \fBdevice delete\fP, \fBbalance cancel\fP, \fBbalance pause\fP\&. Commands
2610 that report space usage: \fBfilesystem df\fP, \fBdevice usage\fP\&. The command
2611 \fBfilesystem usage\fP provides a line in the overall summary:
2612 .INDENT 0.0
2613 .INDENT 3.5
2614 .sp
2615 .EX
2616 Multiple profiles: yes (data, metadata)
2617 .EE
2618 .UNINDENT
2619 .UNINDENT
2620 .SH SEEDING DEVICE
2621 .sp
2622 The COW mechanism and multiple devices under one hood enable an interesting
2623 concept, called a seeding device: extending a read\-only filesystem on a
2624 device with another device that captures all writes. For example
2625 imagine an immutable golden image of an operating system enhanced with another
2626 device that allows to use the data from the golden image and normal operation.
2627 This idea originated on CD\-ROMs with base OS and allowing to use them for live
2628 systems, but this became obsolete. There are technologies providing similar
2629 functionality, like unionmount \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Union_mount>,
2630 overlayfs \%<https://\:en\:.wikipedia\:.org/\:wiki/\:OverlayFS> or
2631 qcow2 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Qcow#\:qcow2> image snapshot.
2632 .sp
2633 The seeding device starts as a normal filesystem, once the contents is ready,
2634 \fBbtrfstune \-S 1\fP is used to flag it as a seeding device. Mounting such device
2635 will not allow any writes, except adding a new device by \fBbtrfs device add\fP\&.
2636 Then the filesystem can be remounted as read\-write.
2637 .sp
2638 Given that the filesystem on the seeding device is always recognized as
2639 read\-only, it can be used to seed multiple filesystems from one device at the
2640 same time. The UUID that is normally attached to a device is automatically
2641 changed to a random UUID on each mount.
2642 .sp
2643 \fBNote:\fP
2644 .INDENT 0.0
2645 .INDENT 3.5
2646 Before v6.17 kernel, a seed device could have been mounted
2647 independently along with sprouted filesystems.
2648 But since 6.17 kernel, a seed device can only be mounted either through
2649 a sprouted filesystem, or the seed device itself, not both at the same time.
2650 .sp
2651 This is to ensure a block device to have only a single filesystem bound
2652 to it, so that runtime device missing events can be properly handled.
2653 .UNINDENT
2654 .UNINDENT
2655 .sp
2656 Once the seeding device is mounted, it needs the writable device. After adding
2657 it, unmounting and mounting with \fBumount /path; mount /dev/writable
2658 /path\fP or remounting read\-write with \fBremount \-o remount,rw\fP makes the
2659 filesystem at \fB/path\fP ready for use.
2660 .sp
2661 \fBNote:\fP
2662 .INDENT 0.0
2663 .INDENT 3.5
2664 There was a known bug with using remount to make the mount writeable:
2665 remount will leave the filesystem in a state where it is unable to
2666 clean deleted snapshots, so it will leak space until it is unmounted
2667 and mounted properly.
2668 .sp
2669 That bug has been fixed in 5.11 and newer kernels.
2670 .UNINDENT
2671 .UNINDENT
2672 .sp
2673 Furthermore, deleting the seeding device from the filesystem can turn it into
2674 a normal filesystem, provided that the writable device can also contain all the
2675 data from the seeding device.
2676 .sp
2677 The seeding device flag can be cleared again by \fBbtrfstune \-f \-S 0\fP, e.g.
2678 allowing to update with newer data but please note that this will invalidate
2679 all existing filesystems that use this particular seeding device. This works
2680 for some use cases, not for others, and the forcing flag to the command is
2681 mandatory to avoid accidental mistakes.
2682 .sp
2683 Example how to create and use one seeding device:
2684 .INDENT 0.0
2685 .INDENT 3.5
2686 .sp
2687 .EX
2688 # mkfs.btrfs /dev/sda
2689 # mount /dev/sda /mnt/mnt1
2690 \&... fill mnt1 with data
2691 # umount /mnt/mnt1
2692
2693 # btrfstune \-S 1 /dev/sda
2694
2695 # mount /dev/sda /mnt/mnt1
2696 # btrfs device add /dev/sdb /mnt/mnt1
2697 # umount /mnt/mnt1
2698 # mount /dev/sdb /mnt/mnt1
2699 \&... /mnt/mnt1 is now writable
2700 .EE
2701 .UNINDENT
2702 .UNINDENT
2703 .sp
2704 Now \fB/mnt/mnt1\fP can be used normally. The device \fB/dev/sda\fP can be mounted
2705 again with a another writable device:
2706 .INDENT 0.0
2707 .INDENT 3.5
2708 .sp
2709 .EX
2710 # mount /dev/sda /mnt/mnt2
2711 # btrfs device add /dev/sdc /mnt/mnt2
2712 # umount /mnt/mnt2
2713 # mount /dev/sdc /mnt/mnt2
2714 \&... /mnt/mnt2 is now writable
2715 .EE
2716 .UNINDENT
2717 .UNINDENT
2718 .sp
2719 The writable device (file:\fI/dev/sdb\fP) can be decoupled from the seeding device and
2720 used independently:
2721 .INDENT 0.0
2722 .INDENT 3.5
2723 .sp
2724 .EX
2725 # btrfs device delete /dev/sda /mnt/mnt1
2726 .EE
2727 .UNINDENT
2728 .UNINDENT
2729 .sp
2730 As the contents originated in the seeding device, it\(aqs possible to turn
2731 \fB/dev/sdb\fP to a seeding device again and repeat the whole process.
2732 .sp
2733 A few things to note:
2734 .INDENT 0.0
2735 .IP \(bu 2
2736 it\(aqs recommended to use only single device for the seeding device, it works
2737 for multiple devices but the \fIsingle\fP profile must be used in order to make
2738 the seeding device deletion work
2739 .IP \(bu 2
2740 block group profiles \fIsingle\fP and \fIdup\fP support the use cases above
2741 .IP \(bu 2
2742 the label is copied from the seeding device and can be changed by \fBbtrfs filesystem label\fP
2743 .IP \(bu 2
2744 each new mount of the seeding device gets a new random UUID
2745 .IP \(bu 2
2746 \fBumount /path; mount /dev/writable /path\fP can be replaced with
2747 \fBmount \-o remount,rw /path\fP
2748 but it won\(aqt reclaim space of deleted subvolumes until the seeding device
2749 is mounted read\-write again before making it seeding again
2750 .UNINDENT
2751 .SS Chained seeding devices
2752 .sp
2753 Though it\(aqs not recommended and is rather an obscure and untested use case,
2754 chaining seeding devices is possible. In the first example, the writable device
2755 \fB/dev/sdb\fP can be turned onto another seeding device again, depending on the
2756 unchanged seeding device \fB/dev/sda\fP\&. Then using \fB/dev/sdb\fP as the primary
2757 seeding device it can be extended with another writable device, say \fB/dev/sdd\fP,
2758 and it continues as before as a simple tree structure on devices.
2759 .INDENT 0.0
2760 .INDENT 3.5
2761 .sp
2762 .EX
2763 # mkfs.btrfs /dev/sda
2764 # mount /dev/sda /mnt/mnt1
2765 \&... fill mnt1 with data
2766 # umount /mnt/mnt1
2767
2768 # btrfstune \-S 1 /dev/sda
2769
2770 # mount /dev/sda /mnt/mnt1
2771 # btrfs device add /dev/sdb /mnt/mnt1
2772 # mount \-o remount,rw /mnt/mnt1
2773 \&... /mnt/mnt1 is now writable
2774 # umount /mnt/mnt1
2775
2776 # btrfstune \-S 1 /dev/sdb
2777
2778 # mount /dev/sdb /mnt/mnt1
2779 # btrfs device add /dev/sdc /mnt
2780 # mount \-o remount,rw /mnt/mnt1
2781 \&... /mnt/mnt1 is now writable
2782 # umount /mnt/mnt1
2783 .EE
2784 .UNINDENT
2785 .UNINDENT
2786 .sp
2787 As a result we have:
2788 .INDENT 0.0
2789 .IP \(bu 2
2790 \fIsda\fP is a single seeding device, with its initial contents
2791 .IP \(bu 2
2792 \fIsdb\fP is a seeding device but requires \fIsda\fP, the contents are from the time
2793 when \fIsdb\fP is made seeding, i.e. contents of \fIsda\fP with any later changes
2794 .IP \(bu 2
2795 \fIsdc\fP last writable, can be made a seeding one the same way as was \fIsdb\fP,
2796 preserving its contents and depending on \fIsda\fP and \fIsdb\fP
2797 .UNINDENT
2798 .sp
2799 As long as the seeding devices are unmodified and available, they can be used
2800 to start another branch.
2801 .SH RAID56 STATUS AND RECOMMENDED PRACTICES
2802 .sp
2803 The RAID56 feature provides striping and parity over several devices, same as
2804 the traditional RAID5/6. There are some implementation and design deficiencies
2805 that make it unreliable for some corner cases and the feature \fBshould not be
2806 used in production, only for evaluation or testing\fP\&. The power failure safety
2807 for metadata with RAID56 is not 100%.
2808 .SS Metadata
2809 .sp
2810 Do not use \fIraid5\fP nor \fIraid6\fP for metadata. Use \fIraid1\fP or \fIraid1c3\fP
2811 respectively.
2812 .sp
2813 The substitute profiles provide the same guarantees against loss of 1 or 2
2814 devices, and in some respect can be an improvement. Recovering from one
2815 missing device will only need to access the remaining 1st or 2nd copy, that in
2816 general may be stored on some other devices due to the way RAID1 works on
2817 btrfs, unlike on a striped profile (similar to \fIraid0\fP) that would need all
2818 devices all the time.
2819 .sp
2820 The space allocation pattern and consumption is different (e.g. on N devices):
2821 for \fIraid5\fP as an example, a 1GiB chunk is reserved on each device, while with
2822 \fIraid1\fP there\(aqs each 1GiB chunk stored on 2 devices. The consumption of each
2823 1GiB of used metadata is then \fIN * 1GiB\fP for vs \fI2 * 1GiB\fP\&. Using \fIraid1\fP
2824 is also more convenient for balancing/converting to other profile due to lower
2825 requirement on the available chunk space.
2826 .SS Missing/incomplete support
2827 .sp
2828 When RAID56 is on the same filesystem with different raid profiles, the space
2829 reporting is inaccurate, e.g. \fBdf\fP, \fBbtrfs filesystem df\fP or
2830 \fBbtrfs filesystem usage\fP\&. When there\(aqs only a one profile per block
2831 group type (e.g. RAID5 for data) the reporting is accurate.
2832 .sp
2833 When scrub is started on a RAID56 filesystem, it\(aqs started on all devices that
2834 degrade the performance. The workaround is to start it on each device
2835 separately. Due to that the device stats may not match the actual state and
2836 some errors might get reported multiple times.
2837 .sp
2838 The \fIwrite hole\fP problem. An unclean shutdown could leave a partially written
2839 stripe in a state where the some stripe ranges and the parity are from the old
2840 writes and some are new. The information which is which is not tracked. Write
2841 journal is not implemented. Alternatively a full read\-modify\-write would make
2842 sure that a full stripe is always written, avoiding the write hole completely,
2843 but performance in that case turned out to be too bad for use.
2844 .sp
2845 The striping happens on all available devices (at the time the chunks were
2846 allocated), so in case a new device is added it may not be utilized
2847 immediately and would require a rebalance. A fixed configured stripe width is
2848 not implemented.
2849 .SH GLOSSARY
2850 .sp
2851 Terms in \fIitalics\fP also appear in this glossary.
2852 .INDENT 0.0
2853 .TP
2854 .B allocator
2855 Usually \fIallocator\fP means the \fIblock\fP allocator, i.e. the logic
2856 inside the filesystem which decides where to place newly allocated blocks
2857 in order to maintain several constraints (like data locality, low
2858 fragmentation).
2859 .sp
2860 In btrfs, allocator may also refer to \fIchunk\fP allocator, i.e. the
2861 logic behind placing chunks on devices.
2862 .TP
2863 .B balance
2864 An operation that can be done to a btrfs filesystem, for example
2865 through \fBbtrfs balance /path\fP\&. A
2866 balance passes all data in the filesystem through the \fIallocator\fP
2867 again. It is primarily intended to rebalance the data in the filesystem
2868 across the \fIdevices\fP when a device is added or removed. A balance
2869 will regenerate missing copies for the redundant \fIRAID\fP levels, if a
2870 device has failed. As of Linux kernel 3.3, a balance operation can be
2871 made selective about which parts of the filesystem are rewritten
2872 using filters \%<#\:man-balance-filters>\&.
2873 .TP
2874 .B barrier
2875 An instruction to the underlying hardware to ensure that everything before
2876 the barrier is physically written to permanent storage before anything
2877 after it. Used in btrfs\(aqs \fIcopy on write\fP approach to ensure
2878 filesystem consistency.
2879 .TP
2880 .B block
2881 A single physically and logically contiguous piece of storage on a
2882 device, of size e.g. 4K. In some contexts also referred to as \fIsector\fP,
2883 though the term \fIblock\fP is preferred.
2884 .TP
2885 .B block group
2886 The unit of allocation of space in btrfs. A block group is laid out on
2887 the disk by the btrfs \fIallocator\fP, and will consist of one or more
2888 \fIchunks\fP, each stored on a different \fIdevice\fP\&. The number of chunks
2889 used in a block group will depend on its \fIRAID\fP level.
2890 .TP
2891 .B B\-tree
2892 The fundamental storage data structure used in btrfs. Except for the
2893 \fIsuperblocks\fP, all of btrfs \fImetadata\fP is stored in one of several
2894 B\-trees on disk. B\-trees store key/item pairs. While the same code is
2895 used to implement all of the B\-trees, there are a few different
2896 categories of B\-tree. The name \fIbtrfs\fP refers to its use of B\-trees.
2897 .TP
2898 .B btrfsck, fsck, btrfs\-check
2899 Tool in \fIbtrfs\-progs\fP that checks an unmounted filesystem (\fIoffline\fP)
2900 and reports on any errors in the filesystem structures it finds. By
2901 default the tool runs in read\-only mode as fixing errors is potentially
2902 dangerous. See also \fIscrub\fP\&.
2903 .TP
2904 .B btrfs\-progs
2905 User mode tools to manage btrfs\-specific features. Maintained at
2906 \%<http://\:github\:.com/\:kdave/\:btrfs-progs\:.git> . The main frontend to btrfs
2907 features is the standalone tool \fIbtrfs\fP, although
2908 other tools such as \fImkfs.btrfs\fP and \fIbtrfstune\fP are also part of
2909 btrfs\-progs.
2910 .TP
2911 .B chunk
2912 A part of a \fIblock group\fP\&. Chunks are either 1 GiB in size (for data)
2913 or 256 MiB (for \fImetadata\fP), depending on the overall filesystem size.
2914 .TP
2915 .B chunk tree
2916 A layer that keeps information about mapping between physical and
2917 logical block addresses. It\(aqs stored within the \fIsystem\fP group.
2918 .TP
2919 .B cleaner
2920 Usually referred to in context of deleted subvolumes. It\(aqs a background
2921 process that removes the actual data once a subvolume has been deleted.
2922 Cleaning can involve lots of IO and CPU activity depending on the
2923 fragmentation and amount of shared data with other subvolumes.
2924 .sp
2925 The cleaner kernel thread also processes defragmentation triggered by the
2926 \fIautodefrag\fP mount option, removing of empty blocks groups and some
2927 other finalization tasks.
2928 .TP
2929 .B copy\-on\-write, COW
2930 Also known as \fICOW\fP\&. The method that btrfs uses for modifying data.
2931 Instead of directly overwriting data in place, btrfs takes a copy of
2932 the data, alters it, and then writes the modified data back to a
2933 different (unused) location on the disk. It then updates the \fImetadata\fP
2934 to reflect the new location of the data. In order to update the
2935 metadata, the affected metadata blocks are also treated in the same
2936 way. In COW filesystems, files tend to fragment as they are modified.
2937 Copy\-on\-write is also used in the implementation of \fIsnapshots\fP and
2938 \fIreflink copies\fP\&. A copy\-on\-write filesystem is, in theory,
2939 \fIalways\fP consistent, provided the underlying hardware supports
2940 \fIbarriers\fP\&.
2941 .TP
2942 .B default subvolume
2943 The \fIsubvolume\fP in a btrfs filesystem which is mounted when mounting
2944 the filesystem without using the \fBsubvol=\fP mount option.
2945 .TP
2946 .B device
2947 A Linux block device, e.g. a whole disk, partition, LVM logical volume,
2948 loopback device, or network block device. A btrfs filesystem can reside
2949 on one or more devices.
2950 .TP
2951 .B df
2952 A standard Unix tool for reporting the amount of space used and free in
2953 a filesystem. The standard tool does not give accurate results, but the
2954 \fIbtrfs\fP command from \fIbtrfs\-progs\fP has
2955 an implementation of \fIdf\fP which shows space available in more detail. See
2956 the
2957 [[FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F|FAQ]]
2958 for a more detailed explanation of btrfs free space accounting.
2959 .TP
2960 .B DUP
2961 A form of \(dq\fIRAID\fP\(dq which stores two copies of each piece of data on
2962 the same \fIdevice\fP\&. This is similar to \fIRAID1\fP, and protects
2963 against \fIblock\fP\-level errors on the device, but does not provide any
2964 guarantees if the entire device fails. By default, btrfs uses \fIDUP\fP
2965 profile for metadata on single device filesystem.s
2966 .TP
2967 .B ENOSPC
2968 Error code returned by the OS to a user program when the filesystem
2969 cannot allocate enough data to fulfill the user request. In most
2970 filesystems, it indicates there is no free space available in the
2971 filesystem. Due to the additional space requirements from btrfs\(aqs
2972 \fICOW\fP behaviour, btrfs can sometimes return ENOSPC when there is
2973 apparently (in terms of \fIdf\fP) a large amount of space free. This is
2974 effectively a bug in btrfs, and (if it is repeatable), using the mount
2975 option \fBenospc_debug\fP may give a report
2976 that will help the btrfs developers. See the
2977 [[FAQ#if_your_device_is_large_.28.3E16GiB.29|FAQ entry]] on free space.
2978 .TP
2979 .B extent
2980 Contiguous sequence of bytes on disk that holds file data. It\(aqs a compact
2981 representation that tracks the start and length of the byte range, so the
2982 logic behind allocating blocks (\fIdelayed allocation\fP) strives for
2983 maximizing the length before writing the extents to the devices.
2984 .TP
2985 .B extent buffer
2986 An abstraction of a \fIb\-tree\fP metadata block storing item keys and item
2987 data. The underlying related structures are physical device block and a
2988 CPU memory page.
2989 .TP
2990 .B fallocate
2991 Command line tool in util\-linux, and a syscall, that reserves space in
2992 the filesystem for a file, without actually writing any file data to
2993 the filesystem. First data write will turn the preallocated extents
2994 into regular ones. See fallocate(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:fallocate\:.1\:.html> and fallocate(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fallocate\:.2\:.html> manual pages
2995 for more details.
2996 .TP
2997 .B filefrag
2998 A tool to show the number of extents in a file, and hence the amount of
2999 fragmentation in the file. It is usually part of the e2fsprogs package
3000 on most Linux distributions. While initially developed for the ext2
3001 filesystem, it works on Btrfs as well. It uses the \fIFIEMAP\fP ioctl.
3002 .TP
3003 .B free space cache
3004 Also known as \(dqspace cache v1\(dq. A separate cache tracking free space as
3005 btrfs only tracks the allocated space. The free space is by definition
3006 any hole between allocated ranges. Finding the free ranges can be
3007 I/O intensive so the cache stores a condensed representation of it.
3008 It is updated every \fItransaction\fP commit.
3009 .sp
3010 The v1 free space cache has been superseded by free space tree.
3011 .TP
3012 .B free space tree
3013 Successor of \fIfree space cache\fP, also known as \(dqspace cache v2\(dq and now
3014 default. The free space is tracked in a better way and using COW
3015 unlike a custom mechanism of v1.
3016 .TP
3017 .B fsync
3018 On Unix and Unix\-like operating systems (of which Linux is the latter),
3019 the fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> system call causes all buffered file
3020 descriptor related data changes to be flushed to the underlying block
3021 device. When a file is modified on a modern operating system the
3022 changes are generally not written to the disk immediately but rather
3023 those changes are buffered in memory for performance reasons,
3024 calling fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> causes any in\-memory changes to be written
3025 to disk.
3026 .TP
3027 .B generation
3028 An internal counter which updates for each \fItransaction\fP\&. When a
3029 \fImetadata\fP block is written (using \fIcopy on write\fP), current
3030 generation is stored in the block, so that blocks which are too new
3031 (and hence possibly inconsistent) can be identified.
3032 .TP
3033 .B key
3034 A fixed sized tuple used to identify and sort items in a \fIB\-tree\fP\&.
3035 The key is broken up into 3 parts: \fIobjectid\fP, \fItype\fP, and
3036 \fIoffset\fP\&. The \fItype\fP field indicates how each of the other two
3037 fields should be used, and what to expect to find in the item.
3038 .TP
3039 .B item
3040 A variable sized structure stored in B\-tree leaves. Items hold
3041 different types of data depending on key type.
3042 .TP
3043 .B log tree
3044 A b\-tree that temporarily tracks ongoing metadata updates until a full
3045 transaction commit is done. It\(aqs a performance optimization of
3046 \fIfsync\fP\&. The log tracked in the tree are replayed if the filesystem
3047 is not unmounted cleanly.
3048 .TP
3049 .B metadata
3050 Data about data. In btrfs, this includes all of the internal data
3051 structures of the filesystem, including directory structures,
3052 filenames, file permissions, checksums, and the location of each file\(aqs
3053 \fIextents\fP\&. All btrfs metadata is stored in \fIB\-trees\fP\&.
3054 .TP
3055 .B mkfs.btrfs
3056 The tool (from \fIbtrfs\-progs\fP) to create a btrfs filesystem.
3057 .TP
3058 .B offline
3059 A filesystem which is not mounted is offline. Some tools (e.g.
3060 \fIbtrfsck\fP) will only work on offline filesystems. Compare \fIonline\fP\&.
3061 .TP
3062 .B online
3063 A filesystem which is mounted is online. Most btrfs tools will only
3064 work on online filesystems. Compare \fIoffline\fP\&.
3065 .TP
3066 .B orphan
3067 A file that\(aqs still in use (opened by a running process) but all
3068 directory entries of that file have been removed.
3069 .TP
3070 .B RAID
3071 A class of different methods for writing some additional redundant data
3072 across multiple \fIdevices\fP so that if one device fails, the missing
3073 data can be reconstructed from the remaining ones. See \fIRAID0\fP,
3074 \fIRAID1\fP, \fIRAID5\fP, \fIRAID6\fP, \fIRAID10\fP, \fIDUP\fP and
3075 \fIsingle\fP\&. Traditional RAID methods operate across multiple devices of
3076 equal size, whereas btrfs\(aq RAID implementation works inside \fIblock
3077 groups\fP\&.
3078 .TP
3079 .B RAID0
3080 A form of \fIRAID\fP which provides no guarantees of error recovery, but
3081 stripes a single copy of data across multiple devices for performance
3082 purposes. The stripe size is fixed to 64KB for now.
3083 .TP
3084 .B RAID1, RAID1C3, RAID1C4
3085 A form of \fIRAID\fP which stores two/three/four complete copies of each
3086 piece of data. Each copy is stored on a different \fIdevice\fP\&. btrfs
3087 requires a minimum of two devices to use RAID\-1 or three/four respectively.
3088 This is the default block group profile for btrfs\(aqs \fImetadata\fP on more
3089 than one device.
3090 .TP
3091 .B RAID5
3092 A form of \fIRAID\fP which stripes a single copy of data across multiple
3093 \fIdevices\fP, including one device\(aqs worth of additional parity data.
3094 Can be used to recover from a single device failure.
3095 .TP
3096 .B RAID6
3097 A form of \fIRAID\fP which stripes a single copy of data across multiple
3098 \fIdevices\fP, including two device\(aqs worth of additional parity data. Can
3099 be used to recover from the failure of two devices.
3100 .TP
3101 .B RAID10
3102 A form of \fIRAID\fP which stores two complete copies of each piece of
3103 data, and also stripes each copy across multiple devices for
3104 performance.
3105 .TP
3106 .B reflink
3107 Commonly used as a reference to a shallow copy of file extents that share
3108 the extents until the first change. Reflinked files (e.g. by the \fBcp\fP)
3109 are different files but point to the same extents, any change will be
3110 detected and new copy of the data created, keeping the files independent.
3111 Related to that is extent range cloning, that works on a range of a file.
3112 .TP
3113 .B relocation
3114 The process of moving block groups within the filesystem while
3115 maintaining full filesystem integrity and consistency. This
3116 functionality is underlying \fIbalance\fP and \fIdevice\fP removing features.
3117 .TP
3118 .B scrub \%<>
3119 An \fIonline\fP filesystem checking tool. Reads all the data and metadata
3120 on the filesystem, verifies \fIchecksums\fP and eventually uses redundant
3121 copies from \fIRAID\fP or \fIDUP\fP repair any corrupt data/metadata.
3122 .TP
3123 .B seed device \%<>
3124 A readonly device can be used as a filesystem seed or template (e.g. a
3125 CD\-ROM containing an OS image). Read/write devices can be added to
3126 store modifications (using \fIcopy on write\fP), changes to the writable
3127 devices are persistent across reboots. The original device remains
3128 unchanged and can be removed at any time (after Btrfs has been
3129 instructed to copy over all missing blocks). Multiple read/write file
3130 systems can be built from the same seed.
3131 .TP
3132 .B single
3133 A block group profile storing a single copy of each piece of data.
3134 .TP
3135 .B snapshot \%<>
3136 A \fIsubvolume\fP which is a \fIcopy on write\fP copy of another subvolume. The
3137 two subvolumes share all of their common (unmodified) data, which means
3138 that snapshots can be used to keep the historical state of a filesystem
3139 very cheaply. After the snapshot is made, the original subvolume and
3140 the snapshot are of equal status: the original does not \(dqown\(dq the
3141 snapshot, and either one can be deleted without affecting the other
3142 one.
3143 .TP
3144 .B subvolume \%<>
3145 A tree of files and directories inside a btrfs that can be mounted as
3146 if it were an independent filesystem. A subvolume is created by taking
3147 a reference on the root of another subvolume. Each btrfs filesystem has
3148 at least one subvolume, the \fItop\-level subvolume\fP, which contains
3149 everything else in the filesystem. Additional subvolumes can be created
3150 and deleted with the \fIbtrfs<\fP tool. All subvolumes share the same pool
3151 of free space in the filesystem. See also \fIdefault subvolume\fP\&.
3152 .TP
3153 .B super block
3154 A special metadata block that is a main access point of the filesystem
3155 structures. It\(aqs size is fixed and there are fixed locations on the devices
3156 used for detecting and opening the filesystem. Updating the superblock
3157 defines one \fItransaction\fP\&. The super blocks contains filesystem
3158 identification (UUID), checksum type, block pointers to fundamental
3159 trees, features and creation parameters.
3160 .TP
3161 .B system array
3162 A technical term for \fIsuper block\fP metadata describing how to assemble a
3163 filesystem from multiple device, storing information about chunks and devices that are
3164 required to be scanned/registered at the time the mount happens.
3165 Scanning is done by command \fBbtrfs device scan\fP, alternatively
3166 all the required devices can be specified by a mount option \fIdevice=/path\fP\&.
3167 .TP
3168 .B top\-level subvolume
3169 The \fIsubvolume\fP at the very top of the filesystem. This is the only
3170 subvolume present in a newly\-created btrfs filesystem, and internally has ID 5,
3171 otherwise could be referenced as 0 (e.g. within the \fIset\-default\fP subcommand of
3172 \fIbtrfs\fP).
3173 .TP
3174 .B transaction
3175 A consistent set of changes. To avoid generating very large amounts of
3176 disk activity, btrfs caches changes in RAM for up to 30 seconds
3177 (sometimes more often if the filesystem is running short on space or
3178 doing a lot of \fIfsync*s), and then writes (commits) these changes out
3179 to disk in one go (using *copy on write\fP behaviour). This period of
3180 caching is called a transaction. Only one transaction is active on the
3181 filesystem at any one time.
3182 .TP
3183 .B transid
3184 An alternative term for \fIgeneration\fP\&.
3185 .TP
3186 .B writeback
3187 \fIWriteback\fP in the context of the Linux kernel can be defined as the
3188 process of writing \(dqdirty\(dq memory from the page cache to the disk,
3189 when certain conditions are met (timeout, number of dirty pages over a
3190 ratio).
3191 .UNINDENT
3192 .SH STORAGE MODEL, HARDWARE CONSIDERATIONS
3193 .SS Storage model
3194 .sp
3195 \fIA storage model is a model that captures key physical aspects of data
3196 structure in a data store. A filesystem is the logical structure organizing
3197 data on top of the storage device.\fP
3198 .sp
3199 The filesystem assumes several features or limitations of the storage device
3200 and utilizes them or applies measures to guarantee reliability. BTRFS in
3201 particular is based on a COW (copy on write) mode of writing, i.e. not updating
3202 data in place but rather writing a new copy to a different location and then
3203 atomically switching the pointers.
3204 .sp
3205 In an ideal world, the device does what it promises. The filesystem assumes
3206 that this may not be true so additional mechanisms are applied to either detect
3207 misbehaving hardware or get valid data by other means. The devices may (and do)
3208 apply their own detection and repair mechanisms but we won\(aqt assume any.
3209 .sp
3210 The following assumptions about storage devices are considered (sorted by
3211 importance, numbers are for further reference):
3212 .INDENT 0.0
3213 .IP 1. 3
3214 atomicity of reads and writes of blocks/sectors (the smallest unit of data
3215 the device presents to the upper layers)
3216 .IP 2. 3
3217 there\(aqs a flush command that instructs the device to forcibly order writes
3218 before and after the command; alternatively there\(aqs a barrier command that
3219 facilitates the ordering but may not flush the data
3220 .IP 3. 3
3221 data sent to write to a given device offset will be written without further
3222 changes to the data and to the offset
3223 .IP 4. 3
3224 writes can be reordered by the device, unless explicitly serialized by the
3225 flush command
3226 .IP 5. 3
3227 reads and writes can be freely reordered and interleaved
3228 .UNINDENT
3229 .sp
3230 The consistency model of BTRFS builds on these assumptions. The logical data
3231 updates are grouped, into a generation, written on the device, serialized by
3232 the flush command and then the super block is written ending the generation.
3233 All logical links among metadata comprising a consistent view of the data may
3234 not cross the generation boundary.
3235 .SS When things go wrong
3236 .sp
3237 \fBNo or partial atomicity of block reads/writes (1)\fP
3238 .INDENT 0.0
3239 .IP \(bu 2
3240 \fIProblem\fP: a partial block contents is written (\fItorn write\fP), e.g. due to a
3241 power glitch or other electronics failure during the read/write
3242 .IP \(bu 2
3243 \fIDetection\fP: checksum mismatch on read
3244 .IP \(bu 2
3245 \fIRepair\fP: use another copy or rebuild from multiple blocks using some encoding
3246 scheme
3247 .UNINDENT
3248 .sp
3249 \fBThe flush command does not flush (2)\fP
3250 .sp
3251 This is perhaps the most serious problem and impossible to mitigate by
3252 filesystem without limitations and design restrictions. What could happen in
3253 the worst case is that writes from one generation bleed to another one, while
3254 still letting the filesystem consider the generations isolated. Crash at any
3255 point would leave data on the device in an inconsistent state without any hint
3256 what exactly got written, what is missing and leading to stale metadata link
3257 information.
3258 .sp
3259 Devices usually honor the flush command, but for performance reasons may do
3260 internal caching, where the flushed data are not yet persistently stored. A
3261 power failure could lead to a similar scenario as above, although it\(aqs less
3262 likely that later writes would be written before the cached ones. This is
3263 beyond what a filesystem can take into account. Devices or controllers are
3264 usually equipped with batteries or capacitors to write the cache contents even
3265 after power is cut. (\fIBattery backed write cache\fP)
3266 .sp
3267 \fBData get silently changed on write (3)\fP
3268 .sp
3269 Such thing should not happen frequently, but still can happen spuriously due
3270 the complex internal workings of devices or physical effects of the storage
3271 media itself.
3272 .INDENT 0.0
3273 .IP \(bu 2
3274 \fIProblem\fP: while the data are written atomically, the contents get changed
3275 .IP \(bu 2
3276 \fIDetection\fP: checksum mismatch on read
3277 .IP \(bu 2
3278 \fIRepair\fP: use another copy or rebuild from multiple blocks using some
3279 encoding scheme
3280 .UNINDENT
3281 .sp
3282 \fBData get silently written to another offset (3)\fP
3283 .sp
3284 This would be another serious problem as the filesystem has no information
3285 when it happens. For that reason the measures have to be done ahead of time.
3286 This problem is also commonly called \fIghost write\fP\&.
3287 .sp
3288 The metadata blocks have the checksum embedded in the blocks, so a correct
3289 atomic write would not corrupt the checksum. It\(aqs likely that after reading
3290 such block the data inside would not be consistent with the rest. To rule that
3291 out there\(aqs embedded block number in the metadata block. It\(aqs the logical
3292 block number because this is what the logical structure expects and verifies.
3293 .sp
3294 The following is based on information publicly available, user feedback,
3295 community discussions or bug report analyses. It\(aqs not complete and further
3296 research is encouraged when in doubt.
3297 .SS Main memory
3298 .sp
3299 The data structures and raw data blocks are temporarily stored in computer
3300 memory before they get written to the device. It is critical that memory is
3301 reliable because even simple bit flips can have vast consequences and lead to
3302 damaged structures, not only in the filesystem but in the whole operating
3303 system.
3304 .sp
3305 Based on experience in the community, memory bit flips are more common than one
3306 would think. When it happens, it\(aqs reported by the tree\-checker or by a checksum
3307 mismatch after reading blocks. There are some very obvious instances of bit
3308 flips that happen, e.g. in an ordered sequence of keys in metadata blocks. We can
3309 easily infer from the other data what values get damaged and how. However, fixing
3310 that is not straightforward and would require cross\-referencing data from the
3311 entire filesystem to see the scope.
3312 .sp
3313 If available, ECC memory should lower the chances of bit flips, but this
3314 type of memory is not available in all cases. A memory test should be performed
3315 in case there\(aqs a visible bit flip pattern, though this may not detect a faulty
3316 memory module because the actual load of the system could be the factor making
3317 the problems appear. In recent years attacks on how the memory modules operate
3318 have been demonstrated (\fIrowhammer\fP) achieving specific bits to be flipped.
3319 While these were targeted, this shows that a series of reads or writes can
3320 affect unrelated parts of memory.
3321 .sp
3322 Block group profiles with redundancy (like RAID1) will not protect against
3323 memory errors as the blocks are first stored in memory before they are written
3324 to the devices from the same source.
3325 .sp
3326 A filesystem mounted read\-only will not affect the underlying block device in
3327 almost 100% (with highly unlikely exceptions). The exception is a tree\-log that
3328 needs to be replayed during mount (and before the read\-only mount takes place),
3329 working memory is needed for that and that can be affected by bit flips.
3330 There\(aqs a theoretical case where bit flip changes the filesystem status from
3331 read\-only to read\-write.
3332 .sp
3333 Further reading:
3334 .INDENT 0.0
3335 .IP \(bu 2
3336 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Row_hammer>
3337 .IP \(bu 2
3338 memory overclocking, XMP, potential risks
3339 .UNINDENT
3340 .sp
3341 What to do:
3342 .INDENT 0.0
3343 .IP \(bu 2
3344 run \fImemtest\fP, note that sometimes memory errors happen only when the system
3345 is under heavy load that the default memtest cannot trigger
3346 .IP \(bu 2
3347 memory errors may appear as filesystem going read\-only due to \(dqpre write\(dq
3348 check, that verify meta data before they get written but fail some basic
3349 consistency checks
3350 .IP \(bu 2
3351 newly built systems should be tested before being put to production use,
3352 ideally start a IO/CPU load that will be run on such system later; namely
3353 systems that will utilize overclocking or special performance features
3354 .UNINDENT
3355 .SS Direct memory access (DMA)
3356 .sp
3357 Another class of errors is related to DMA (direct memory access) performed
3358 by device drivers. While this could be considered a software error, the
3359 data transfers that happen without CPU assistance may accidentally corrupt
3360 other pages. Storage devices utilize DMA for performance reasons, the
3361 filesystem structures and data pages are passed back and forth, making
3362 errors possible in case page life time is not properly tracked.
3363 .sp
3364 There are lots of quirks (device\-specific workarounds) in Linux kernel
3365 drivers (regarding not only DMA) that are added when found. The quirks
3366 may avoid specific errors or disable some features to avoid worse problems.
3367 .sp
3368 What to do:
3369 .INDENT 0.0
3370 .IP \(bu 2
3371 use up\-to\-date kernel (recent releases or maintained long term support versions)
3372 .IP \(bu 2
3373 as this may be caused by faulty drivers, keep the systems up\-to\-date
3374 .UNINDENT
3375 .SS Rotational disks (HDD)
3376 .sp
3377 Rotational HDDs typically fail at the level of individual sectors or small clusters.
3378 Read failures are caught on the levels below the filesystem and are returned to
3379 the user as \fIEIO \- Input/output error\fP\&. Reading the blocks repeatedly may
3380 return the data eventually, but this is better done by specialized tools and
3381 filesystem takes the result of the lower layers. Rewriting the sectors may
3382 trigger internal remapping but this inevitably leads to data loss.
3383 .sp
3384 Disk firmware is technically software but from the filesystem perspective is
3385 part of the hardware. IO requests are processed, and caching or various
3386 other optimizations are performed, which may lead to bugs under high load or
3387 unexpected physical conditions or unsupported use cases.
3388 .sp
3389 Disks are connected by cables with two ends, both of which can cause problems
3390 when not attached properly. Data transfers are protected by checksums and the
3391 lower layers try hard to transfer the data correctly or not at all. The errors
3392 from badly\-connecting cables may manifest as large amount of failed read or
3393 write requests, or as short error bursts depending on physical conditions.
3394 .sp
3395 What to do:
3396 .INDENT 0.0
3397 .IP \(bu 2
3398 check \fBsmartctl\fP for potential issues
3399 .UNINDENT
3400 .SS Solid state drives (SSD)
3401 .sp
3402 The mechanism of information storage is different from HDDs and this affects
3403 the failure mode as well. The data are stored in cells grouped in large blocks
3404 with limited number of resets and other write constraints. The firmware tries
3405 to avoid unnecessary resets and performs optimizations to maximize the storage
3406 media lifetime. The known techniques are deduplication (blocks with same
3407 fingerprint/hash are mapped to same physical block), compression or internal
3408 remapping and garbage collection of used memory cells. Due to the additional
3409 processing there are measures to verify the data e.g. by ECC codes.
3410 .sp
3411 The observations of failing SSDs show that the whole electronic fails at once
3412 or affects a lot of data (e.g. stored on one chip). Recovering such data
3413 may need specialized equipment and reading data repeatedly does not help as
3414 it\(aqs possible with HDDs.
3415 .sp
3416 There are several technologies of the memory cells with different
3417 characteristics and price. The lifetime is directly affected by the type and
3418 frequency of data written. Writing \(dqtoo much\(dq distinct data (e.g. encrypted)
3419 may render the internal deduplication ineffective and lead to a lot of rewrites
3420 and increased wear of the memory cells.
3421 .sp
3422 There are several technologies and manufacturers so it\(aqs hard to describe them
3423 but there are some that exhibit similar behaviour:
3424 .INDENT 0.0
3425 .IP \(bu 2
3426 expensive SSD will use more durable memory cells and is optimized for
3427 reliability and high load
3428 .IP \(bu 2
3429 cheap SSD is projected for a lower load (\(dqdesktop user\(dq) and is optimized for
3430 cost, it may employ the optimizations and/or extended error reporting
3431 partially or not at all
3432 .UNINDENT
3433 .sp
3434 It\(aqs not possible to reliably determine the expected lifetime of an SSD due to
3435 lack of information about how it works or due to lack of reliable stats provided
3436 by the device.
3437 .sp
3438 Metadata writes tend to be the biggest component of lifetime writes to a SSD,
3439 so there is some value in reducing them. Depending on the device class (high
3440 end/low end) the features like DUP block group profiles may affect the
3441 reliability in both ways:
3442 .INDENT 0.0
3443 .IP \(bu 2
3444 \fIhigh end\fP are typically more reliable and using \fIsingle\fP for data and
3445 metadata could be suitable to reduce device wear
3446 .IP \(bu 2
3447 \fIlow end\fP could lack ability to identify errors so an additional redundancy
3448 at the filesystem level (checksums, \fIDUP\fP) could help
3449 .UNINDENT
3450 .sp
3451 Only users who consume 50 to 100% of the SSD\(aqs actual lifetime writes need to be
3452 concerned by the write amplification of btrfs DUP metadata. Most users will be
3453 far below 50% of the actual lifetime, or will write the drive to death and
3454 discover how many writes 100% of the actual lifetime was. SSD firmware often
3455 adds its own write multipliers that can be arbitrary and unpredictable and
3456 dependent on application behavior, and these will typically have far greater
3457 effect on SSD lifespan than DUP metadata. It\(aqs more or less impossible to
3458 predict when a SSD will run out of lifetime writes to within a factor of two, so
3459 it\(aqs hard to justify wear reduction as a benefit.
3460 .sp
3461 Further reading:
3462 .INDENT 0.0
3463 .IP \(bu 2
3464 \%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-and-deduplication-end-spinning-disk-2012>
3465 .IP \(bu 2
3466 \%<https://\:www\:.snia\:.org/\:educational-library/\:realities-solid-state-storage-2013-2013>
3467 .IP \(bu 2
3468 \%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-performance-primer-2013>
3469 .IP \(bu 2
3470 \%<https://\:www\:.snia\:.org/\:educational-library/\:how-controllers-maximize-ssd-life-2013>
3471 .UNINDENT
3472 .sp
3473 What to do:
3474 .INDENT 0.0
3475 .IP \(bu 2
3476 run \fBsmartctl\fP or self\-tests to look for potential issues
3477 .IP \(bu 2
3478 keep the firmware up\-to\-date
3479 .UNINDENT
3480 .SS NVM express, non\-volatile memory (NVMe)
3481 .sp
3482 NVMe is a type of persistent memory usually connected over a system bus (PCIe)
3483 or similar interface and the speeds are an order of magnitude faster than SSD.
3484 It is also a non\-rotating type of storage, and is not typically connected by a
3485 cable. It\(aqs not a SCSI type device either but rather a complete specification
3486 for logical device interface.
3487 .sp
3488 In a way the errors could be compared to a combination of SSD class and regular
3489 memory. Errors may exhibit as random bit flips or IO failures. There are tools
3490 to access the internal log (\fBnvme log\fP and \fBnvme\-cli\fP) for a more detailed
3491 analysis.
3492 .sp
3493 There are separate error detection and correction steps performed e.g. on the
3494 bus level and in most cases never making in to the filesystem level. Once this
3495 happens it could mean there\(aqs some systematic error like overheating or bad
3496 physical connection of the device. You may want to run self\-tests (using
3497 \fBsmartctl\fP).
3498 .INDENT 0.0
3499 .IP \(bu 2
3500 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:NVM_Express>
3501 .IP \(bu 2
3502 \%<https://\:www\:.smartmontools\:.org/\:wiki/\:NVMe_Support>
3503 .UNINDENT
3504 .SS Drive firmware
3505 .sp
3506 Firmware is technically still software but embedded into the hardware. As all
3507 software has bugs, so does firmware. Storage devices can update the firmware
3508 and fix known bugs. In some cases the it\(aqs possible to avoid certain bugs by
3509 quirks (device\-specific workarounds) in Linux kernel.
3510 .sp
3511 A faulty firmware can cause wide range of corruptions from small and localized
3512 to large affecting lots of data. Self\-repair capabilities may not be sufficient.
3513 .sp
3514 What to do:
3515 .INDENT 0.0
3516 .IP \(bu 2
3517 check for firmware updates in case there are known problems, note that
3518 updating firmware can be risky on itself
3519 .IP \(bu 2
3520 use up\-to\-date kernel (recent releases or maintained long term support versions)
3521 .UNINDENT
3522 .SS SD flash cards
3523 .sp
3524 There are a lot of devices with low power consumption and thus using storage
3525 media based on low power consumption too, typically flash memory stored on
3526 a chip enclosed in a detachable card package. An improperly inserted card may be
3527 damaged by electrical spikes when the device is turned on or off. The chips
3528 storing data in turn may be damaged permanently. All types of flash memory
3529 have a limited number of rewrites, so the data are internally translated by FTL
3530 (flash translation layer). This is implemented in firmware (technically a
3531 software) and prone to bugs that manifest as hardware errors.
3532 .sp
3533 Adding redundancy like using DUP profiles for both data and metadata can help
3534 in some cases but a full backup might be the best option once problems appear
3535 and replacing the card could be required as well.
3536 .SS Hardware as the main source of filesystem corruptions
3537 .sp
3538 \fBIf you use unreliable hardware and don\(aqt know about that, don\(aqt blame the
3539 filesystem when it tells you.\fP
3540 .SH SEE ALSO
3541 .sp
3542 acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html>,
3543 btrfs(8) \%<>,
3544 chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>,
3545 fstrim(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:fstrim\:.8\:.html>,
3546 ioctl(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:ioctl\:.2\:.html>,
3547 btrfs\-ioctl(2) \%<>,
3548 mkfs.btrfs(8) \%<>,
3549 mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html>,
3550 swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html>
3551 .\" End of generated man page.