Debian Codegraph @

upstream/debian-trixie/man5/btrfs.5

raw
1 '\" t
2 .\" Man page generated from reStructuredText.
3 .
4 .
5 .nr rst2man-indent-level 0
6 .
7 .de1 rstReportMargin
8 \\$1 \\n[an-margin]
9 level \\n[rst2man-indent-level]
10 level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
11 -
12 \\n[rst2man-indent0]
13 \\n[rst2man-indent1]
14 \\n[rst2man-indent2]
15 ..
16 .de1 INDENT
17 .\" .rstReportMargin pre:
18 . RS \\$1
19 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
20 . nr rst2man-indent-level +1
21 .\" .rstReportMargin post:
22 ..
23 .de UNINDENT
24 . RE
25 .\" indent \\n[an-margin]
26 .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
27 .nr rst2man-indent-level -1
28 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
29 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
30 ..
31 .TH "BTRFS" "5" "May 18, 2026" "7.0" "BTRFS"
32 .SH NAME
33 btrfs \- topics about the BTRFS filesystem (mount options, supported file attributes and other)
34 .SH DESCRIPTION
35 .sp
36 This document describes topics related to BTRFS that are not specific to the
37 tools. Currently covers:
38 .INDENT 0.0
39 .IP 1. 4
40 \fI\%mount options\fP
41 .IP 2. 4
42 \fI\%filesystem features\fP
43 .IP 3. 4
44 \fI\%swapfile support\fP
45 .IP 4. 4
46 \fI\%checksum algorithms\fP
47 .IP 5. 4
48 \fI\%compression\fP
49 .IP 6. 4
50 \fI\%sysfs interface\fP
51 .IP 7. 4
52 \fI\%filesystem exclusive operations\fP
53 .IP 8. 4
54 \fI\%filesystem limits\fP
55 .IP 9. 4
56 \fI\%bootloader support\fP
57 .IP 10. 4
58 \fI\%file attributes\fP
59 .IP 11. 4
60 \fI\%zoned mode\fP
61 .IP 12. 4
62 \fI\%control device\fP
63 .IP 13. 4
64 \fI\%filesystems with multiple block group profiles\fP
65 .IP 14. 4
66 \fI\%seeding device\fP
67 .IP 15. 4
68 \fI\%RAID56 status and recommended practices\fP
69 .IP 16. 4
70 \fI\%glossary\fP
71 .IP 17. 4
72 \fI\%storage model, hardware considerations\fP
73 .UNINDENT
74 .SH MOUNT OPTIONS
75 .SS BTRFS SPECIFIC MOUNT OPTIONS
76 .sp
77 This section describes mount options specific to BTRFS. For the generic mount
78 options please refer to \X'tty: link https://man7.org/linux/man-pages/man8/mount.8.html'\fI\%mount(8)\fP\X'tty: link' manual page and also see the section
79 with BTRFS specifics \fI\%below\fP\&. The options are
80 sorted alphabetically (discarding the \fIno\fP prefix).
81 .sp
82 \fBNOTE:\fP
83 .INDENT 0.0
84 .INDENT 3.5
85 Most mount options apply to the whole filesystem and only options in the
86 first mounted subvolume will take effect. This is due to lack of implementation
87 and may change in the future. This means that (for example) you can\(aqt set
88 per\-subvolume \fInodatacow\fP, \fInodatasum\fP, or \fIcompress\fP using mount options. This
89 should eventually be fixed, but it has proved to be difficult to implement
90 correctly within the Linux VFS framework.
91 .UNINDENT
92 .UNINDENT
93 .sp
94 Mount options are processed in order, only the last occurrence of an option
95 takes effect and may disable other options due to constraints (see e.g.
96 \fInodatacow\fP and \fIcompress\fP). The output of \fBmount\fP command shows which options
97 have been applied.
98 .INDENT 0.0
99 .TP
100 .B acl, noacl
101 (default: on)
102 .sp
103 Enable/disable support for POSIX Access Control Lists (ACLs). See the
104 \X'tty: link https://man7.org/linux/man-pages/man5/acl.5.html'\fI\%acl(5)\fP\X'tty: link' manual page for more information about ACLs.
105 .sp
106 The support for ACL is build\-time configurable (BTRFS_FS_POSIX_ACL) and
107 mount fails if \fIacl\fP is requested but the feature is not compiled in.
108 .UNINDENT
109 .INDENT 0.0
110 .TP
111 .B autodefrag, noautodefrag
112 (since: 3.0, default: off)
113 .sp
114 Enable automatic file defragmentation.
115 When enabled, small random writes into files (in a range of tens of kilobytes,
116 currently it\(aqs 64KiB) are detected and queued up for the defragmentation process.
117 May not be well suited for large database workloads.
118 .sp
119 The read latency may increase due to reading the adjacent blocks that make up the
120 range for defragmentation, successive write will merge the blocks in the new
121 location.
122 .sp
123 \fBWARNING:\fP
124 .INDENT 7.0
125 .INDENT 3.5
126 Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14\-rc2 as
127 well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or
128 ≥ 3.13.4 will break up the reflinks of COW data (for example files
129 copied with \fBcp \-\-reflink\fP, snapshots or de\-duplicated data).
130 This may cause considerable increase of space usage depending on the
131 broken up reflinks.
132 .UNINDENT
133 .UNINDENT
134 .TP
135 .B barrier, nobarrier
136 (default: on)
137 .sp
138 Ensure that all IO write operations make it through the device cache and are stored
139 permanently when the filesystem is at its consistency checkpoint. This
140 typically means that a flush command is sent to the device that will
141 synchronize all pending data and ordinary metadata blocks, then writes the
142 superblock and issues another flush.
143 .sp
144 The write flushes incur a slight hit and also prevent the IO block
145 scheduler to reorder requests in a more effective way. Disabling barriers gets
146 rid of that penalty but will most certainly lead to a corrupted filesystem in
147 case of a crash or power loss. The ordinary metadata blocks could be yet
148 unwritten at the time the new superblock is stored permanently, expecting that
149 the block pointers to metadata were stored permanently before.
150 .sp
151 On a device with a volatile battery\-backed write\-back cache, the \fInobarrier\fP
152 option will not lead to filesystem corruption as the pending blocks are
153 supposed to make it to the permanent storage.
154 .TP
155 .B clear_cache
156 Force clearing and rebuilding of the free space cache if something
157 has gone wrong.
158 This option only takes effect on the first read\-write mount.
159 .sp
160 For free space cache \fIv1\fP, this only clears (and, unless \fInospace_cache\fP is
161 used, rebuilds) the free space cache for block groups that are modified while
162 the filesystem is mounted with that option. To actually clear an entire free
163 space cache \fIv1\fP, see \fBbtrfs check \-\-clear\-space\-cache v1\fP\&.
164 .sp
165 For free space cache \fIv2\fP, this clears the entire free space cache.
166 To do so without requiring to mounting the filesystem, see
167 \fBbtrfs check \-\-clear\-space\-cache v2\fP\&.
168 .sp
169 See also: \fIspace_cache\fP\&.
170 .TP
171 .B commit=<seconds>
172 (since: 3.12, default: 30)
173 .sp
174 Set the interval of periodic transaction commit when data are synchronized
175 to permanent storage. Higher interval values lead to larger amount of unwritten
176 data to accumulate in memory, which has obvious consequences when the
177 system crashes. The upper bound is not forced, but a warning is
178 printed if it\(aqs more than 300 seconds (5 minutes). Use with care.
179 .sp
180 The periodic commit is not the only mechanism to do the transaction commit,
181 this can also happen by explicit \fBsync\fP or indirectly by other
182 commands that affect the global filesystem state or internal kernel
183 mechanisms that flush based on various thresholds or policies (e.g. cgroups).
184 .TP
185 .B compress, compress=<type[:level]>, compress\-force, compress\-force=<type[:level]>
186 (default: off, level support since: 5.1)
187 .sp
188 Control BTRFS file data compression. Type may be specified as \fIzlib\fP,
189 \fIlzo\fP, \fIzstd\fP or \fIno\fP (for no compression, used for remounting). If no type
190 is specified, \fIzlib\fP is used. If \fIcompress\-force\fP is specified,
191 then compression will always be attempted, but the data may end up uncompressed
192 if the compression would make them larger.
193 .sp
194 Both \fIzlib\fP and \fIzstd\fP (since version 5.1) expose the compression level as a
195 tunable knob with higher levels trading speed and memory (\fIzstd\fP) for higher
196 compression ratios. This can be set by appending a colon and the desired level.
197 ZLIB accepts the range [1, 9] and ZSTD accepts [1, 15]. If no level is set,
198 both currently use a default level of 3. The value 0 is an alias for the
199 default level.
200 .sp
201 Otherwise some simple heuristics are applied to detect an incompressible file.
202 If the first blocks written to a file are not compressible, the whole file is
203 permanently marked to skip compression. As this is too simple, the
204 \fIcompress\-force\fP is a workaround that will compress most of the files at the
205 cost of some wasted CPU cycles on failed attempts.
206 Since kernel 4.15, a set of heuristic algorithms have been improved by using
207 frequency sampling, repeated pattern detection and Shannon entropy calculation
208 to avoid that.
209 .sp
210 \fBNOTE:\fP
211 .INDENT 7.0
212 .INDENT 3.5
213 If compression is enabled, \fInodatacow\fP and \fInodatasum\fP are disabled.
214 .UNINDENT
215 .UNINDENT
216 .TP
217 .B datacow, nodatacow
218 (default: on)
219 .sp
220 Enable data copy\-on\-write for newly created files.
221 \fINodatacow\fP implies \fInodatasum\fP, and disables \fIcompression\fP\&. All files created
222 under \fInodatacow\fP are also set the NOCOW file attribute (see \X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link').
223 .sp
224 \fBNOTE:\fP
225 .INDENT 7.0
226 .INDENT 3.5
227 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
228 .UNINDENT
229 .UNINDENT
230 .sp
231 Updates in\-place improve performance for workloads that do frequent overwrites,
232 at the cost of potential partial writes, in case the write is interrupted
233 (system crash, device failure).
234 .TP
235 .B datasum, nodatasum
236 (default: on)
237 .sp
238 Enable data checksumming for newly created files.
239 \fIDatasum\fP implies \fIdatacow\fP, i.e. the normal mode of operation. All files created
240 under \fInodatasum\fP inherit the \(dqno checksums\(dq property, however there\(aqs no
241 corresponding file attribute (see \X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link').
242 .sp
243 \fBNOTE:\fP
244 .INDENT 7.0
245 .INDENT 3.5
246 If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled.
247 .UNINDENT
248 .UNINDENT
249 .sp
250 There is a slight performance gain when checksums are turned off, the
251 corresponding metadata blocks holding the checksums do not need to updated.
252 The cost of checksumming of the blocks in memory is much lower than the IO,
253 modern CPUs feature hardware support of the checksumming algorithm.
254 .UNINDENT
255 .INDENT 0.0
256 .TP
257 .B degraded
258 (default: off)
259 .sp
260 Allow mounts with fewer devices than the RAID profile constraints
261 require. A read\-write mount (or remount) may fail when there are too many devices
262 missing, for example if a stripe member is completely missing from RAID0.
263 .sp
264 Since 4.14, the constraint checks have been improved and are verified on the
265 chunk level, not at the device level. This allows degraded mounts of
266 filesystems with mixed RAID profiles for data and metadata, even if the
267 device number constraints would not be satisfied for some of the profiles.
268 .sp
269 Example: metadata \-\- raid1, data \-\- single, devices \-\- \fB/dev/sda\fP, \fB/dev/sdb\fP
270 .sp
271 Suppose the data are completely stored on \fIsda\fP, then missing \fIsdb\fP will not
272 prevent the mount, even if 1 missing device would normally prevent (any)
273 \fIsingle\fP profile to mount. In case some of the data chunks are stored on \fIsdb\fP,
274 then the constraint of single/data is not satisfied and the filesystem
275 cannot be mounted.
276 .UNINDENT
277 .INDENT 0.0
278 .TP
279 .B device=<devicepath>
280 Specify a path to a device that will be scanned for BTRFS filesystem during
281 mount. This is usually done automatically by a device manager (like udev) or
282 using the \fBbtrfs device scan\fP command (e.g. run from the initial ramdisk). In
283 cases where this is not possible the \fIdevice\fP mount option can help.
284 .sp
285 \fBNOTE:\fP
286 .INDENT 7.0
287 .INDENT 3.5
288 Booting e.g. a RAID1 system may fail even if all filesystem\(aqs \fIdevice\fP
289 paths are provided as the actual device nodes may not be discovered by the
290 system at that point.
291 .UNINDENT
292 .UNINDENT
293 .TP
294 .B discard, discard=sync, discard=async, nodiscard
295 (default: async when devices support it since 6.2, async support since: 5.6)
296 .sp
297 Enable discarding of freed file blocks. This is useful for SSD/NVMe
298 devices, thinly provisioned LUNs, or virtual machine images; however,
299 every storage layer must support discard for it to work.
300 .sp
301 In the synchronous mode (\fIsync\fP or without option value), lack of asynchronous
302 queued TRIM on the backing device TRIM can severely degrade performance,
303 because a synchronous TRIM operation will be attempted instead. Queued TRIM
304 requires SATA devices with chipsets revision newer than 3.1 and devices.
305 .sp
306 The asynchronous mode (\fIasync\fP) gathers extents in larger chunks before sending
307 them to the devices for TRIM. The overhead and performance impact should be
308 negligible compared to the previous mode and it\(aqs supposed to be the preferred
309 mode if needed.
310 .sp
311 If it is not necessary to immediately discard freed blocks, then the \fBfstrim\fP
312 tool can be used to discard all free blocks in a batch. Scheduling a TRIM
313 during a period of low system activity will prevent latent interference with
314 the performance of other operations. Also, a device may ignore the TRIM command
315 if the range is too small, so running a batch discard has a greater probability
316 of actually discarding the blocks.
317 .TP
318 .B enospc_debug, noenospc_debug
319 (default: off)
320 .sp
321 Enable verbose output for some ENOSPC conditions. It\(aqs safe to use but can
322 be noisy if the system reaches near\-full state.
323 .TP
324 .B fatal_errors=<action>
325 (since: 3.4, default: bug)
326 .sp
327 Action to take when encountering a fatal error.
328 .INDENT 7.0
329 .TP
330 .B bug
331 \fIBUG()\fP on a fatal error, the system will stay in the crashed state and may be
332 still partially usable, but reboot is required for full operation
333 .TP
334 .B panic
335 \fIpanic()\fP on a fatal error, depending on other system configuration, this may
336 be followed by a reboot. Please refer to the documentation of kernel boot
337 parameters, e.g. \fIpanic\fP, \fIoops\fP or \fIcrashkernel\fP\&.
338 .UNINDENT
339 .TP
340 .B flushoncommit, noflushoncommit
341 (default: off)
342 .sp
343 This option forces any data dirtied by a write in a prior transaction to commit
344 as part of the current commit, effectively a full filesystem sync.
345 .sp
346 This makes the committed state a fully consistent view of the file system from
347 the application\(aqs perspective (i.e. it includes all completed file system
348 operations). This was previously the behavior only when a snapshot was
349 created.
350 .sp
351 When off, the filesystem is consistent but buffered writes may last more than
352 one transaction commit.
353 .TP
354 .B fragment=<type>
355 (depends on compile\-time option CONFIG_BTRFS_DEBUG, since: 4.4, default: off)
356 .sp
357 A debugging helper to intentionally fragment given \fItype\fP of block groups. The
358 type can be \fIdata\fP, \fImetadata\fP or \fIall\fP\&. This mount option should not be used
359 outside of debugging environments and is not recognized if the kernel config
360 option \fICONFIG_BTRFS_DEBUG\fP is not enabled.
361 .TP
362 .B nologreplay
363 (default: off, even read\-only)
364 .sp
365 The tree\-log contains pending updates to the filesystem until the full commit.
366 The log is replayed on next mount, this can be disabled by this option. See
367 also \fItreelog\fP\&. Note that \fInologreplay\fP is the same as \fInorecovery\fP\&.
368 .sp
369 \fBWARNING:\fP
370 .INDENT 7.0
371 .INDENT 3.5
372 Currently, the tree log is replayed even with a read\-only mount! To
373 disable that behaviour, mount also with \fInologreplay\fP\&.
374 .UNINDENT
375 .UNINDENT
376 .TP
377 .B max_inline=<bytes>
378 (default: min(2048, page size) )
379 .sp
380 Specify the maximum amount of space, that can be inlined in
381 a metadata b\-tree leaf. The value is specified in bytes, optionally
382 with a K suffix (case insensitive). In practice, this value
383 is limited by the filesystem block size (named \fIsectorsize\fP at mkfs time),
384 and memory page size of the system. In case of sectorsize limit, there\(aqs
385 some space unavailable due to b\-tree leaf headers. For example, a 4KiB
386 sectorsize, maximum size of inline data is about 3900 bytes.
387 .sp
388 Inlining can be completely turned off by specifying 0. This will increase data
389 block slack if file sizes are much smaller than block size but will reduce
390 metadata consumption in return.
391 .sp
392 \fBNOTE:\fP
393 .INDENT 7.0
394 .INDENT 3.5
395 The default value has changed to 2048 in kernel 4.6.
396 .UNINDENT
397 .UNINDENT
398 .TP
399 .B metadata_ratio=<value>
400 (default: 0, internal logic)
401 .sp
402 Specifies that 1 metadata chunk should be allocated after every \fIvalue\fP data
403 chunks. Default behaviour depends on internal logic, some percent of unused
404 metadata space is attempted to be maintained but is not always possible if
405 there\(aqs not enough space left for chunk allocation. The option could be useful to
406 override the internal logic in favor of the metadata allocation if the expected
407 workload is supposed to be metadata intense (snapshots, reflinks, xattrs,
408 inlined files).
409 .TP
410 .B norecovery
411 (since: 4.5, default: off)
412 .sp
413 Do not attempt any data recovery at mount time. This will disable \fIlogreplay\fP
414 and avoids other write operations. Note that this option is the same as
415 \fInologreplay\fP\&.
416 .sp
417 \fBNOTE:\fP
418 .INDENT 7.0
419 .INDENT 3.5
420 The opposite option \fIrecovery\fP used to have different meaning but was
421 changed for consistency with other filesystems, where \fInorecovery\fP is used for
422 skipping log replay. BTRFS does the same and in general will try to avoid any
423 write operations.
424 .UNINDENT
425 .UNINDENT
426 .UNINDENT
427 .INDENT 0.0
428 .TP
429 .B rescan_uuid_tree
430 (since: 3.12, default: off)
431 .sp
432 Force check and rebuild procedure of the UUID tree. This should not
433 normally be needed. Alternatively the tree can be cleared from
434 userspace by command \fI\%btrfs rescue clear\-uuid\-tree\fP
435 and then it will be automatically rebuilt in kernel (the mount option
436 is not needed in that case).
437 .TP
438 .B rescue
439 (since: 5.9)
440 .sp
441 Modes allowing mount with damaged filesystem structures, all requires
442 the filesystem to be mounted read\-only and doesn\(aqt allow remount to read\-write.
443 This is supposed to provide unified and more fine grained tuning of
444 errors that affect filesystem operation.
445 .INDENT 7.0
446 .IP \(bu 2
447 \fIusebackuproot\fP (since 5.9)
448 .sp
449 Try to use backup root slots inside super block.
450 Replaces standalone option \fIusebackuproot\fP
451 .IP \(bu 2
452 \fInologreplay\fP (since 5.9)
453 .sp
454 Do not replay any dirty logs.
455 Replaces standalone option \fInologreplay\fP
456 .IP \(bu 2
457 \fIignorebadroots\fP, \fIibadroots\fP (since: 5.11)
458 .sp
459 Ignore bad tree roots, greatly improve the chance for data salvage.
460 .IP \(bu 2
461 \fIignoredatacsums\fP, \fIidatacsums\fP (since: 5.11)
462 .sp
463 Ignore data checksum verification.
464 .IP \(bu 2
465 \fIignoremetacsums\fP, \fIimetacsums\fP (since 6.12)
466 .sp
467 Ignore metadata checksum verification, useful for interrupted checksum conversion.
468 .IP \(bu 2
469 \fIall\fP (since: 5.9)
470 .sp
471 Enable all supported rescue options.
472 .UNINDENT
473 .TP
474 .B skip_balance
475 (since: 3.3, default: off)
476 .sp
477 Skip automatic resume of an interrupted balance operation. The operation can
478 later be resumed with \fBbtrfs balance resume\fP, or the paused state can be
479 removed with \fBbtrfs balance cancel\fP\&. The default behaviour is to resume an
480 interrupted balance immediately after the filesystem is mounted.
481 .TP
482 .B space_cache, space_cache=<version>, nospace_cache
483 (\fInospace_cache\fP since: 3.2, \fIspace_cache=v1\fP and \fIspace_cache=v2\fP since 4.5, default: \fIspace_cache=v2\fP)
484 .sp
485 Options to control the free space cache. The free space cache greatly improves
486 performance when reading block group free space into memory. However, managing
487 the space cache consumes some resources, including a small amount of disk
488 space.
489 .sp
490 There are two implementations of the free space cache. The original
491 one, referred to as \fIv1\fP, used to be a safe default but has been
492 superseded by \fIv2\fP\&. The \fIv1\fP space cache can be disabled at mount time
493 with \fInospace_cache\fP without clearing.
494 .sp
495 On very large filesystems (many terabytes) and certain workloads, the
496 performance of the \fIv1\fP space cache may degrade drastically. The \fIv2\fP
497 implementation, which adds a new b\-tree called the free space tree, addresses
498 this issue. Once enabled, the \fIv2\fP space cache will always be used and cannot
499 be disabled unless it is cleared. Use \fIclear_cache,space_cache=v1\fP or
500 \fIclear_cache,nospace_cache\fP to do so. If \fIv2\fP is enabled, and \fIv1\fP space
501 cache will be cleared (at the first mount) and kernels without \fIv2\fP
502 support will only be able to mount the filesystem in read\-only mode.
503 On an unmounted filesystem the caches (both versions) can be cleared by
504 \(dqbtrfs check \-\-clear\-space\-cache\(dq.
505 .sp
506 The \fI\%btrfs\-check(8)\fP and \fI:doc:\(gamkfs.btrfs\fP commands have full \fIv2\fP free space
507 cache support since v4.19.
508 .sp
509 If a version is not explicitly specified, the default implementation will be
510 chosen, which is \fIv2\fP\&.
511 .TP
512 .B ssd, ssd_spread, nossd, nossd_spread
513 (default: SSD autodetected)
514 .sp
515 Options to control SSD allocation schemes. By default, BTRFS will
516 enable or disable SSD optimizations depending on status of a device with
517 respect to rotational or non\-rotational type. This is determined by the
518 contents of \fB/sys/block/DEV/queue/rotational\fP). If it is 0, the
519 \fIssd\fP option is turned on. The option \fInossd\fP will disable the
520 autodetection.
521 .sp
522 The optimizations make use of the absence of the seek penalty that\(aqs inherent
523 for the rotational devices. The blocks can be typically written faster and
524 are not offloaded to separate threads.
525 .sp
526 \fBNOTE:\fP
527 .INDENT 7.0
528 .INDENT 3.5
529 Since 4.14, the block layout optimizations have been dropped. This used
530 to help with first generations of SSD devices. Their FTL (flash translation
531 layer) was not effective and the optimization was supposed to improve the wear
532 by better aligning blocks. This is no longer true with modern SSD devices and
533 the optimization had no real benefit. Furthermore it caused increased
534 fragmentation. The layout tuning has been kept intact for the option
535 \fIssd_spread\fP\&.
536 .UNINDENT
537 .UNINDENT
538 .sp
539 The \fIssd_spread\fP mount option attempts to allocate into bigger and aligned
540 chunks of unused space, and may perform better on low\-end SSDs. \fIssd_spread\fP
541 implies \fIssd\fP, enabling all other SSD heuristics as well. The option \fInossd\fP
542 will disable all SSD options while \fInossd_spread\fP only disables \fIssd_spread\fP\&.
543 .TP
544 .B subvol=<path>
545 Mount subvolume from \fIpath\fP rather than the toplevel subvolume. The
546 \fIpath\fP is always treated as relative to the toplevel subvolume.
547 This mount option overrides the default subvolume set for the given filesystem.
548 .TP
549 .B subvolid=<subvolid>
550 Mount subvolume specified by a \fIsubvolid\fP number rather than the toplevel
551 subvolume. You can use \fBbtrfs subvolume list\fP of \fBbtrfs subvolume show\fP to see
552 subvolume ID numbers.
553 This mount option overrides the default subvolume set for the given filesystem.
554 .sp
555 \fBNOTE:\fP
556 .INDENT 7.0
557 .INDENT 3.5
558 If both \fIsubvolid\fP and \fIsubvol\fP are specified, they must point at the
559 same subvolume, otherwise the mount will fail.
560 .UNINDENT
561 .UNINDENT
562 .TP
563 .B thread_pool=<number>
564 (default: min(NRCPUS + 2, 8) )
565 .sp
566 The number of worker threads to start. NRCPUS is number of on\-line CPUs
567 detected at the time of mount. Small number leads to less parallelism in
568 processing data and metadata, higher numbers could lead to a performance hit
569 due to increased locking contention, process scheduling, cache\-line bouncing or
570 costly data transfers between local CPU memories.
571 .TP
572 .B treelog, notreelog
573 (default: on)
574 .sp
575 Enable the tree logging used for \fIfsync\fP and \fIO_SYNC\fP writes. The tree log
576 stores changes without the need of a full filesystem sync. The log operations
577 are flushed at sync and transaction commit. If the system crashes between two
578 such syncs, the pending tree log operations are replayed during mount.
579 .sp
580 \fBWARNING:\fP
581 .INDENT 7.0
582 .INDENT 3.5
583 Currently, the tree log is replayed even with a read\-only mount! To
584 disable that behaviour, also mount with \fInologreplay\fP\&.
585 .UNINDENT
586 .UNINDENT
587 .sp
588 The tree log could contain new files/directories, these would not exist on
589 a mounted filesystem if the log is not replayed.
590 .TP
591 .B usebackuproot
592 (since: 4.6, default: off)
593 .sp
594 Enable autorecovery attempts if a bad tree root is found at mount time.
595 Currently this scans a backup list of several previous tree roots and tries to
596 use the first readable. This can be used with read\-only mounts as well.
597 .sp
598 \fBNOTE:\fP
599 .INDENT 7.0
600 .INDENT 3.5
601 This option has replaced \fIrecovery\fP which has been deprecated.
602 .UNINDENT
603 .UNINDENT
604 .TP
605 .B user_subvol_rm_allowed
606 (default: off)
607 .sp
608 Allow subvolumes to be deleted by their respective owner. Otherwise, only the
609 root user can do that.
610 .sp
611 \fBNOTE:\fP
612 .INDENT 7.0
613 .INDENT 3.5
614 Historically, any user could create a snapshot even if he was not owner
615 of the source subvolume, the subvolume deletion has been restricted for that
616 reason. The subvolume creation has been restricted but this mount option is
617 still required. This is a usability issue.
618 Since 4.18, the \X'tty: link https://man7.org/linux/man-pages/man2/rmdir.2.html'\fI\%rmdir(2)\fP\X'tty: link' syscall can delete an empty subvolume just like an
619 ordinary directory. Whether this is possible can be detected at runtime, see
620 \fIrmdir_subvol\fP feature in \fIFILESYSTEM FEATURES\fP\&.
621 .UNINDENT
622 .UNINDENT
623 .UNINDENT
624 .SS DEPRECATED MOUNT OPTIONS
625 .sp
626 List of mount options that have been removed, kept for backward compatibility.
627 .INDENT 0.0
628 .TP
629 .B recovery
630 (since: 3.2, default: off, deprecated since: 4.5)
631 .sp
632 \fBNOTE:\fP
633 .INDENT 7.0
634 .INDENT 3.5
635 This option has been replaced by \fIusebackuproot\fP and should not be used
636 but will work on 4.5+ kernels.
637 .UNINDENT
638 .UNINDENT
639 .TP
640 .B inode_cache, noinode_cache
641 (removed in: 5.11, since: 3.0, default: off)
642 .sp
643 \fBNOTE:\fP
644 .INDENT 7.0
645 .INDENT 3.5
646 The functionality has been removed in 5.11, any stale data created by
647 previous use of the \fIinode_cache\fP option can be removed by
648 \fI\%btrfs rescue clear\-ino\-cache\fP\&.
649 .UNINDENT
650 .UNINDENT
651 .TP
652 .B check_int, check_int_data, check_int_print_mask=<value>
653 (removed in: 6.7, since: 3.0, default: off)
654 .sp
655 These debugging options control the behavior of the integrity checking
656 module (the BTRFS_FS_CHECK_INTEGRITY config option required). The main goal is
657 to verify that all blocks from a given transaction period are properly linked.
658 .sp
659 \fIcheck_int\fP enables the integrity checker module, which examines all
660 block write requests to ensure on\-disk consistency, at a large
661 memory and CPU cost.
662 .sp
663 \fIcheck_int_data\fP includes extent data in the integrity checks, and
664 implies the \fIcheck_int\fP option.
665 .sp
666 \fIcheck_int_print_mask\fP takes a bit mask of BTRFSIC_PRINT_MASK_* values
667 as defined in \fIfs/btrfs/check\-integrity.c\fP, to control the integrity
668 checker module behavior.
669 .sp
670 See comments at the top of \fIfs/btrfs/check\-integrity.c\fP
671 for more information.
672 .UNINDENT
673 .SS NOTES ON GENERIC MOUNT OPTIONS
674 .sp
675 Some of the general mount options from \X'tty: link https://man7.org/linux/man-pages/man8/mount.8.html'\fI\%mount(8)\fP\X'tty: link' that affect BTRFS and are
676 worth mentioning.
677 .INDENT 0.0
678 .TP
679 .B context
680 The context refers to the SELinux contexts and policy definitions passed
681 as mount options. This works properly since version v6.8 (because the
682 mount option parser of BTRFS was ported to new API that also understood
683 the options).
684 .TP
685 .B noatime
686 under read intensive work\-loads, specifying \fInoatime\fP significantly improves
687 performance because no new access time information needs to be written. Without
688 this option, the default is \fIrelatime\fP, which only reduces the number of
689 inode atime updates in comparison to the traditional \fIstrictatime\fP\&. The worst
690 case for atime updates under \fIrelatime\fP occurs when many files are read whose
691 atime is older than 24 h and which are freshly snapshotted. In that case the
692 atime is updated and COW happens \- for each file \- in bulk. See also
693 \X'tty: link https://lwn.net/Articles/499293/'\fI\%https://lwn.net/Articles/499293/\fP\X'tty: link' \- \fIAtime and btrfs: a bad combination? (LWN, 2012\-05\-31)\fP\&.
694 .sp
695 Note that \fInoatime\fP may break applications that rely on atime uptimes like
696 the venerable Mutt (unless you use \fImaildir\fP mailboxes).
697 .UNINDENT
698 .SH FILESYSTEM FEATURES
699 .sp
700 The basic set of filesystem features gets extended over time. The backward
701 compatibility is maintained and the features are optional, need to be
702 explicitly asked for so accidental use will not create incompatibilities.
703 .sp
704 There are several classes and the respective tools to manage the features:
705 .INDENT 0.0
706 .TP
707 .B at mkfs time only
708 This is namely for core structures, like the b\-tree nodesize or checksum
709 algorithm, see \fI\%mkfs.btrfs(8)\fP for more details.
710 .TP
711 .B after mkfs, on an unmounted filesystem
712 Features that may optimize internal structures or add new structures to support
713 new functionality, see \fI\%btrfstune(8)\fP\&. The command
714 \fBbtrfs inspect\-internal dump\-super /dev/sdx\fP
715 will dump a superblock, you can map the value of
716 \fIincompat_flags\fP to the features listed below
717 .TP
718 .B after mkfs, on a mounted filesystem
719 The features of a filesystem (with a given UUID) are listed in
720 \fB/sys/fs/btrfs/UUID/features/\fP, one file per feature. The status is stored
721 inside the file. The value \fI1\fP is for enabled and active, while \fI0\fP means the
722 feature was enabled at mount time but turned off afterwards.
723 .sp
724 Whether a particular feature can be turned on a mounted filesystem can be found
725 in the directory \fB/sys/fs/btrfs/features/\fP, one file per feature. The value \fI1\fP
726 means the feature can be enabled.
727 .UNINDENT
728 .sp
729 List of features (see also \fI\%mkfs.btrfs(8)\fP section
730 \fI\%FILESYSTEM FEATURES\fP):
731 .INDENT 0.0
732 .TP
733 .B big_metadata
734 (since: 3.4)
735 .sp
736 the filesystem uses \fInodesize\fP for metadata blocks, this can be bigger than the
737 page size
738 .TP
739 .B block_group_tree
740 (since: 6.1)
741 .sp
742 block group item representation using a dedicated b\-tree, this can greatly
743 reduce mount time for large filesystems
744 .TP
745 .B compress_lzo
746 (since: 2.6.38)
747 .sp
748 the \fIlzo\fP compression has been used on the filesystem, either as a mount option
749 or via \fBbtrfs filesystem defrag\fP\&.
750 .TP
751 .B compress_zstd
752 (since: 4.14)
753 .sp
754 the \fIzstd\fP compression has been used on the filesystem, either as a mount option
755 or via \fBbtrfs filesystem defrag\fP\&.
756 .TP
757 .B default_subvol
758 (since: 2.6.34)
759 .sp
760 the default subvolume has been set on the filesystem
761 .TP
762 .B extended_iref
763 (since: 3.7)
764 .sp
765 increased hardlink limit per file in a directory to 65536, older kernels
766 supported a varying number of hardlinks depending on the sum of all file name
767 sizes that can be stored into one metadata block
768 .TP
769 .B free_space_tree
770 (since: 4.5)
771 .sp
772 free space representation using a dedicated b\-tree, successor of v1 space cache
773 .TP
774 .B metadata_uuid
775 (since: 5.0)
776 .sp
777 the main filesystem UUID is the metadata_uuid, which stores the new UUID only
778 in the superblock while all metadata blocks still have the UUID set at mkfs
779 time, see \fI\%btrfstune(8)\fP for more
780 .TP
781 .B mixed_backref
782 (since: 2.6.31)
783 .sp
784 the last major disk format change, improved backreferences, now default
785 .TP
786 .B mixed_groups
787 (since: 2.6.37)
788 .sp
789 mixed data and metadata block groups, i.e. the data and metadata are not
790 separated and occupy the same block groups, this mode is suitable for small
791 volumes as there are no constraints how the remaining space should be used
792 (compared to the split mode, where empty metadata space cannot be used for data
793 and vice versa)
794 .sp
795 on the other hand, the final layout is quite unpredictable and possibly highly
796 fragmented, which means worse performance
797 .TP
798 .B no_holes
799 (since: 3.14)
800 .sp
801 improved representation of file extents where holes are not explicitly
802 stored as an extent, saves a few percent of metadata if sparse files are used
803 .TP
804 .B raid1c34
805 (since: 5.5)
806 .sp
807 extended RAID1 mode with copies on 3 or 4 devices respectively
808 .TP
809 .B raid_stripe_tree
810 (since: 6.7)
811 .sp
812 a separate tree for tracking file extents on RAID profiles
813 .TP
814 .B RAID56
815 (since: 3.9)
816 .sp
817 the filesystem contains or contained a RAID56 profile of block groups
818 .TP
819 .B rmdir_subvol
820 (since: 4.18)
821 .sp
822 indicate that \X'tty: link https://man7.org/linux/man-pages/man2/rmdir.2.html'\fI\%rmdir(2)\fP\X'tty: link' syscall can delete an empty subvolume just like an
823 ordinary directory. Note that this feature only depends on the kernel version.
824 .TP
825 .B skinny_metadata
826 (since: 3.10)
827 .sp
828 reduced\-size metadata for extent references, saves a few percent of metadata
829 .TP
830 .B send_stream_version
831 (since: 5.10)
832 .sp
833 number of the highest supported send stream version
834 .TP
835 .B simple_quota
836 (since: 6.7)
837 .sp
838 simplified quota accounting
839 .TP
840 .B supported_checksums
841 (since: 5.5)
842 .sp
843 list of checksum algorithms supported by the kernel module, the respective
844 modules or built\-in implementing the algorithms need to be present to mount
845 the filesystem, see section \fI\%CHECKSUM ALGORITHMS\fP\&.
846 .TP
847 .B supported_sectorsizes
848 (since: 5.13)
849 .sp
850 list of values that are accepted as sector sizes (\fBmkfs.btrfs \-\-sectorsize\fP) by
851 the running kernel
852 .TP
853 .B supported_rescue_options
854 (since: 5.11)
855 .sp
856 list of values for the mount option \fIrescue\fP that are supported by the running
857 kernel, see \fI\%btrfs(5)\fP
858 .TP
859 .B zoned
860 (since: 5.12)
861 .sp
862 zoned mode is allocation/write friendly to host\-managed zoned devices,
863 allocation space is partitioned into fixed\-size zones that must be updated
864 sequentially, see section \fI\%ZONED MODE\fP
865 .UNINDENT
866 .SH SWAPFILE SUPPORT
867 .sp
868 A swapfile, when active, is a file\-backed swap area. It is supported since kernel 5.0.
869 Use \X'tty: link https://man7.org/linux/man-pages/man8/swapon.8.html'\fI\%swapon(8)\fP\X'tty: link' to activate it, until then (respectively again after deactivating it
870 with \X'tty: link https://man7.org/linux/man-pages/man8/swapoff.8.html'\fI\%swapoff(8)\fP\X'tty: link') it\(aqs just a normal file (with NODATACOW set), for which the special
871 restrictions for active swapfiles don\(aqt apply.
872 .sp
873 There are some limitations of the implementation in BTRFS and Linux swap
874 subsystem:
875 .INDENT 0.0
876 .IP \(bu 2
877 filesystem \- must be only single device
878 .IP \(bu 2
879 filesystem \- must have only \fIsingle\fP data profile
880 .IP \(bu 2
881 subvolume \- cannot be snapshotted if it contains any active swapfiles
882 .IP \(bu 2
883 swapfile \- must be preallocated (i.e. no holes)
884 .IP \(bu 2
885 swapfile \- must be NODATACOW (i.e. also NODATASUM, no compression)
886 .UNINDENT
887 .sp
888 The limitations come namely from the COW\-based design and mapping layer of
889 blocks that allows the advanced features like relocation and multi\-device
890 filesystems. However, the swap subsystem expects simpler mapping and no
891 background changes of the file block location once they\(aqve been assigned to
892 swap. The constraints mentioned above (single device and single profile) are
893 related to the swapfile itself, i.e. the extents and their placement. It is
894 possible to create swapfile on multi\-device filesystem as long as the extents
895 are on one device but this cannot be affected by user and depends on free space
896 fragmentation and available unused space for new chunks.
897 .sp
898 With active swapfiles, the following whole\-filesystem operations will skip
899 some block groups or may fail:
900 .INDENT 0.0
901 .IP \(bu 2
902 balance \- block groups with extents of any active swapfiles are skipped and
903 reported through dmesg, the rest will be processed normally
904 .IP \(bu 2
905 scrub \- ditto
906 .IP \(bu 2
907 resize grow \- unaffected
908 .IP \(bu 2
909 resize shrink \- works as long as the extents of any active swapfiles are
910 outside of the shrunk range
911 .IP \(bu 2
912 device add \- if the new devices do not interfere with any already active swapfiles
913 this operation will work, though no new swapfile can be activated
914 afterwards
915 .IP \(bu 2
916 device delete \- if the device has been added as above, it can be also deleted
917 .IP \(bu 2
918 device replace \- ditto
919 .UNINDENT
920 .sp
921 The above limitations will significantly affect btrfs maintenance routines,
922 namely balance and scrub. Balance and scrub will skip those block groups which
923 have extents of an active swapfile on them, and those block groups may also
924 contain extents belonging to other files. This means swapfiles will have an
925 amplified impact, much larger than the size of the swapfile itself.
926 .sp
927 For balance, it means data block groups may not be properly balanced, causing
928 unbalanced data/metadata usage, which can lead to early ENOSPC errors.
929 .sp
930 For scrub, it means data block groups may not be verified, degrading the
931 integrity of data.
932 .sp
933 Users should be aware of the impact when using swapfiles alongside other data.
934 This is especially discouraged for root filesystem.
935 .sp
936 When there are no active swapfiles and a whole\-filesystem exclusive operation
937 is running (e.g. balance, device delete, shrink), the swapfiles cannot be
938 temporarily activated. The operation must finish first.
939 .sp
940 To create and activate a swapfile run the following commands:
941 .INDENT 0.0
942 .INDENT 3.5
943 .sp
944 .EX
945 # truncate \-s 0 swapfile
946 # chattr +C swapfile
947 # fallocate \-l 2G swapfile
948 # chmod 0600 swapfile
949 # mkswap swapfile
950 # swapon swapfile
951 .EE
952 .UNINDENT
953 .UNINDENT
954 .sp
955 Since version 6.1 it\(aqs possible to create the swapfile in a single command
956 (except the activation):
957 .INDENT 0.0
958 .INDENT 3.5
959 .sp
960 .EX
961 # btrfs filesystem mkswapfile \-\-size 2G swapfile
962 # swapon swapfile
963 .EE
964 .UNINDENT
965 .UNINDENT
966 .sp
967 Please note that the UUID returned by the \fImkswap\fP utility identifies the swap
968 \(dqfilesystem\(dq and because it\(aqs stored in a file, it\(aqs not generally visible and
969 usable as an identifier unlike if it was on a block device.
970 .sp
971 Once activated the file will appear in \fB/proc/swaps\fP:
972 .INDENT 0.0
973 .INDENT 3.5
974 .sp
975 .EX
976 # cat /proc/swaps
977 Filename Type Size Used Priority
978 /path/swapfile file 2097152 0 \-2
979 .EE
980 .UNINDENT
981 .UNINDENT
982 .sp
983 The swapfile can be created as one\-time operation or, once properly created,
984 activated on each boot by the \fBswapon \-a\fP command (usually started by the
985 service manager). Add the following entry to \fI/etc/fstab\fP, assuming the
986 filesystem that provides the \fI/path\fP has been already mounted at this point.
987 Additional mount options relevant for the swapfile can be set too (like
988 priority, not the BTRFS mount options).
989 .INDENT 0.0
990 .INDENT 3.5
991 .sp
992 .EX
993 /path/swapfile none swap defaults 0 0
994 .EE
995 .UNINDENT
996 .UNINDENT
997 .sp
998 From now on the subvolume with the active swapfile cannot be snapshotted until
999 the swapfile is deactivated again by \fBswapoff\fP\&. Then the swapfile is a
1000 regular file and the subvolume can be snapshotted again, though this would prevent
1001 another activation any swapfile that has been snapshotted. New swapfiles (not
1002 snapshotted) can be created and activated.
1003 .sp
1004 Otherwise, an inactive swapfile does not affect the containing subvolume. Activation
1005 creates a temporary in\-memory status and prevents some file operations, but is
1006 not stored permanently.
1007 .SH HIBERNATION
1008 .sp
1009 A swapfile can be used for hibernation but it\(aqs not straightforward. Before
1010 hibernation a resume offset must be written to file \fI/sys/power/resume_offset\fP
1011 or the kernel command line parameter \fIresume_offset\fP must be set.
1012 .sp
1013 The value is the physical offset on the device. Note that \fBthis is not the same
1014 value that\fP \fBfilefrag\fP \fBprints as physical offset!\fP
1015 .sp
1016 Btrfs filesystem uses mapping between logical and physical addresses but here
1017 the physical can still map to one or more device\-specific physical block
1018 addresses. It\(aqs the device\-specific physical offset that is suitable as resume
1019 offset.
1020 .sp
1021 Since version 6.1 there\(aqs a command \fI\%btrfs inspect\-internal map\-swapfile\fP
1022 that will print the device physical offset and the adjusted value for
1023 \fB/sys/power/resume_offset\fP\&. Note that the value is divided by page size, i.e.
1024 it\(aqs not the offset itself.
1025 .INDENT 0.0
1026 .INDENT 3.5
1027 .sp
1028 .EX
1029 # btrfs filesystem mkswapfile swapfile
1030 # btrfs inspect\-internal map\-swapfile swapfile
1031 Physical start: 811511726080
1032 Resume offset: 198122980
1033 .EE
1034 .UNINDENT
1035 .UNINDENT
1036 .sp
1037 For scripting and convenience the option \fI\-r\fP will print just the offset:
1038 .INDENT 0.0
1039 .INDENT 3.5
1040 .sp
1041 .EX
1042 # btrfs inspect\-internal map\-swapfile \-r swapfile
1043 198122980
1044 .EE
1045 .UNINDENT
1046 .UNINDENT
1047 .sp
1048 The command \fBmap\-swapfile\fP also verifies all the requirements, i.e. no holes,
1049 single device, etc.
1050 .SH TROUBLESHOOTING
1051 .sp
1052 If the swapfile activation fails please verify that you followed all the steps
1053 above or check the system log (e.g. \fBdmesg\fP or \fBjournalctl\fP) for more
1054 information.
1055 .sp
1056 Notably, the \fBswapon\fP utility exits with a message that does not say what
1057 failed:
1058 .INDENT 0.0
1059 .INDENT 3.5
1060 .sp
1061 .EX
1062 # swapon /path/swapfile
1063 swapon: /path/swapfile: swapon failed: Invalid argument
1064 .EE
1065 .UNINDENT
1066 .UNINDENT
1067 .sp
1068 The specific reason is likely to be printed to the system log by the btrfs
1069 module:
1070 .INDENT 0.0
1071 .INDENT 3.5
1072 .sp
1073 .EX
1074 # journalctl \-t kernel | grep swapfile
1075 kernel: BTRFS warning (device sda): swapfile must have single data profile
1076 .EE
1077 .UNINDENT
1078 .UNINDENT
1079 .SH CHECKSUM ALGORITHMS
1080 .sp
1081 Data and metadata are checksummed by default. The checksum is calculated before
1082 writing and verified after reading the blocks from devices. The whole metadata
1083 block has an inline checksum stored in the b\-tree node header. Each data block
1084 has a detached checksum stored in the checksum tree.
1085 .sp
1086 \fBNOTE:\fP
1087 .INDENT 0.0
1088 .INDENT 3.5
1089 Since a data checksum is calculated just before submitting to the block
1090 device, btrfs has a strong requirement that the corresponding data block must
1091 not be modified until the writeback is finished.
1092 .sp
1093 This requirement is met for a buffered write as btrfs has the full control on
1094 its page cache, but a direct write (\fBO_DIRECT\fP) bypasses page cache, and
1095 btrfs can not control the direct IO buffer (as it can be in user space memory).
1096 Thus it\(aqs possible that a user space program modifies its direct write buffer
1097 before the buffer is fully written back, and this can lead to a data
1098 checksum mismatch.
1099 .sp
1100 To avoid this, kernel starting with version 6.14 will force a direct
1101 write to fall back to buffered, if the inode requires a data checksum.
1102 This will bring a small performance penalty. If you require true zero\-copy
1103 direct writes, then set the \fBNODATASUM\fP flag for the inode and make
1104 sure the direct IO buffer is fully aligned to block size.
1105 .UNINDENT
1106 .UNINDENT
1107 .sp
1108 There are several checksum algorithms supported. The default and backward
1109 compatible algorithm is \fIcrc32c\fP\&. Since kernel 5.5 there are three more with different
1110 characteristics and trade\-offs regarding speed and strength. The following list
1111 may help you to decide which one to select.
1112 .INDENT 0.0
1113 .TP
1114 .B CRC32C (32 bits digest)
1115 Default, best backward compatibility. Very fast, modern CPUs have
1116 instruction\-level support, not collision\-resistant but still good error
1117 detection capabilities.
1118 .TP
1119 .B XXHASH (64 bits digest)
1120 Can be used as CRC32C successor. Very fast, optimized for modern CPUs utilizing
1121 instruction pipelining, good collision resistance and error detection.
1122 .TP
1123 .B SHA256 (256 bits digest)
1124 Cryptographic\-strength hash. Relatively slow but with possible CPU
1125 instruction acceleration or specialized hardware cards. FIPS certified and
1126 in wide use.
1127 .TP
1128 .B BLAKE2b (256 bits digest)
1129 Cryptographic\-strength hash. Relatively fast, with possible CPU acceleration
1130 using SIMD extensions. Not standardized but based on BLAKE which was a SHA3
1131 finalist, in wide use. The algorithm used is BLAKE2b\-256 that\(aqs optimized for
1132 64\-bit platforms.
1133 .UNINDENT
1134 .sp
1135 The \fIdigest size\fP affects overall size of data block checksums stored in the
1136 filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so
1137 there\(aqs no increase. Each data block has a separate checksum stored, with
1138 additional overhead of the b\-tree leaves.
1139 .sp
1140 Approximate relative performance of the algorithms, measured against CRC32C
1141 using implementations on a 11th gen 3.6GHz intel CPU:
1142 .TS
1143 box center;
1144 l|l|l|l.
1145 T{
1146 Digest
1147 T} T{
1148 Cycles/4KiB
1149 T} T{
1150 Ratio
1151 T} T{
1152 Implementation
1153 T}
1154 _
1155 T{
1156 CRC32C
1157 T} T{
1158 470
1159 T} T{
1160 1.00
1161 T} T{
1162 CPU instruction, PCL combination
1163 T}
1164 _
1165 T{
1166 XXHASH
1167 T} T{
1168 870
1169 T} T{
1170 1.9
1171 T} T{
1172 reference impl.
1173 T}
1174 _
1175 T{
1176 SHA256
1177 T} T{
1178 7600
1179 T} T{
1180 16
1181 T} T{
1182 libgcrypt
1183 T}
1184 _
1185 T{
1186 SHA256
1187 T} T{
1188 8500
1189 T} T{
1190 18
1191 T} T{
1192 openssl
1193 T}
1194 _
1195 T{
1196 SHA256
1197 T} T{
1198 8700
1199 T} T{
1200 18
1201 T} T{
1202 botan
1203 T}
1204 _
1205 T{
1206 SHA256
1207 T} T{
1208 32000
1209 T} T{
1210 68
1211 T} T{
1212 builtin, CPU instruction
1213 T}
1214 _
1215 T{
1216 SHA256
1217 T} T{
1218 37000
1219 T} T{
1220 78
1221 T} T{
1222 libsodium
1223 T}
1224 _
1225 T{
1226 SHA256
1227 T} T{
1228 78000
1229 T} T{
1230 166
1231 T} T{
1232 builtin, reference impl.
1233 T}
1234 _
1235 T{
1236 BLAKE2b
1237 T} T{
1238 10000
1239 T} T{
1240 21
1241 T} T{
1242 builtin/AVX2
1243 T}
1244 _
1245 T{
1246 BLAKE2b
1247 T} T{
1248 10900
1249 T} T{
1250 23
1251 T} T{
1252 libgcrypt
1253 T}
1254 _
1255 T{
1256 BLAKE2b
1257 T} T{
1258 13500
1259 T} T{
1260 29
1261 T} T{
1262 builtin/SSE41
1263 T}
1264 _
1265 T{
1266 BLAKE2b
1267 T} T{
1268 13700
1269 T} T{
1270 29
1271 T} T{
1272 libsodium
1273 T}
1274 _
1275 T{
1276 BLAKE2b
1277 T} T{
1278 14100
1279 T} T{
1280 30
1281 T} T{
1282 openssl
1283 T}
1284 _
1285 T{
1286 BLAKE2b
1287 T} T{
1288 14500
1289 T} T{
1290 31
1291 T} T{
1292 kcapi
1293 T}
1294 _
1295 T{
1296 BLAKE2b
1297 T} T{
1298 14500
1299 T} T{
1300 34
1301 T} T{
1302 builtin, reference impl.
1303 T}
1304 .TE
1305 .sp
1306 Many kernels are configured with SHA256 as built\-in and not as a module.
1307 Up to kernel v6.15 the accelerated versions are however provided by the
1308 modules and must be loaded
1309 explicitly (\fBmodprobe sha256\fP) before mounting the filesystem to make use of
1310 them. You can check in \fB/sys/fs/btrfs/FSID/checksum\fP which one is used. If you
1311 see \fIsha256\-generic\fP, then you may want to unmount and mount the filesystem
1312 again. Changing that on a mounted filesystem is not possible.
1313 .sp
1314 Since kernel v6.16 the accelerated implementation is always used if available.
1315 .sp
1316 Check the file \fB/proc/crypto\fP, when the implementation is built\-in, you\(aqd find:
1317 .INDENT 0.0
1318 .INDENT 3.5
1319 .sp
1320 .EX
1321 name : sha256
1322 driver : sha256\-generic
1323 module : kernel
1324 priority : 100
1325 \&...
1326 .EE
1327 .UNINDENT
1328 .UNINDENT
1329 .sp
1330 While accelerated implementation is e.g.:
1331 .INDENT 0.0
1332 .INDENT 3.5
1333 .sp
1334 .EX
1335 name : sha256
1336 driver : sha256\-avx2
1337 module : sha256_ssse3
1338 priority : 170
1339 \&...
1340 .EE
1341 .UNINDENT
1342 .UNINDENT
1343 .SH COMPRESSION
1344 .sp
1345 Btrfs supports transparent file compression. There are three algorithms
1346 available: ZLIB, LZO and ZSTD (since v4.14), with various levels.
1347 The compression happens on the level of file extents and the algorithm is
1348 selected by file property, mount option or by a defrag command.
1349 You can have a single btrfs mount point that has some files that are
1350 uncompressed, some that are compressed with LZO, some with ZLIB, for instance
1351 (though you may not want it that way, it is supported).
1352 .sp
1353 Once the compression is set, all newly written data will be compressed, i.e.
1354 existing data are untouched. Data are split into smaller chunks (128KiB) before
1355 compression to make random rewrites possible without a high performance hit. Due
1356 to the increased number of extents the metadata consumption is higher. The
1357 chunks are compressed in parallel.
1358 .sp
1359 The algorithms can be characterized as follows regarding the speed/ratio
1360 trade\-offs:
1361 .INDENT 0.0
1362 .TP
1363 .B ZLIB
1364 .INDENT 7.0
1365 .IP \(bu 2
1366 slower, higher compression ratio
1367 .IP \(bu 2
1368 levels: 1 to 9, mapped directly, default level is 3
1369 .IP \(bu 2
1370 good backward compatibility
1371 .UNINDENT
1372 .TP
1373 .B LZO
1374 .INDENT 7.0
1375 .IP \(bu 2
1376 faster compression and decompression than ZLIB, worse compression ratio, designed to be fast
1377 .IP \(bu 2
1378 no levels
1379 .IP \(bu 2
1380 good backward compatibility
1381 .UNINDENT
1382 .TP
1383 .B ZSTD
1384 .INDENT 7.0
1385 .IP \(bu 2
1386 compression comparable to ZLIB with higher compression/decompression speeds and different ratio
1387 .IP \(bu 2
1388 levels: \-15..15, mapped directly, default is 3
1389 .IP \(bu 2
1390 support since 4.14
1391 .IP \(bu 2
1392 levels 1..15 supported since 5.1
1393 .IP \(bu 2
1394 levels \-15..\-1 supported since 6.15
1395 .UNINDENT
1396 .UNINDENT
1397 .sp
1398 The differences depend on the actual data set and cannot be expressed by a
1399 single number or recommendation. Higher levels consume more CPU time and may
1400 not bring a significant improvement, lower levels are close to real time.
1401 .SH HOW TO ENABLE COMPRESSION
1402 .sp
1403 Typically the compression can be enabled on the whole filesystem, specified for
1404 the mount point. Note that the compression mount options are shared among all
1405 mounts of the same filesystem, either bind mounts or subvolume mounts.
1406 Please refer to \fI\%btrfs(5)\fP section
1407 \fI\%MOUNT OPTIONS\fP\&.
1408 .INDENT 0.0
1409 .INDENT 3.5
1410 .sp
1411 .EX
1412 $ mount \-o compress=zstd /dev/sdx /mnt
1413 .EE
1414 .UNINDENT
1415 .UNINDENT
1416 .sp
1417 This will enable the \fBzstd\fP algorithm on the default level (which is 3).
1418 The level can be specified manually too like \fBzstd:3\fP\&. Higher levels compress
1419 better at the cost of time. This in turn may cause increased write latency, low
1420 levels are suitable for real\-time compression and on reasonably fast CPU don\(aqt
1421 cause noticeable performance drops.
1422 .INDENT 0.0
1423 .INDENT 3.5
1424 .sp
1425 .EX
1426 $ btrfs filesystem defrag \-czstd file
1427 .EE
1428 .UNINDENT
1429 .UNINDENT
1430 .sp
1431 The command above will start defragmentation of the whole \fIfile\fP and apply
1432 the compression, regardless of the mount option. The compression level can be
1433 also specified with the \fB\-\-level\fP or \fB\-L\fP argument as of version \fB6.14\fP\&.
1434 The compression algorithm is not persistent and applies only
1435 to the defragmentation command, for any other writes other compression settings
1436 apply.
1437 .sp
1438 Persistent settings on a per\-file basis can be set in two ways:
1439 .INDENT 0.0
1440 .INDENT 3.5
1441 .sp
1442 .EX
1443 $ chattr +c file
1444 $ btrfs property set file compression zstd
1445 .EE
1446 .UNINDENT
1447 .UNINDENT
1448 .sp
1449 The first command is using legacy interface of file attributes inherited from
1450 ext2 filesystem and is not flexible, so by default the \fIzlib\fP compression is
1451 set. The other command sets a property on the file with the given algorithm.
1452 (Note: setting level that way is not yet implemented.)
1453 .SH COMPRESSION LEVELS
1454 .sp
1455 The level support of ZLIB has been added in v4.14, LZO does not support levels
1456 (the kernel implementation provides only one), ZSTD level support has been added
1457 in v5.1 and the negative levels in v6.15.
1458 .sp
1459 There are 9 levels of ZLIB supported (1 to 9), mapping 1:1 from the mount option
1460 to the algorithm defined level. The default is level 3, which provides the
1461 reasonably good compression ratio and is still reasonably fast. The difference
1462 in compression gain of levels 7, 8 and 9 is comparable but the higher levels
1463 take longer.
1464 .sp
1465 The ZSTD support includes levels \-15..15, a subset of full range of what ZSTD
1466 provides. Levels \-15..\-1 are real\-time with worse compression ratio, levels
1467 1..3 are near real\-time with good compression, 4..8 are slower with improved
1468 compression and 9..15 try even harder though the resulting size may not be
1469 significantly improved. Higher levels also require more memory and as they need
1470 more CPU the system performance is affected.
1471 .sp
1472 Level 0 always maps to the default. The compression level does not affect
1473 compatibility.
1474 .SH EXCEPTIONS
1475 .sp
1476 Any file that has been touched by the \fIfallocate\fP system call will always be
1477 excepted from compression even if \fIforce\-compress\fP mount option is used.
1478 .sp
1479 The reason for this is that a successful \fIfallocate\fP call must guarantee that
1480 future writes to the allocated range will not fail because of lack of space.
1481 This is difficult to guarantee in a COW filesystem. To reduce the chances of
1482 it happening, btrfs preallocates space and disables compression for the file.
1483 .sp
1484 As a workaround, one can trigger a compressed rewrite for such a file using the
1485 \fIbtrfs defrag\fP command. Be aware that if the file is touched again by the
1486 \fIfallocate\fP system call, it will be excepted again from compression for all the
1487 new data written to it.
1488 .SH INCOMPRESSIBLE DATA
1489 .sp
1490 Files with already compressed data or with data that won\(aqt compress well with
1491 the CPU and memory constraints of the kernel implementations are using a simple
1492 decision logic. If the first portion of data being compressed is not smaller
1493 than the original, the compression of the whole file is disabled. Unless the
1494 filesystem is mounted with \fIcompress\-force\fP in which case btrfs will try
1495 compressing every block, falling back to storing the uncompressed version for
1496 each block that ends up larger after compression. This is not optimal and
1497 subject to optimizations and further development.
1498 .sp
1499 If a file is identified as incompressible, a flag is set (\fINOCOMPRESS\fP) and it\(aqs
1500 sticky. On that file compression won\(aqt be performed unless forced. The flag
1501 can be also set by \fBchattr +m\fP (since e2fsprogs 1.46.2) or by properties with
1502 value \fIno\fP or \fInone\fP\&. Empty value will reset it to the default that\(aqs currently
1503 applicable on the mounted filesystem.
1504 .sp
1505 There are two ways to detect incompressible data:
1506 .INDENT 0.0
1507 .IP \(bu 2
1508 actual compression attempt \- data are compressed, if the result is not smaller,
1509 it\(aqs discarded, so this depends on the algorithm and level
1510 .IP \(bu 2
1511 pre\-compression heuristics \- a quick statistical evaluation on the data is
1512 performed and based on the result either compression is performed or skipped,
1513 the NOCOMPRESS bit is not set just by the heuristic, only if the compression
1514 algorithm does not make an improvement
1515 .UNINDENT
1516 .INDENT 0.0
1517 .INDENT 3.5
1518 .sp
1519 .EX
1520 $ lsattr file
1521 \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-m file
1522 .EE
1523 .UNINDENT
1524 .UNINDENT
1525 .sp
1526 Using the forcing compression is not recommended, the heuristics are
1527 supposed to decide that and compression algorithms internally detect
1528 incompressible data too.
1529 .SH PRE-COMPRESSION HEURISTICS
1530 .sp
1531 The heuristics aim to do a few quick statistical tests on the compressed data
1532 in order to avoid probably costly compression that would turn out to be
1533 inefficient. Compression algorithms could have internal detection of
1534 incompressible data too but this leads to more overhead as the compression is
1535 done in another thread and has to write the data anyway. The heuristic is
1536 read\-only and can utilize cached memory.
1537 .sp
1538 The tests performed based on the following: data sampling, long repeated
1539 pattern detection, byte frequency, Shannon entropy.
1540 .SH COMPATIBILITY
1541 .sp
1542 Compression requires both data checksums and COW, so either \fInodatasum\fP or
1543 \fInodatasum\fP mount option/inode flag will result in no compression.
1544 .sp
1545 Direct IO reads of compressed data will always fallback to buffered reads.
1546 .sp
1547 Direct IO write behavior depends on the inode flag.
1548 For inodes with data checksum, direct IO writes always fallback to buffered
1549 writes, thus can generate compressed data if the mount option/inode flags
1550 allows that.
1551 .sp
1552 For inodes without data checksums, direct IO writes will not populate page cache,
1553 and since the inode has no data checksums, no compressed data will be generated
1554 anyway.
1555 .sp
1556 The compression algorithms have been added over time so the version
1557 compatibility should be also considered, together with other tools that may
1558 access the compressed data like bootloaders.
1559 .SH SYSFS INTERFACE
1560 .sp
1561 Btrfs has a sysfs interface to provide extra knobs.
1562 .sp
1563 The top level path is \fB/sys/fs/btrfs/\fP, and the main directory layout is the following:
1564 .TS
1565 box center;
1566 l|l|l.
1567 T{
1568 Relative Path
1569 T} T{
1570 Description
1571 T} T{
1572 Version
1573 T}
1574 _
1575 T{
1576 features/
1577 T} T{
1578 All supported features
1579 T} T{
1580 3.14
1581 T}
1582 _
1583 T{
1584 <UUID>/
1585 T} T{
1586 Mounted fs UUID
1587 T} T{
1588 3.14
1589 T}
1590 _
1591 T{
1592 <UUID>/allocation/
1593 T} T{
1594 Space allocation info
1595 T} T{
1596 3.14
1597 T}
1598 _
1599 T{
1600 <UUID>/bdi/
1601 T} T{
1602 Backing device info (writeback)
1603 T} T{
1604 5.9
1605 T}
1606 _
1607 T{
1608 <UUID>/devices/<DEVID>/
1609 T} T{
1610 Symlink to each block device sysfs
1611 T} T{
1612 5.6
1613 T}
1614 _
1615 T{
1616 <UUID>/devinfo/<DEVID>/
1617 T} T{
1618 Btrfs specific info for each device
1619 T} T{
1620 5.6
1621 T}
1622 _
1623 T{
1624 <UUID>/discard/
1625 T} T{
1626 Discard stats and tunables
1627 T} T{
1628 6.1
1629 T}
1630 _
1631 T{
1632 <UUID>/features/
1633 T} T{
1634 Features of the filesystem
1635 T} T{
1636 3.14
1637 T}
1638 _
1639 T{
1640 <UUID>/qgroups/
1641 T} T{
1642 Global qgroup info
1643 T} T{
1644 5.9
1645 T}
1646 _
1647 T{
1648 <UUID>/qgroups/<LEVEL>_<ID>/
1649 T} T{
1650 Info for each qgroup
1651 T} T{
1652 5.9
1653 T}
1654 .TE
1655 .sp
1656 For \fB/sys/fs/btrfs/features/\fP directory, each file means a supported feature
1657 of the current kernel. Most files have value 0. Otherwise it depends on the file,
1658 value \fI1\fP typically means the feature can be turned on a mounted filesystem.
1659 .sp
1660 For \fB/sys/fs/btrfs/<UUID>/features/\fP directory, each file means an enabled
1661 feature on the mounted filesystem.
1662 .sp
1663 The features share the same name in section
1664 \fI\%FILESYSTEM FEATURES\fP\&.
1665 .SS UUID
1666 .sp
1667 Files in \fB/sys/fs/btrfs/<UUID>/\fP directory are:
1668 .INDENT 0.0
1669 .TP
1670 .B bg_reclaim_threshold
1671 (RW, since: 5.19)
1672 .sp
1673 Used space percentage of total device space to start auto block group claim.
1674 Mostly for zoned devices.
1675 .TP
1676 .B checksum
1677 (RO, since: 5.5)
1678 .sp
1679 The checksum used for the mounted filesystem.
1680 This includes both the checksum type (see section
1681 \fI\%CHECKSUM ALGORITHMS\fP)
1682 and the implemented driver (mostly shows if it\(aqs hardware accelerated).
1683 .TP
1684 .B clone_alignment
1685 (RO, since: 3.16)
1686 .sp
1687 The bytes alignment for \fIclone\fP and \fIdedupe\fP ioctls.
1688 .TP
1689 .B commit_stats
1690 (RW, since: 6.0)
1691 .sp
1692 The performance statistics for btrfs transaction commit since the first
1693 mount. Mostly for debugging purposes.
1694 .sp
1695 Writing into this file will reset the maximum commit duration
1696 (\fImax_commit_ms\fP) to 0. The file looks like:
1697 .INDENT 7.0
1698 .INDENT 3.5
1699 .sp
1700 .EX
1701 commits 70649
1702 last_commit_ms 2
1703 max_commit_ms 131
1704 total_commit_ms 170840
1705 .EE
1706 .UNINDENT
1707 .UNINDENT
1708 .INDENT 7.0
1709 .IP \(bu 2
1710 \fIcommits\fP \- number of transaction commits since the first mount
1711 .IP \(bu 2
1712 \fIlast_commit_ms\fP \- duration in milliseconds of the last commit
1713 .IP \(bu 2
1714 \fImax_commit_ms\fP \- maximum time a transaction commit took since first mount or last reset
1715 .IP \(bu 2
1716 \fItotal_commit_ms\fP \- sum of all transaction commit times
1717 .UNINDENT
1718 .TP
1719 .B exclusive_operation
1720 (RO, since: 5.10)
1721 .sp
1722 Shows the running exclusive operation.
1723 Check section
1724 \fI\%FILESYSTEM EXCLUSIVE OPERATIONS\fP
1725 for details.
1726 .TP
1727 .B generation
1728 (RO, since: 5.11)
1729 .sp
1730 Show the generation of the mounted filesystem.
1731 .TP
1732 .B label
1733 (RW, since: 3.14)
1734 .sp
1735 Show the current label of the mounted filesystem.
1736 .TP
1737 .B metadata_uuid
1738 (RO, since: 5.0)
1739 .sp
1740 Shows the metadata UUID of the mounted filesystem.
1741 Check \fImetadata_uuid\fP feature for more details.
1742 .TP
1743 .B nodesize
1744 (RO, since: 3.14)
1745 .sp
1746 Show the nodesize of the mounted filesystem.
1747 .TP
1748 .B quota_override
1749 (RW, since: 4.13)
1750 .sp
1751 Shows the current quota override status.
1752 0 means no quota override.
1753 1 means quota override, quota can ignore the existing limit settings.
1754 .TP
1755 .B read_policy
1756 (RW, since: 5.11)
1757 .sp
1758 Shows the current balance policy for reads.
1759 Currently only \fBpid\fP (balance using the process id (pid) value) is
1760 supported. More balancing policies are available in experimental
1761 build, namely round\-robin.
1762 .TP
1763 .B sectorsize
1764 (RO, since: 3.14)
1765 .sp
1766 Shows the sectorsize of the mounted filesystem.
1767 .TP
1768 .B temp_fsid
1769 (RO, since 6.7)
1770 .sp
1771 Indicate that this filesystem got assigned a temporary FSID at mount time,
1772 making possible to mount devices with the same FSID.
1773 .UNINDENT
1774 .SS UUID/allocations
1775 .sp
1776 Files and directories in \fB/sys/fs/btrfs/<UUID>/allocations\fP directory are:
1777 .INDENT 0.0
1778 .TP
1779 .B global_rsv_reserved
1780 (RO, since: 3.14)
1781 .sp
1782 The used bytes of the global reservation.
1783 .TP
1784 .B global_rsv_size
1785 (RO, since: 3.14)
1786 .sp
1787 The total size of the global reservation.
1788 .TP
1789 .B \fIdata/\fP, \fImetadata/\fP and \fIsystem/\fP directories
1790 (RO, since: 5.14)
1791 .sp
1792 Space info accounting for the 3 block group types.
1793 .UNINDENT
1794 .SS UUID/allocations/{data,metadata,system}
1795 .sp
1796 Files in \fB/sys/fs/btrfs/<UUID>/allocations/\fP\fIdata,metadata,system\fP directory are:
1797 .INDENT 0.0
1798 .TP
1799 .B bg_reclaim_threshold
1800 (RW, since: 5.19)
1801 .sp
1802 Reclaimable space percentage of block group\(aqs size (excluding
1803 permanently unusable space) to reclaim the block group.
1804 Can be used on regular or zoned devices.
1805 .TP
1806 .B bytes_*
1807 (RO)
1808 .sp
1809 Values of the corresponding data structures for the given block group
1810 type and profile that are used internally and may change rapidly depending
1811 on the load.
1812 .sp
1813 Complete list: bytes_may_use, bytes_pinned, bytes_readonly,
1814 bytes_reserved, bytes_used, bytes_zone_unusable
1815 .TP
1816 .B chunk_size
1817 (RW, since: 6.0)
1818 .sp
1819 Shows the chunk size. Can be changed for data and metadata (independently)
1820 and cannot be set for system block group type.
1821 Cannot be set for zoned devices as it depends on the fixed device zone size.
1822 Upper bound is 10% of the filesystem size, the value must be multiple of 256MiB
1823 and greater than 0.
1824 .TP
1825 .B size_classes
1826 (RO, since: 6.3)
1827 .sp
1828 Numbers of block groups of a given classes based on heuristics that
1829 measure extent length, age and fragmentation.
1830 .INDENT 7.0
1831 .INDENT 3.5
1832 .sp
1833 .EX
1834 none 136
1835 small 374
1836 medium 282
1837 large 93
1838 .EE
1839 .UNINDENT
1840 .UNINDENT
1841 .UNINDENT
1842 .SS UUID/bdi
1843 .sp
1844 Symlink to the sysfs directory of the backing device info (BDI), which is
1845 related to writeback process and infrastructure.
1846 .SS UUID/devices
1847 .sp
1848 Files in \fB/sys/fs/btrfs/<UUID>/devices\fP directory are symlinks named
1849 after device nodes (e.g. sda, dm\-0) and pointing to their sysfs directory.
1850 .SS UUID/devinfo
1851 .sp
1852 The directory contains subdirectories named after device ids (numeric values). Each
1853 subdirectory has information about the device of the given \fIdevid\fP\&.
1854 .SS UUID/devinfo/DEVID
1855 .sp
1856 Files in \fB/sys/fs/btrfs/<UUID>/devinfo/<DEVID>\fP directory are:
1857 .INDENT 0.0
1858 .TP
1859 .B error_stats:
1860 (RO, since: 5.14)
1861 .sp
1862 Shows device stats of this device, same as \fBbtrfs device stats\fP (\fI\%btrfs\-device(8)\fP).
1863 .INDENT 7.0
1864 .INDENT 3.5
1865 .sp
1866 .EX
1867 write_errs 0
1868 read_errs 0
1869 flush_errs 0
1870 corruption_errs 0
1871 generation_errs 0
1872 .EE
1873 .UNINDENT
1874 .UNINDENT
1875 .TP
1876 .B fsid:
1877 (RO, since: 5.17)
1878 .sp
1879 Shows the fsid which the device belongs to.
1880 It can be different than the \fBUUID\fP if it\(aqs a seed device.
1881 .TP
1882 .B in_fs_metadata
1883 (RO, since: 5.6)
1884 .sp
1885 Shows whether we have found the device.
1886 Should always be 1, as if this turns to 0, the \fBDEVID\fP directory
1887 would get removed automatically.
1888 .TP
1889 .B missing
1890 (RO, since: 5.6)
1891 .sp
1892 Shows whether the device is considered missing by the kernel module.
1893 .TP
1894 .B replace_target
1895 (RO, since: 5.6)
1896 .sp
1897 Shows whether the device is the replace target.
1898 If no device replace is running, this value is 0.
1899 .TP
1900 .B scrub_speed_max
1901 (RW, since: 5.14)
1902 .sp
1903 Shows the scrub speed limit for this device. The unit is Bytes/s.
1904 0 means no limit. The value can be set but is not persistent.
1905 .TP
1906 .B writeable
1907 (RO, since: 5.6)
1908 .sp
1909 Show if the device is writeable.
1910 .UNINDENT
1911 .SS UUID/qgroups
1912 .sp
1913 Files in \fB/sys/fs/btrfs/<UUID>/qgroups/\fP directory are:
1914 .INDENT 0.0
1915 .TP
1916 .B enabled
1917 (RO, since: 6.1)
1918 .sp
1919 Shows if qgroup is enabled.
1920 Also, if qgroup is disabled, the \fBqgroups\fP directory will
1921 be removed automatically.
1922 .TP
1923 .B inconsistent
1924 (RO, since: 6.1)
1925 .sp
1926 Shows if the qgroup numbers are inconsistent.
1927 If 1, it\(aqs recommended to do a qgroup rescan.
1928 .TP
1929 .B drop_subtree_threshold
1930 (RW, since: 6.1)
1931 .sp
1932 Shows the subtree drop threshold to automatically mark qgroup inconsistent.
1933 .sp
1934 When dropping large subvolumes with qgroup enabled, there would be a huge
1935 load for qgroup accounting.
1936 If we have a subtree whose level is larger than or equal to this value,
1937 we will not trigger qgroup account at all, but mark qgroup inconsistent to
1938 avoid the huge workload.
1939 .sp
1940 Default value is 3, which means that trees of low height will be accounted
1941 properly as this is sufficiently fast. The value was 8 until 6.13 where
1942 no subtree drop can trigger qgroup rescan making it less useful.
1943 .sp
1944 Lower value can reduce qgroup workload, at the cost of extra qgroup rescan
1945 to re\-calculate the numbers.
1946 .UNINDENT
1947 .SS UUID/qgroups/LEVEL_ID
1948 .sp
1949 Files in each \fB/sys/fs/btrfs/<UUID>/qgroups/<LEVEL>_<ID>/\fP directory are:
1950 .INDENT 0.0
1951 .TP
1952 .B exclusive
1953 (RO, since: 5.9)
1954 .sp
1955 Shows the exclusively owned bytes of the qgroup.
1956 .TP
1957 .B limit_flags
1958 (RO, since: 5.9)
1959 .sp
1960 Shows the numeric value of the limit flags.
1961 If 0, means no limit implied.
1962 .TP
1963 .B max_exclusive
1964 (RO, since: 5.9)
1965 .sp
1966 Shows the limits on exclusively owned bytes.
1967 .TP
1968 .B max_referenced
1969 (RO, since: 5.9)
1970 .sp
1971 Shows the limits on referenced bytes.
1972 .TP
1973 .B referenced
1974 (RO, since: 5.9)
1975 .sp
1976 Shows the referenced bytes of the qgroup.
1977 .TP
1978 .B rsv_data
1979 (RO, since: 5.9)
1980 .sp
1981 Shows the reserved bytes for data.
1982 .TP
1983 .B rsv_meta_pertrans
1984 (RO, since: 5.9)
1985 .sp
1986 Shows the reserved bytes for per transaction metadata.
1987 .TP
1988 .B rsv_meta_prealloc
1989 (RO, since: 5.9)
1990 .sp
1991 Shows the reserved bytes for preallocated metadata.
1992 .UNINDENT
1993 .SS UUID/discard
1994 .sp
1995 Files in \fB/sys/fs/btrfs/<UUID>/discard/\fP directory are:
1996 .INDENT 0.0
1997 .TP
1998 .B discardable_bytes
1999 (RO, since: 6.1)
2000 .sp
2001 Shows amount of bytes that can be discarded in the async discard and
2002 nodiscard mode.
2003 .TP
2004 .B discardable_extents
2005 (RO, since: 6.1)
2006 .sp
2007 Shows number of extents to be discarded in the async discard and
2008 nodiscard mode.
2009 .TP
2010 .B discard_bitmap_bytes
2011 (RO, since: 6.1)
2012 .sp
2013 Shows amount of discarded bytes from data tracked as bitmaps.
2014 .TP
2015 .B discard_extent_bytes
2016 (RO, since: 6.1)
2017 .sp
2018 Shows amount of discarded extents from data tracked as bitmaps.
2019 .TP
2020 .B discard_bytes_saved
2021 (RO, since: 6.1)
2022 .sp
2023 Shows the amount of bytes that were reallocated without being discarded.
2024 .TP
2025 .B kbps_limit
2026 (RW, since: 6.1)
2027 .sp
2028 Tunable limit of kilobytes per second issued as discard IO in the async
2029 discard mode.
2030 .TP
2031 .B iops_limit
2032 (RW, since: 6.1)
2033 .sp
2034 Tunable limit of number of discard IO operations to be issued in the
2035 async discard mode.
2036 .TP
2037 .B max_discard_size
2038 (RW, since: 6.1)
2039 .sp
2040 Tunable limit for size of one IO discard request.
2041 .UNINDENT
2042 .SH FILESYSTEM EXCLUSIVE OPERATIONS
2043 .sp
2044 There are several operations that affect the whole filesystem and cannot be run
2045 in parallel. Attempt to start one while another is running will fail (see
2046 exceptions below).
2047 .sp
2048 Since kernel 5.10 the currently running operation can be obtained from
2049 \fB/sys/fs/UUID/exclusive_operation\fP with following values and operations:
2050 .INDENT 0.0
2051 .IP \(bu 2
2052 balance
2053 .IP \(bu 2
2054 balance paused (since 5.17)
2055 .IP \(bu 2
2056 device add
2057 .IP \(bu 2
2058 device delete
2059 .IP \(bu 2
2060 device replace
2061 .IP \(bu 2
2062 resize
2063 .IP \(bu 2
2064 swapfile activate
2065 .IP \(bu 2
2066 none
2067 .UNINDENT
2068 .sp
2069 Enqueuing is supported for several btrfs subcommands so they can be started
2070 at once and then serialized.
2071 .sp
2072 There\(aqs an exception when a paused balance allows to start a device add
2073 operation as they don\(aqt really collide and this can be used to add more space
2074 for the balance to finish.
2075 .SH FILESYSTEM LIMITS
2076 .INDENT 0.0
2077 .TP
2078 .B maximum file name length
2079 255
2080 .sp
2081 This limit is imposed by Linux VFS, the structures of BTRFS could store
2082 larger file names.
2083 .TP
2084 .B maximum symlink target length
2085 depends on the \fInodesize\fP value, for 4KiB it\(aqs 3949 bytes, for larger nodesize
2086 it\(aqs 4095 due to the system limit PATH_MAX
2087 .sp
2088 The symlink target may not be a valid path, i.e. the path name components
2089 can exceed the limits (NAME_MAX), there\(aqs no content validation at \X'tty: link https://man7.org/linux/man-pages/man3/symlink.3.html'\fI\%symlink(3)\fP\X'tty: link'
2090 creation.
2091 .TP
2092 .B maximum number of inodes
2093 2\s-2\u64\d\s0 but depends on the available metadata space as the inodes are created
2094 dynamically
2095 .sp
2096 Each subvolume is an independent namespace of inodes and thus their
2097 numbers, so the limit is per subvolume, not for the whole filesystem.
2098 .TP
2099 .B inode numbers
2100 minimum number: 256 (for subvolumes), regular files and directories: 257,
2101 maximum number: (2\s-2\u64\d\s0 \- 256)
2102 .sp
2103 The inode numbers that can be assigned to user created files are from
2104 the whole 64bit space except first 256 and last 256 in that range that
2105 are reserved for internal b\-tree identifiers.
2106 .TP
2107 .B maximum file length
2108 inherent limit of BTRFS is 2\s-2\u64\d\s0 (16 EiB) but the practical
2109 limit of Linux VFS is 2\s-2\u63\d\s0 (8 EiB)
2110 .TP
2111 .B maximum number of subvolumes
2112 the subvolume ids can go up to 2\s-2\u48\d\s0 but the number of actual subvolumes
2113 depends on the available metadata space
2114 .sp
2115 The space consumed by all subvolume metadata includes bookkeeping of
2116 shared extents can be large (MiB, GiB). The range is not the full 64bit
2117 range because of qgroups that use the upper 16 bits for another
2118 purposes.
2119 .TP
2120 .B maximum number of hardlinks of a file in a directory
2121 65536 when the \fIextref\fP feature is turned on during mkfs (default), roughly
2122 100 otherwise and depends on file name length that fits into one metadata node
2123 .TP
2124 .B minimum filesystem size
2125 the minimal size of each device depends on the \fImixed\-bg\fP feature, without that
2126 (the default) it\(aqs about 109MiB, with mixed\-bg it\(aqs is 16MiB
2127 .UNINDENT
2128 .SH BOOTLOADER SUPPORT
2129 .sp
2130 GRUB2 (\X'tty: link https://www.gnu.org/software/grub'\fI\%https://www.gnu.org/software/grub\fP\X'tty: link') has the most advanced support of
2131 booting from BTRFS with respect to features.
2132 .sp
2133 U\-Boot (\X'tty: link https://www.denx.de/wiki/U-Boot/'\fI\%https://www.denx.de/wiki/U\-Boot/\fP\X'tty: link') has decent support for booting but
2134 not all BTRFS features are implemented, check the documentation.
2135 .sp
2136 In general, the first 1MiB on each device is unused with the exception of
2137 primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be
2138 freely used by bootloaders or for other system information. Note that booting
2139 from a filesystem on \fI\%zoned device\fP is not supported.
2140 .SH FILE ATTRIBUTES
2141 .sp
2142 The btrfs filesystem supports setting file attributes or flags. Note there are
2143 old and new interfaces, with confusing names. The following list should clarify
2144 that:
2145 .INDENT 0.0
2146 .IP \(bu 2
2147 \fIattributes\fP: \X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link' or \X'tty: link https://man7.org/linux/man-pages/man1/lsattr.1.html'\fI\%lsattr(1)\fP\X'tty: link' utilities (the ioctls are
2148 FS_IOC_GETFLAGS and FS_IOC_SETFLAGS), due to the ioctl names the attributes
2149 are also called flags
2150 .IP \(bu 2
2151 \fIxflags\fP: to distinguish from the previous, it\(aqs extended flags, with tunable
2152 bits similar to the attributes but extensible and new bits will be added in
2153 the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR but they
2154 are not related to extended attributes that are also called xattrs), there\(aqs
2155 no standard tool to change the bits, there\(aqs support in \X'tty: link https://man7.org/linux/man-pages/man8/xfs_io.8.html'\fI\%xfs_io(8)\fP\X'tty: link' as
2156 command \fBxfs_io \-c chattr\fP
2157 .UNINDENT
2158 .sp
2159 Attributes have constraints associated and not all combinations can be set, the
2160 order of setting them also matters. Most attributes apply to files and
2161 directories but the semantics may differ. For directories the attribute may
2162 only mean to set this attribute to all new files (\fIinheritable\fP in the list
2163 below). Some attributes need root privileges to be set.
2164 .SS Attributes
2165 .INDENT 0.0
2166 .TP
2167 .B a
2168 (file, dir, root)
2169 \fIappend only\fP, new writes are always written at the end of the file
2170 .TP
2171 .B A
2172 (file, dir)
2173 \fIno atime updates\fP
2174 .TP
2175 .B c
2176 (file, dir, inherited)
2177 \fIcompress data\fP, all data written after this attribute is set will be compressed.
2178 Please note that compression is also affected by the mount options or the parent
2179 directory attributes.
2180 .sp
2181 When set on a directory, all newly created files will inherit this attribute.
2182 This attribute cannot be set with \(aqm\(aq at the same time.
2183 .TP
2184 .B C
2185 (file, dir, inherited)
2186 \fIno copy\-on\-write\fP, file data modifications are done in\-place
2187 .sp
2188 When set on a directory, all newly created files will inherit this attribute.
2189 .sp
2190 \fBNOTE:\fP
2191 .INDENT 7.0
2192 .INDENT 3.5
2193 Due to implementation limitations, this flag can be set/unset only on
2194 empty files.
2195 .UNINDENT
2196 .UNINDENT
2197 .TP
2198 .B d
2199 (file)
2200 \fIno dump\fP, makes sense with 3rd party tools like \X'tty: link https://man7.org/linux/man-pages/man8/dump.8.html'\fI\%dump(8)\fP\X'tty: link', on BTRFS the
2201 attribute can be set/unset but no other special handling is done
2202 .TP
2203 .B D
2204 (dir)
2205 \fIsynchronous directory updates\fP, for more details search \X'tty: link https://man7.org/linux/man-pages/man2/open.2.html'\fI\%open(2)\fP\X'tty: link' for \fIO_SYNC\fP
2206 and \fIO_DSYNC\fP
2207 .TP
2208 .B i
2209 (file, dir, root)
2210 \fIimmutable\fP, no file data and metadata changes allowed even to the root user as
2211 long as this attribute is set (obviously the exception is unsetting the attribute)
2212 .TP
2213 .B m
2214 (file, dir)
2215 \fIno compression\fP, permanently turn off compression on the given file. Any
2216 compression mount options will not affect this file. (\X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link' support added in
2217 1.46.2)
2218 .sp
2219 When set on a directory, all newly created files will inherit this attribute.
2220 This attribute cannot be set with \fIc\fP at the same time.
2221 .TP
2222 .B S
2223 (file)
2224 \fIsynchronous updates\fP, for more details search \X'tty: link https://man7.org/linux/man-pages/man2/open.2.html'\fI\%open(2)\fP\X'tty: link' for \fIO_SYNC\fP and
2225 \fIO_DSYNC\fP
2226 .TP
2227 .B V
2228 (file, read\-only)
2229 \fIfs\-verity enabled\fP on the file
2230 .UNINDENT
2231 .sp
2232 No other attributes are supported. For the complete list please refer to the
2233 \X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link' manual page.
2234 .SS XFLAGS
2235 .sp
2236 There\(aqs an overlap of letters assigned to the bits with the attributes, this list
2237 refers to what \X'tty: link https://man7.org/linux/man-pages/man8/xfs_io.8.html'\fI\%xfs_io(8)\fP\X'tty: link' provides:
2238 .INDENT 0.0
2239 .TP
2240 .B i
2241 \fIimmutable\fP, same as the attribute
2242 .TP
2243 .B a
2244 \fIappend only\fP, same as the attribute
2245 .TP
2246 .B s
2247 \fIsynchronous updates\fP, same as the attribute \fIS\fP
2248 .TP
2249 .B A
2250 \fIno atime updates\fP, same as the attribute
2251 .TP
2252 .B d
2253 \fIno dump\fP, same as the attribute
2254 .UNINDENT
2255 .SH ZONED MODE
2256 .sp
2257 Since version 5.12 btrfs supports so called \fIzoned mode\fP\&. This is a special
2258 on\-disk format and allocation/write strategy that\(aqs friendly to zoned devices.
2259 In short, a device is partitioned into fixed\-size zones and each zone can be
2260 updated by append\-only manner, or reset. As btrfs has no fixed data structures,
2261 except the super blocks, the zoned mode only requires block placement that
2262 follows the device constraints. You can learn about the whole architecture at
2263 \X'tty: link https://zonedstorage.io'\fI\%https://zonedstorage.io\fP\X'tty: link' .
2264 .sp
2265 The devices are also called SMR/ZBC/ZNS, in \fIhost\-managed\fP mode. Note that
2266 there are devices that appear as non\-zoned but actually are, this is
2267 \fIdrive\-managed\fP and using zoned mode won\(aqt help.
2268 .sp
2269 The zone size depends on the device, typical sizes are 256MiB or 1GiB. In
2270 general it must be a power of two. Emulated zoned devices like \fInull_blk\fP allow
2271 to set various zone sizes.
2272 .SS Requirements, limitations
2273 .INDENT 0.0
2274 .IP \(bu 2
2275 all devices must have the same zone size
2276 .IP \(bu 2
2277 maximum zone size is 8GiB
2278 .IP \(bu 2
2279 minimum zone size is 4MiB
2280 .IP \(bu 2
2281 mixing zoned and non\-zoned devices is possible, the zone writes are emulated,
2282 but this is namely for testing
2283 .IP \(bu 2
2284 the super block is handled in a special way and is at different locations than on a non\-zoned filesystem:
2285 .INDENT 2.0
2286 .IP \(bu 2
2287 primary: 0B (and the next two zones)
2288 .IP \(bu 2
2289 secondary: 512GiB (and the next two zones)
2290 .IP \(bu 2
2291 tertiary: 4TiB (4096GiB, and the next two zones)
2292 .UNINDENT
2293 .UNINDENT
2294 .SS Incompatible features
2295 .sp
2296 The main constraint of the zoned devices is lack of in\-place update of the data.
2297 This is inherently incompatible with some features:
2298 .INDENT 0.0
2299 .IP \(bu 2
2300 NODATACOW \- overwrite in\-place, cannot create such files
2301 .IP \(bu 2
2302 fallocate \- preallocating space for in\-place first write
2303 .IP \(bu 2
2304 mixed\-bg \- unordered writes to data and metadata, fixing that means using
2305 separate data and metadata block groups
2306 .IP \(bu 2
2307 booting \- the zone at offset 0 contains superblock, resetting the zone would
2308 destroy the bootloader data
2309 .UNINDENT
2310 .sp
2311 Initial support lacks some features but they\(aqre planned:
2312 .INDENT 0.0
2313 .IP \(bu 2
2314 only single (data, metadata) and DUP (metadata) profile is supported
2315 .IP \(bu 2
2316 fstrim \- due to dependency on free space cache v1
2317 .UNINDENT
2318 .SS Super block
2319 .sp
2320 As said above, super block is handled in a special way. In order to be crash
2321 safe, at least one zone in a known location must contain a valid superblock.
2322 This is implemented as a ring buffer in two consecutive zones, starting from
2323 known offsets 0B, 512GiB and 4TiB.
2324 .sp
2325 The values are different than on non\-zoned devices. Each new super block is
2326 appended to the end of the zone, once it\(aqs filled, the zone is reset and writes
2327 continue to the next one. Looking up the latest super block needs to read
2328 offsets of both zones and determine the last written version.
2329 .sp
2330 The amount of space reserved for super block depends on the zone size. The
2331 secondary and tertiary copies are at distant offsets as the capacity of the
2332 devices is expected to be large, tens of terabytes. Maximum zone size supported
2333 is 8GiB, which would mean that e.g. offset 0\-16GiB would be reserved just for
2334 the super block on a hypothetical device of that zone size. This is wasteful
2335 but required to guarantee crash safety.
2336 .SS Zone reclaim, garbage collection
2337 .sp
2338 As the zones are append\-only, overwriting data or COW changes in metadata
2339 make parts of the zones used but not connected to the filesystem structures.
2340 This makes the space unusable and grows over time. Once the ratio hits a
2341 (configurable) threshold a background reclaim process is started and relocates
2342 the remaining blocks in use to a new zone. The old one is reset and can be used
2343 again.
2344 .sp
2345 This process may take some time depending on other background work or
2346 amount of new data written. It is possible to hit an intermittent ENOSPC.
2347 Some devices also limit number of active zones.
2348 .SS Devices
2349 .SS Real hardware
2350 .sp
2351 The WD Ultrastar series 600 advertises HM\-SMR, i.e. the host\-managed zoned
2352 mode. There are two more: DA (device managed, no zoned information exported to
2353 the system), HA (host aware, can be used as regular disk but zoned writes
2354 improve performance). There are not many devices available at the moment, the
2355 information about exact zoned mode is hard to find, check data sheets or
2356 community sources gathering information from real devices.
2357 .sp
2358 Note: zoned mode won\(aqt work with DM\-SMR disks.
2359 .INDENT 0.0
2360 .IP \(bu 2
2361 Ultrastar® DC ZN540 NVMe ZNS SSD (\X'tty: link https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/product-brief/product-brief-ultrastar-dc-zn540.pdf'\fI\%product
2362 brief\fP\X'tty: link')
2363 .UNINDENT
2364 .SS Emulated: null_blk
2365 .sp
2366 The driver \fInull_blk\fP provides memory backed device and is suitable for
2367 testing. There are some quirks setting up the devices. The module must be
2368 loaded with \fInr_devices=0\fP or the numbering of device nodes will be offset. The
2369 \fIconfigfs\fP must be mounted at \fI/sys/kernel/config\fP and the administration of
2370 the null_blk devices is done in \fI/sys/kernel/config/nullb\fP\&. The device nodes
2371 are named like \fB/dev/nullb0\fP and are numbered sequentially. NOTE: the device
2372 name may be different than the named directory in sysfs!
2373 .sp
2374 Setup:
2375 .INDENT 0.0
2376 .INDENT 3.5
2377 .sp
2378 .EX
2379 modprobe\ configfs
2380 modprobe\ null_blk\ nr_devices=0
2381 .EE
2382 .UNINDENT
2383 .UNINDENT
2384 .sp
2385 Create a device \fImydev\fP, assuming no other previously created devices, size is
2386 2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal
2387 example taking defaults:
2388 .INDENT 0.0
2389 .INDENT 3.5
2390 .sp
2391 .EX
2392 cd\ /sys/kernel/config/nullb/
2393 mkdir\ mydev
2394 cd\ mydev
2395 echo\ 2048\ >\ size
2396 echo\ 1\ >\ zoned
2397 echo\ 1\ >\ memory_backed
2398 echo\ 256\ >\ zone_size
2399 echo\ 1\ >\ power
2400 .EE
2401 .UNINDENT
2402 .UNINDENT
2403 .sp
2404 This will create a device \fB/dev/nullb0\fP and the value of file \fIindex\fP will
2405 match the ending number of the device node.
2406 .sp
2407 Remove the device:
2408 .INDENT 0.0
2409 .INDENT 3.5
2410 .sp
2411 .EX
2412 rmdir\ /sys/kernel/config/nullb/mydev
2413 .EE
2414 .UNINDENT
2415 .UNINDENT
2416 .sp
2417 Then continue with \fBmkfs.btrfs /dev/nullb0\fP, the zoned mode is auto\-detected.
2418 .sp
2419 For convenience, there\(aqs a script wrapping the basic null_blk management operations
2420 \X'tty: link https://github.com/kdave/nullb.git'\fI\%https://github.com/kdave/nullb.git\fP\X'tty: link', the above commands become:
2421 .INDENT 0.0
2422 .INDENT 3.5
2423 .sp
2424 .EX
2425 nullb setup
2426 nullb create \-s 2g \-z 256
2427 mkfs.btrfs /dev/nullb0
2428 \&...
2429 nullb rm nullb0
2430 .EE
2431 .UNINDENT
2432 .UNINDENT
2433 .SS Emulated: TCMU runner
2434 .sp
2435 TCMU is a framework to emulate SCSI devices in userspace, providing various
2436 backends for the storage, with zoned support as well. A file\-backed zoned
2437 device can provide more options for larger storage and zone size. Please follow
2438 the instructions at \X'tty: link https://zonedstorage.io/projects/tcmu-runner/'\fI\%https://zonedstorage.io/projects/tcmu\-runner/\fP\X'tty: link' .
2439 .SS Compatibility, incompatibility
2440 .INDENT 0.0
2441 .IP \(bu 2
2442 the feature sets an incompat bit and requires new kernel to access the
2443 filesystem (for both read and write)
2444 .IP \(bu 2
2445 superblock needs to be handled in a special way, there are still 3 copies
2446 but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a
2447 ring buffer of the superblocks, finding the latest one needs reading it from
2448 the write pointer or do a full scan of the zones
2449 .IP \(bu 2
2450 mixing zoned and non zoned devices is possible (zones are emulated) but is
2451 recommended only for testing
2452 .IP \(bu 2
2453 mixing zoned devices with different zone sizes is not possible
2454 .IP \(bu 2
2455 zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB
2456 or 1GiB, larger size is expected, maximum zone size supported by btrfs is
2457 8GiB
2458 .UNINDENT
2459 .SS Status, stability, reporting bugs
2460 .sp
2461 The zoned mode has been released in 5.12 and there are still some rough edges
2462 and corner cases one can hit during testing. Please report bugs to
2463 \X'tty: link https://github.com/naota/linux/issues/'\fI\%https://github.com/naota/linux/issues/\fP\X'tty: link' .
2464 .SS References
2465 .INDENT 0.0
2466 .IP \(bu 2
2467 \X'tty: link https://zonedstorage.io'\fI\%https://zonedstorage.io\fP\X'tty: link'
2468 .INDENT 2.0
2469 .IP \(bu 2
2470 \X'tty: link https://zonedstorage.io/projects/libzbc/'\fI\%https://zonedstorage.io/projects/libzbc/\fP\X'tty: link' \-\- \fIlibzbc\fP is library and set
2471 of tools to directly manipulate devices with ZBC/ZAC support
2472 .IP \(bu 2
2473 \X'tty: link https://zonedstorage.io/projects/libzbd/'\fI\%https://zonedstorage.io/projects/libzbd/\fP\X'tty: link' \-\- \fIlibzbd\fP uses the kernel
2474 provided zoned block device interface based on the ioctl() system calls
2475 .UNINDENT
2476 .IP \(bu 2
2477 \X'tty: link https://hddscan.com/blog/2020/hdd-wd-smr.html'\fI\%https://hddscan.com/blog/2020/hdd\-wd\-smr.html\fP\X'tty: link' \-\- some details about exact device types
2478 .IP \(bu 2
2479 \X'tty: link https://lwn.net/Articles/853308/'\fI\%https://lwn.net/Articles/853308/\fP\X'tty: link' \-\- \fIBtrfs on zoned block devices\fP
2480 .IP \(bu 2
2481 \X'tty: link https://www.usenix.org/conference/vault20/presentation/bjorling'\fI\%https://www.usenix.org/conference/vault20/presentation/bjorling\fP\X'tty: link' \-\- Zone
2482 Append: A New Way of Writing to Zoned Storage
2483 .UNINDENT
2484 .SH CONTROL DEVICE
2485 .sp
2486 There\(aqs a character special device \fB/dev/btrfs\-control\fP with major and minor
2487 numbers 10 and 234 (the device can be found under the \fImisc\fP category).
2488 .INDENT 0.0
2489 .INDENT 3.5
2490 .sp
2491 .EX
2492 $ ls \-l /dev/btrfs\-control
2493 crw\-\-\-\-\-\-\- 1 root root 10, 234 Jan 1 12:00 /dev/btrfs\-control
2494 .EE
2495 .UNINDENT
2496 .UNINDENT
2497 .sp
2498 The device accepts some ioctl calls that can perform following actions on the
2499 filesystem module:
2500 .INDENT 0.0
2501 .IP \(bu 2
2502 scan devices for btrfs filesystem (i.e. to let multi\-device filesystems mount
2503 automatically) and register them with the kernel module
2504 .IP \(bu 2
2505 similar to scan, but also wait until the device scanning process is finished
2506 for a given filesystem
2507 .IP \(bu 2
2508 get the supported features (can be also found under \fB/sys/fs/btrfs/features\fP)
2509 .UNINDENT
2510 .sp
2511 The device is created when btrfs is initialized, either as a module or a
2512 built\-in functionality and makes sense only in connection with that. Running
2513 e.g. mkfs without the module loaded will not register the device and will
2514 probably warn about that.
2515 .sp
2516 In rare cases when the module is loaded but the device is not present (most
2517 likely accidentally deleted), it\(aqs possible to recreate it by
2518 .INDENT 0.0
2519 .INDENT 3.5
2520 .sp
2521 .EX
2522 # mknod \-\-mode=600 /dev/btrfs\-control c 10 234
2523 .EE
2524 .UNINDENT
2525 .UNINDENT
2526 .sp
2527 or (since 5.11) by a convenience command
2528 .INDENT 0.0
2529 .INDENT 3.5
2530 .sp
2531 .EX
2532 # btrfs rescue create\-control\-device
2533 .EE
2534 .UNINDENT
2535 .UNINDENT
2536 .sp
2537 The control device is not strictly required but the device scanning will not
2538 work and a workaround would need to be used to mount a multi\-device filesystem.
2539 The mount option \fIdevice\fP can trigger the device scanning during mount, see
2540 also \fBbtrfs device scan\fP\&.
2541 .SH FILESYSTEM WITH MULTIPLE PROFILES
2542 .sp
2543 It is possible that a btrfs filesystem contains multiple block group profiles
2544 of the same type. This could happen when a profile conversion using balance
2545 filters is interrupted (see \fI\%btrfs\-balance(8)\fP). Some
2546 \fBbtrfs\fP commands perform
2547 a test to detect this kind of condition and print a warning like this:
2548 .INDENT 0.0
2549 .INDENT 3.5
2550 .sp
2551 .EX
2552 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2553 WARNING: Data: single, raid1
2554 WARNING: Metadata: single, raid1
2555 .EE
2556 .UNINDENT
2557 .UNINDENT
2558 .sp
2559 The corresponding output of \fBbtrfs filesystem df\fP might look like:
2560 .INDENT 0.0
2561 .INDENT 3.5
2562 .sp
2563 .EX
2564 WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq.
2565 WARNING: Data: single, raid1
2566 WARNING: Metadata: single, raid1
2567 Data, RAID1: total=832.00MiB, used=0.00B
2568 Data, single: total=1.63GiB, used=0.00B
2569 System, single: total=4.00MiB, used=16.00KiB
2570 Metadata, single: total=8.00MiB, used=112.00KiB
2571 Metadata, RAID1: total=64.00MiB, used=32.00KiB
2572 GlobalReserve, single: total=16.25MiB, used=0.00B
2573 .EE
2574 .UNINDENT
2575 .UNINDENT
2576 .sp
2577 There\(aqs more than one line for type \fIData\fP and \fIMetadata\fP, while the profiles
2578 are \fIsingle\fP and \fIRAID1\fP\&.
2579 .sp
2580 This state of the filesystem OK but most likely needs the user/administrator to
2581 take an action and finish the interrupted tasks. This cannot be easily done
2582 automatically, also the user knows the expected final profiles.
2583 .sp
2584 In the example above, the filesystem started as a single device and \fIsingle\fP
2585 block group profile. Then another device was added, followed by balance with
2586 \fIconvert=raid1\fP but for some reason hasn\(aqt finished. Restarting the balance
2587 with \fIconvert=raid1\fP will continue and end up with filesystem with all block
2588 group profiles \fIRAID1\fP\&.
2589 .sp
2590 \fBNOTE:\fP
2591 .INDENT 0.0
2592 .INDENT 3.5
2593 If you\(aqre familiar with balance filters, you can use
2594 \fIconvert=raid1,profiles=single,soft\fP, which will take only the unconverted
2595 \fIsingle\fP profiles and convert them to \fIraid1\fP\&. This may speed up the conversion
2596 as it would not try to rewrite the already convert \fIraid1\fP profiles.
2597 .UNINDENT
2598 .UNINDENT
2599 .sp
2600 Having just one profile is desired as this also clearly defines the profile of
2601 newly allocated block groups, otherwise this depends on internal allocation
2602 policy. When there are multiple profiles present, the order of selection is
2603 RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are
2604 satisfied.
2605 .sp
2606 Commands that print the warning were chosen so they\(aqre brought to user
2607 attention when the filesystem state is being changed in that regard. This is:
2608 \fBdevice add\fP, \fBdevice delete\fP, \fBbalance cancel\fP, \fBbalance pause\fP\&. Commands
2609 that report space usage: \fBfilesystem df\fP, \fBdevice usage\fP\&. The command
2610 \fBfilesystem usage\fP provides a line in the overall summary:
2611 .INDENT 0.0
2612 .INDENT 3.5
2613 .sp
2614 .EX
2615 Multiple profiles: yes (data, metadata)
2616 .EE
2617 .UNINDENT
2618 .UNINDENT
2619 .SH SEEDING DEVICE
2620 .sp
2621 The COW mechanism and multiple devices under one hood enable an interesting
2622 concept, called a seeding device: extending a read\-only filesystem on a
2623 device with another device that captures all writes. For example
2624 imagine an immutable golden image of an operating system enhanced with another
2625 device that allows to use the data from the golden image and normal operation.
2626 This idea originated on CD\-ROMs with base OS and allowing to use them for live
2627 systems, but this became obsolete. There are technologies providing similar
2628 functionality, like \X'tty: link https://en.wikipedia.org/wiki/Union_mount'\fI\%unionmount\fP\X'tty: link',
2629 \X'tty: link https://en.wikipedia.org/wiki/OverlayFS'\fI\%overlayfs\fP\X'tty: link' or
2630 \X'tty: link https://en.wikipedia.org/wiki/Qcow#qcow2'\fI\%qcow2\fP\X'tty: link' image snapshot.
2631 .sp
2632 The seeding device starts as a normal filesystem, once the contents is ready,
2633 \fBbtrfstune \-S 1\fP is used to flag it as a seeding device. Mounting such device
2634 will not allow any writes, except adding a new device by \fBbtrfs device add\fP\&.
2635 Then the filesystem can be remounted as read\-write.
2636 .sp
2637 Given that the filesystem on the seeding device is always recognized as
2638 read\-only, it can be used to seed multiple filesystems from one device at the
2639 same time. The UUID that is normally attached to a device is automatically
2640 changed to a random UUID on each mount.
2641 .sp
2642 \fBNOTE:\fP
2643 .INDENT 0.0
2644 .INDENT 3.5
2645 Before v6.17 kernel, a seed device could have been mounted
2646 independently along with sprouted filesystems.
2647 But since 6.17 kernel, a seed device can only be mounted either through
2648 a sprouted filesystem, or the seed device itself, not both at the same time.
2649 .sp
2650 This is to ensure a block device to have only a single filesystem bound
2651 to it, so that runtime device missing events can be properly handled.
2652 .UNINDENT
2653 .UNINDENT
2654 .sp
2655 Once the seeding device is mounted, it needs the writable device. After adding
2656 it, unmounting and mounting with \fBumount /path; mount /dev/writable
2657 /path\fP or remounting read\-write with \fBremount \-o remount,rw\fP makes the
2658 filesystem at \fB/path\fP ready for use.
2659 .sp
2660 \fBNOTE:\fP
2661 .INDENT 0.0
2662 .INDENT 3.5
2663 There was a known bug with using remount to make the mount writeable:
2664 remount will leave the filesystem in a state where it is unable to
2665 clean deleted snapshots, so it will leak space until it is unmounted
2666 and mounted properly.
2667 .sp
2668 That bug has been fixed in 5.11 and newer kernels.
2669 .UNINDENT
2670 .UNINDENT
2671 .sp
2672 Furthermore, deleting the seeding device from the filesystem can turn it into
2673 a normal filesystem, provided that the writable device can also contain all the
2674 data from the seeding device.
2675 .sp
2676 The seeding device flag can be cleared again by \fBbtrfstune \-f \-S 0\fP, e.g.
2677 allowing to update with newer data but please note that this will invalidate
2678 all existing filesystems that use this particular seeding device. This works
2679 for some use cases, not for others, and the forcing flag to the command is
2680 mandatory to avoid accidental mistakes.
2681 .sp
2682 Example how to create and use one seeding device:
2683 .INDENT 0.0
2684 .INDENT 3.5
2685 .sp
2686 .EX
2687 # mkfs.btrfs /dev/sda
2688 # mount /dev/sda /mnt/mnt1
2689 \&... fill mnt1 with data
2690 # umount /mnt/mnt1
2691
2692 # btrfstune \-S 1 /dev/sda
2693
2694 # mount /dev/sda /mnt/mnt1
2695 # btrfs device add /dev/sdb /mnt/mnt1
2696 # umount /mnt/mnt1
2697 # mount /dev/sdb /mnt/mnt1
2698 \&... /mnt/mnt1 is now writable
2699 .EE
2700 .UNINDENT
2701 .UNINDENT
2702 .sp
2703 Now \fB/mnt/mnt1\fP can be used normally. The device \fB/dev/sda\fP can be mounted
2704 again with a another writable device:
2705 .INDENT 0.0
2706 .INDENT 3.5
2707 .sp
2708 .EX
2709 # mount /dev/sda /mnt/mnt2
2710 # btrfs device add /dev/sdc /mnt/mnt2
2711 # umount /mnt/mnt2
2712 # mount /dev/sdc /mnt/mnt2
2713 \&... /mnt/mnt2 is now writable
2714 .EE
2715 .UNINDENT
2716 .UNINDENT
2717 .sp
2718 The writable device (file:\fI/dev/sdb\fP) can be decoupled from the seeding device and
2719 used independently:
2720 .INDENT 0.0
2721 .INDENT 3.5
2722 .sp
2723 .EX
2724 # btrfs device delete /dev/sda /mnt/mnt1
2725 .EE
2726 .UNINDENT
2727 .UNINDENT
2728 .sp
2729 As the contents originated in the seeding device, it\(aqs possible to turn
2730 \fB/dev/sdb\fP to a seeding device again and repeat the whole process.
2731 .sp
2732 A few things to note:
2733 .INDENT 0.0
2734 .IP \(bu 2
2735 it\(aqs recommended to use only single device for the seeding device, it works
2736 for multiple devices but the \fIsingle\fP profile must be used in order to make
2737 the seeding device deletion work
2738 .IP \(bu 2
2739 block group profiles \fIsingle\fP and \fIdup\fP support the use cases above
2740 .IP \(bu 2
2741 the label is copied from the seeding device and can be changed by \fBbtrfs filesystem label\fP
2742 .IP \(bu 2
2743 each new mount of the seeding device gets a new random UUID
2744 .IP \(bu 2
2745 \fBumount /path; mount /dev/writable /path\fP can be replaced with
2746 \fBmount \-o remount,rw /path\fP
2747 but it won\(aqt reclaim space of deleted subvolumes until the seeding device
2748 is mounted read\-write again before making it seeding again
2749 .UNINDENT
2750 .SS Chained seeding devices
2751 .sp
2752 Though it\(aqs not recommended and is rather an obscure and untested use case,
2753 chaining seeding devices is possible. In the first example, the writable device
2754 \fB/dev/sdb\fP can be turned onto another seeding device again, depending on the
2755 unchanged seeding device \fB/dev/sda\fP\&. Then using \fB/dev/sdb\fP as the primary
2756 seeding device it can be extended with another writable device, say \fB/dev/sdd\fP,
2757 and it continues as before as a simple tree structure on devices.
2758 .INDENT 0.0
2759 .INDENT 3.5
2760 .sp
2761 .EX
2762 # mkfs.btrfs /dev/sda
2763 # mount /dev/sda /mnt/mnt1
2764 \&... fill mnt1 with data
2765 # umount /mnt/mnt1
2766
2767 # btrfstune \-S 1 /dev/sda
2768
2769 # mount /dev/sda /mnt/mnt1
2770 # btrfs device add /dev/sdb /mnt/mnt1
2771 # mount \-o remount,rw /mnt/mnt1
2772 \&... /mnt/mnt1 is now writable
2773 # umount /mnt/mnt1
2774
2775 # btrfstune \-S 1 /dev/sdb
2776
2777 # mount /dev/sdb /mnt/mnt1
2778 # btrfs device add /dev/sdc /mnt
2779 # mount \-o remount,rw /mnt/mnt1
2780 \&... /mnt/mnt1 is now writable
2781 # umount /mnt/mnt1
2782 .EE
2783 .UNINDENT
2784 .UNINDENT
2785 .sp
2786 As a result we have:
2787 .INDENT 0.0
2788 .IP \(bu 2
2789 \fIsda\fP is a single seeding device, with its initial contents
2790 .IP \(bu 2
2791 \fIsdb\fP is a seeding device but requires \fIsda\fP, the contents are from the time
2792 when \fIsdb\fP is made seeding, i.e. contents of \fIsda\fP with any later changes
2793 .IP \(bu 2
2794 \fIsdc\fP last writable, can be made a seeding one the same way as was \fIsdb\fP,
2795 preserving its contents and depending on \fIsda\fP and \fIsdb\fP
2796 .UNINDENT
2797 .sp
2798 As long as the seeding devices are unmodified and available, they can be used
2799 to start another branch.
2800 .SH RAID56 STATUS AND RECOMMENDED PRACTICES
2801 .sp
2802 The RAID56 feature provides striping and parity over several devices, same as
2803 the traditional RAID5/6. There are some implementation and design deficiencies
2804 that make it unreliable for some corner cases and the feature \fBshould not be
2805 used in production, only for evaluation or testing\fP\&. The power failure safety
2806 for metadata with RAID56 is not 100%.
2807 .SS Metadata
2808 .sp
2809 Do not use \fIraid5\fP nor \fIraid6\fP for metadata. Use \fIraid1\fP or \fIraid1c3\fP
2810 respectively.
2811 .sp
2812 The substitute profiles provide the same guarantees against loss of 1 or 2
2813 devices, and in some respect can be an improvement. Recovering from one
2814 missing device will only need to access the remaining 1st or 2nd copy, that in
2815 general may be stored on some other devices due to the way RAID1 works on
2816 btrfs, unlike on a striped profile (similar to \fIraid0\fP) that would need all
2817 devices all the time.
2818 .sp
2819 The space allocation pattern and consumption is different (e.g. on N devices):
2820 for \fIraid5\fP as an example, a 1GiB chunk is reserved on each device, while with
2821 \fIraid1\fP there\(aqs each 1GiB chunk stored on 2 devices. The consumption of each
2822 1GiB of used metadata is then \fIN * 1GiB\fP for vs \fI2 * 1GiB\fP\&. Using \fIraid1\fP
2823 is also more convenient for balancing/converting to other profile due to lower
2824 requirement on the available chunk space.
2825 .SS Missing/incomplete support
2826 .sp
2827 When RAID56 is on the same filesystem with different raid profiles, the space
2828 reporting is inaccurate, e.g. \fBdf\fP, \fBbtrfs filesystem df\fP or
2829 \fBbtrfs filesystem usage\fP\&. When there\(aqs only a one profile per block
2830 group type (e.g. RAID5 for data) the reporting is accurate.
2831 .sp
2832 When scrub is started on a RAID56 filesystem, it\(aqs started on all devices that
2833 degrade the performance. The workaround is to start it on each device
2834 separately. Due to that the device stats may not match the actual state and
2835 some errors might get reported multiple times.
2836 .sp
2837 The \fIwrite hole\fP problem. An unclean shutdown could leave a partially written
2838 stripe in a state where the some stripe ranges and the parity are from the old
2839 writes and some are new. The information which is which is not tracked. Write
2840 journal is not implemented. Alternatively a full read\-modify\-write would make
2841 sure that a full stripe is always written, avoiding the write hole completely,
2842 but performance in that case turned out to be too bad for use.
2843 .sp
2844 The striping happens on all available devices (at the time the chunks were
2845 allocated), so in case a new device is added it may not be utilized
2846 immediately and would require a rebalance. A fixed configured stripe width is
2847 not implemented.
2848 .SH GLOSSARY
2849 .sp
2850 Terms in \fIitalics\fP also appear in this glossary.
2851 .INDENT 0.0
2852 .TP
2853 .B allocator
2854 Usually \fIallocator\fP means the \fIblock\fP allocator, i.e. the logic
2855 inside the filesystem which decides where to place newly allocated blocks
2856 in order to maintain several constraints (like data locality, low
2857 fragmentation).
2858 .sp
2859 In btrfs, allocator may also refer to \fIchunk\fP allocator, i.e. the
2860 logic behind placing chunks on devices.
2861 .TP
2862 .B balance
2863 An operation that can be done to a btrfs filesystem, for example
2864 through \fBbtrfs balance /path\fP\&. A
2865 balance passes all data in the filesystem through the \fIallocator\fP
2866 again. It is primarily intended to rebalance the data in the filesystem
2867 across the \fIdevices\fP when a device is added or removed. A balance
2868 will regenerate missing copies for the redundant \fIRAID\fP levels, if a
2869 device has failed. As of Linux kernel 3.3, a balance operation can be
2870 made selective about which parts of the filesystem are rewritten
2871 using \fI\%filters\fP\&.
2872 .TP
2873 .B barrier
2874 An instruction to the underlying hardware to ensure that everything before
2875 the barrier is physically written to permanent storage before anything
2876 after it. Used in btrfs\(aqs \fIcopy on write\fP approach to ensure
2877 filesystem consistency.
2878 .TP
2879 .B block
2880 A single physically and logically contiguous piece of storage on a
2881 device, of size e.g. 4K. In some contexts also referred to as \fIsector\fP,
2882 though the term \fIblock\fP is preferred.
2883 .TP
2884 .B block group
2885 The unit of allocation of space in btrfs. A block group is laid out on
2886 the disk by the btrfs \fIallocator\fP, and will consist of one or more
2887 \fIchunks\fP, each stored on a different \fIdevice\fP\&. The number of chunks
2888 used in a block group will depend on its \fIRAID\fP level.
2889 .TP
2890 .B B\-tree
2891 The fundamental storage data structure used in btrfs. Except for the
2892 \fIsuperblocks\fP, all of btrfs \fImetadata\fP is stored in one of several
2893 B\-trees on disk. B\-trees store key/item pairs. While the same code is
2894 used to implement all of the B\-trees, there are a few different
2895 categories of B\-tree. The name \fIbtrfs\fP refers to its use of B\-trees.
2896 .TP
2897 .B btrfsck, fsck, btrfs\-check
2898 Tool in \fIbtrfs\-progs\fP that checks an unmounted filesystem (\fIoffline\fP)
2899 and reports on any errors in the filesystem structures it finds. By
2900 default the tool runs in read\-only mode as fixing errors is potentially
2901 dangerous. See also \fIscrub\fP\&.
2902 .TP
2903 .B btrfs\-progs
2904 User mode tools to manage btrfs\-specific features. Maintained at
2905 \X'tty: link http://github.com/kdave/btrfs-progs.git'\fI\%http://github.com/kdave/btrfs\-progs.git\fP\X'tty: link' . The main frontend to btrfs
2906 features is the standalone tool \fIbtrfs\fP, although
2907 other tools such as \fImkfs.btrfs\fP and \fIbtrfstune\fP are also part of
2908 btrfs\-progs.
2909 .TP
2910 .B chunk
2911 A part of a \fIblock group\fP\&. Chunks are either 1 GiB in size (for data)
2912 or 256 MiB (for \fImetadata\fP), depending on the overall filesystem size.
2913 .TP
2914 .B chunk tree
2915 A layer that keeps information about mapping between physical and
2916 logical block addresses. It\(aqs stored within the \fIsystem\fP group.
2917 .TP
2918 .B cleaner
2919 Usually referred to in context of deleted subvolumes. It\(aqs a background
2920 process that removes the actual data once a subvolume has been deleted.
2921 Cleaning can involve lots of IO and CPU activity depending on the
2922 fragmentation and amount of shared data with other subvolumes.
2923 .sp
2924 The cleaner kernel thread also processes defragmentation triggered by the
2925 \fIautodefrag\fP mount option, removing of empty blocks groups and some
2926 other finalization tasks.
2927 .TP
2928 .B copy\-on\-write, COW
2929 Also known as \fICOW\fP\&. The method that btrfs uses for modifying data.
2930 Instead of directly overwriting data in place, btrfs takes a copy of
2931 the data, alters it, and then writes the modified data back to a
2932 different (unused) location on the disk. It then updates the \fImetadata\fP
2933 to reflect the new location of the data. In order to update the
2934 metadata, the affected metadata blocks are also treated in the same
2935 way. In COW filesystems, files tend to fragment as they are modified.
2936 Copy\-on\-write is also used in the implementation of \fIsnapshots\fP and
2937 \fIreflink copies\fP\&. A copy\-on\-write filesystem is, in theory,
2938 \fIalways\fP consistent, provided the underlying hardware supports
2939 \fIbarriers\fP\&.
2940 .TP
2941 .B default subvolume
2942 The \fIsubvolume\fP in a btrfs filesystem which is mounted when mounting
2943 the filesystem without using the \fBsubvol=\fP mount option.
2944 .TP
2945 .B device
2946 A Linux block device, e.g. a whole disk, partition, LVM logical volume,
2947 loopback device, or network block device. A btrfs filesystem can reside
2948 on one or more devices.
2949 .TP
2950 .B df
2951 A standard Unix tool for reporting the amount of space used and free in
2952 a filesystem. The standard tool does not give accurate results, but the
2953 \fIbtrfs\fP command from \fIbtrfs\-progs\fP has
2954 an implementation of \fIdf\fP which shows space available in more detail. See
2955 the
2956 [[FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F|FAQ]]
2957 for a more detailed explanation of btrfs free space accounting.
2958 .TP
2959 .B DUP
2960 A form of \(dq\fIRAID\fP\(dq which stores two copies of each piece of data on
2961 the same \fIdevice\fP\&. This is similar to \fIRAID1\fP, and protects
2962 against \fIblock\fP\-level errors on the device, but does not provide any
2963 guarantees if the entire device fails. By default, btrfs uses \fIDUP\fP
2964 profile for metadata on single device filesystem.s
2965 .TP
2966 .B ENOSPC
2967 Error code returned by the OS to a user program when the filesystem
2968 cannot allocate enough data to fulfill the user request. In most
2969 filesystems, it indicates there is no free space available in the
2970 filesystem. Due to the additional space requirements from btrfs\(aqs
2971 \fICOW\fP behaviour, btrfs can sometimes return ENOSPC when there is
2972 apparently (in terms of \fIdf\fP) a large amount of space free. This is
2973 effectively a bug in btrfs, and (if it is repeatable), using the mount
2974 option \fBenospc_debug\fP may give a report
2975 that will help the btrfs developers. See the
2976 [[FAQ#if_your_device_is_large_.28.3E16GiB.29|FAQ entry]] on free space.
2977 .TP
2978 .B extent
2979 Contiguous sequence of bytes on disk that holds file data. It\(aqs a compact
2980 representation that tracks the start and length of the byte range, so the
2981 logic behind allocating blocks (\fIdelayed allocation\fP) strives for
2982 maximizing the length before writing the extents to the devices.
2983 .TP
2984 .B extent buffer
2985 An abstraction of a \fIb\-tree\fP metadata block storing item keys and item
2986 data. The underlying related structures are physical device block and a
2987 CPU memory page.
2988 .TP
2989 .B fallocate
2990 Command line tool in util\-linux, and a syscall, that reserves space in
2991 the filesystem for a file, without actually writing any file data to
2992 the filesystem. First data write will turn the preallocated extents
2993 into regular ones. See \X'tty: link https://man7.org/linux/man-pages/man1/fallocate.1.html'\fI\%fallocate(1)\fP\X'tty: link' and \X'tty: link https://man7.org/linux/man-pages/man2/fallocate.2.html'\fI\%fallocate(2)\fP\X'tty: link' manual pages
2994 for more details.
2995 .TP
2996 .B filefrag
2997 A tool to show the number of extents in a file, and hence the amount of
2998 fragmentation in the file. It is usually part of the e2fsprogs package
2999 on most Linux distributions. While initially developed for the ext2
3000 filesystem, it works on Btrfs as well. It uses the \fIFIEMAP\fP ioctl.
3001 .TP
3002 .B free space cache
3003 Also known as \(dqspace cache v1\(dq. A separate cache tracking free space as
3004 btrfs only tracks the allocated space. The free space is by definition
3005 any hole between allocated ranges. Finding the free ranges can be
3006 I/O intensive so the cache stores a condensed representation of it.
3007 It is updated every \fItransaction\fP commit.
3008 .sp
3009 The v1 free space cache has been superseded by free space tree.
3010 .TP
3011 .B free space tree
3012 Successor of \fIfree space cache\fP, also known as \(dqspace cache v2\(dq and now
3013 default. The free space is tracked in a better way and using COW
3014 unlike a custom mechanism of v1.
3015 .TP
3016 .B fsync
3017 On Unix and Unix\-like operating systems (of which Linux is the latter),
3018 the \X'tty: link https://man7.org/linux/man-pages/man2/fsync.2.html'\fI\%fsync(2)\fP\X'tty: link' system call causes all buffered file
3019 descriptor related data changes to be flushed to the underlying block
3020 device. When a file is modified on a modern operating system the
3021 changes are generally not written to the disk immediately but rather
3022 those changes are buffered in memory for performance reasons,
3023 calling \X'tty: link https://man7.org/linux/man-pages/man2/fsync.2.html'\fI\%fsync(2)\fP\X'tty: link' causes any in\-memory changes to be written
3024 to disk.
3025 .TP
3026 .B generation
3027 An internal counter which updates for each \fItransaction\fP\&. When a
3028 \fImetadata\fP block is written (using \fIcopy on write\fP), current
3029 generation is stored in the block, so that blocks which are too new
3030 (and hence possibly inconsistent) can be identified.
3031 .TP
3032 .B key
3033 A fixed sized tuple used to identify and sort items in a \fIB\-tree\fP\&.
3034 The key is broken up into 3 parts: \fIobjectid\fP, \fItype\fP, and
3035 \fIoffset\fP\&. The \fItype\fP field indicates how each of the other two
3036 fields should be used, and what to expect to find in the item.
3037 .TP
3038 .B item
3039 A variable sized structure stored in B\-tree leaves. Items hold
3040 different types of data depending on key type.
3041 .TP
3042 .B log tree
3043 A b\-tree that temporarily tracks ongoing metadata updates until a full
3044 transaction commit is done. It\(aqs a performance optimization of
3045 \fIfsync\fP\&. The log tracked in the tree are replayed if the filesystem
3046 is not unmounted cleanly.
3047 .TP
3048 .B metadata
3049 Data about data. In btrfs, this includes all of the internal data
3050 structures of the filesystem, including directory structures,
3051 filenames, file permissions, checksums, and the location of each file\(aqs
3052 \fIextents\fP\&. All btrfs metadata is stored in \fIB\-trees\fP\&.
3053 .TP
3054 .B mkfs.btrfs
3055 The tool (from \fIbtrfs\-progs\fP) to create a btrfs filesystem.
3056 .TP
3057 .B offline
3058 A filesystem which is not mounted is offline. Some tools (e.g.
3059 \fIbtrfsck\fP) will only work on offline filesystems. Compare \fIonline\fP\&.
3060 .TP
3061 .B online
3062 A filesystem which is mounted is online. Most btrfs tools will only
3063 work on online filesystems. Compare \fIoffline\fP\&.
3064 .TP
3065 .B orphan
3066 A file that\(aqs still in use (opened by a running process) but all
3067 directory entries of that file have been removed.
3068 .TP
3069 .B RAID
3070 A class of different methods for writing some additional redundant data
3071 across multiple \fIdevices\fP so that if one device fails, the missing
3072 data can be reconstructed from the remaining ones. See \fIRAID0\fP,
3073 \fIRAID1\fP, \fIRAID5\fP, \fIRAID6\fP, \fIRAID10\fP, \fIDUP\fP and
3074 \fIsingle\fP\&. Traditional RAID methods operate across multiple devices of
3075 equal size, whereas btrfs\(aq RAID implementation works inside \fIblock
3076 groups\fP\&.
3077 .TP
3078 .B RAID0
3079 A form of \fIRAID\fP which provides no guarantees of error recovery, but
3080 stripes a single copy of data across multiple devices for performance
3081 purposes. The stripe size is fixed to 64KB for now.
3082 .TP
3083 .B RAID1, RAID1C3, RAID1C4
3084 A form of \fIRAID\fP which stores two/three/four complete copies of each
3085 piece of data. Each copy is stored on a different \fIdevice\fP\&. btrfs
3086 requires a minimum of two devices to use RAID\-1 or three/four respectively.
3087 This is the default block group profile for btrfs\(aqs \fImetadata\fP on more
3088 than one device.
3089 .TP
3090 .B RAID5
3091 A form of \fIRAID\fP which stripes a single copy of data across multiple
3092 \fIdevices\fP, including one device\(aqs worth of additional parity data.
3093 Can be used to recover from a single device failure.
3094 .TP
3095 .B RAID6
3096 A form of \fIRAID\fP which stripes a single copy of data across multiple
3097 \fIdevices\fP, including two device\(aqs worth of additional parity data. Can
3098 be used to recover from the failure of two devices.
3099 .TP
3100 .B RAID10
3101 A form of \fIRAID\fP which stores two complete copies of each piece of
3102 data, and also stripes each copy across multiple devices for
3103 performance.
3104 .TP
3105 .B reflink
3106 Commonly used as a reference to a shallow copy of file extents that share
3107 the extents until the first change. Reflinked files (e.g. by the \fBcp\fP)
3108 are different files but point to the same extents, any change will be
3109 detected and new copy of the data created, keeping the files independent.
3110 Related to that is extent range cloning, that works on a range of a file.
3111 .TP
3112 .B relocation
3113 The process of moving block groups within the filesystem while
3114 maintaining full filesystem integrity and consistency. This
3115 functionality is underlying \fIbalance\fP and \fIdevice\fP removing features.
3116 .TP
3117 .B \fI\%scrub\fP
3118 An \fIonline\fP filesystem checking tool. Reads all the data and metadata
3119 on the filesystem, verifies \fIchecksums\fP and eventually uses redundant
3120 copies from \fIRAID\fP or \fIDUP\fP repair any corrupt data/metadata.
3121 .TP
3122 .B \fI\%seed device\fP
3123 A readonly device can be used as a filesystem seed or template (e.g. a
3124 CD\-ROM containing an OS image). Read/write devices can be added to
3125 store modifications (using \fIcopy on write\fP), changes to the writable
3126 devices are persistent across reboots. The original device remains
3127 unchanged and can be removed at any time (after Btrfs has been
3128 instructed to copy over all missing blocks). Multiple read/write file
3129 systems can be built from the same seed.
3130 .TP
3131 .B single
3132 A block group profile storing a single copy of each piece of data.
3133 .TP
3134 .B \fI\%snapshot\fP
3135 A \fIsubvolume\fP which is a \fIcopy on write\fP copy of another subvolume. The
3136 two subvolumes share all of their common (unmodified) data, which means
3137 that snapshots can be used to keep the historical state of a filesystem
3138 very cheaply. After the snapshot is made, the original subvolume and
3139 the snapshot are of equal status: the original does not \(dqown\(dq the
3140 snapshot, and either one can be deleted without affecting the other
3141 one.
3142 .TP
3143 .B \fI\%subvolume\fP
3144 A tree of files and directories inside a btrfs that can be mounted as
3145 if it were an independent filesystem. A subvolume is created by taking
3146 a reference on the root of another subvolume. Each btrfs filesystem has
3147 at least one subvolume, the \fItop\-level subvolume\fP, which contains
3148 everything else in the filesystem. Additional subvolumes can be created
3149 and deleted with the \fIbtrfs<\fP tool. All subvolumes share the same pool
3150 of free space in the filesystem. See also \fIdefault subvolume\fP\&.
3151 .TP
3152 .B super block
3153 A special metadata block that is a main access point of the filesystem
3154 structures. It\(aqs size is fixed and there are fixed locations on the devices
3155 used for detecting and opening the filesystem. Updating the superblock
3156 defines one \fItransaction\fP\&. The super blocks contains filesystem
3157 identification (UUID), checksum type, block pointers to fundamental
3158 trees, features and creation parameters.
3159 .TP
3160 .B system array
3161 A technical term for \fIsuper block\fP metadata describing how to assemble a
3162 filesystem from multiple device, storing information about chunks and devices that are
3163 required to be scanned/registered at the time the mount happens.
3164 Scanning is done by command \fBbtrfs device scan\fP, alternatively
3165 all the required devices can be specified by a mount option \fIdevice=/path\fP\&.
3166 .TP
3167 .B top\-level subvolume
3168 The \fIsubvolume\fP at the very top of the filesystem. This is the only
3169 subvolume present in a newly\-created btrfs filesystem, and internally has ID 5,
3170 otherwise could be referenced as 0 (e.g. within the \fIset\-default\fP subcommand of
3171 \fIbtrfs\fP).
3172 .TP
3173 .B transaction
3174 A consistent set of changes. To avoid generating very large amounts of
3175 disk activity, btrfs caches changes in RAM for up to 30 seconds
3176 (sometimes more often if the filesystem is running short on space or
3177 doing a lot of \fIfsync*s), and then writes (commits) these changes out
3178 to disk in one go (using *copy on write\fP behaviour). This period of
3179 caching is called a transaction. Only one transaction is active on the
3180 filesystem at any one time.
3181 .TP
3182 .B transid
3183 An alternative term for \fIgeneration\fP\&.
3184 .TP
3185 .B writeback
3186 \fIWriteback\fP in the context of the Linux kernel can be defined as the
3187 process of writing \(dqdirty\(dq memory from the page cache to the disk,
3188 when certain conditions are met (timeout, number of dirty pages over a
3189 ratio).
3190 .UNINDENT
3191 .SH STORAGE MODEL, HARDWARE CONSIDERATIONS
3192 .SS Storage model
3193 .sp
3194 \fIA storage model is a model that captures key physical aspects of data
3195 structure in a data store. A filesystem is the logical structure organizing
3196 data on top of the storage device.\fP
3197 .sp
3198 The filesystem assumes several features or limitations of the storage device
3199 and utilizes them or applies measures to guarantee reliability. BTRFS in
3200 particular is based on a COW (copy on write) mode of writing, i.e. not updating
3201 data in place but rather writing a new copy to a different location and then
3202 atomically switching the pointers.
3203 .sp
3204 In an ideal world, the device does what it promises. The filesystem assumes
3205 that this may not be true so additional mechanisms are applied to either detect
3206 misbehaving hardware or get valid data by other means. The devices may (and do)
3207 apply their own detection and repair mechanisms but we won\(aqt assume any.
3208 .sp
3209 The following assumptions about storage devices are considered (sorted by
3210 importance, numbers are for further reference):
3211 .INDENT 0.0
3212 .IP 1. 3
3213 atomicity of reads and writes of blocks/sectors (the smallest unit of data
3214 the device presents to the upper layers)
3215 .IP 2. 3
3216 there\(aqs a flush command that instructs the device to forcibly order writes
3217 before and after the command; alternatively there\(aqs a barrier command that
3218 facilitates the ordering but may not flush the data
3219 .IP 3. 3
3220 data sent to write to a given device offset will be written without further
3221 changes to the data and to the offset
3222 .IP 4. 3
3223 writes can be reordered by the device, unless explicitly serialized by the
3224 flush command
3225 .IP 5. 3
3226 reads and writes can be freely reordered and interleaved
3227 .UNINDENT
3228 .sp
3229 The consistency model of BTRFS builds on these assumptions. The logical data
3230 updates are grouped, into a generation, written on the device, serialized by
3231 the flush command and then the super block is written ending the generation.
3232 All logical links among metadata comprising a consistent view of the data may
3233 not cross the generation boundary.
3234 .SS When things go wrong
3235 .sp
3236 \fBNo or partial atomicity of block reads/writes (1)\fP
3237 .INDENT 0.0
3238 .IP \(bu 2
3239 \fIProblem\fP: a partial block contents is written (\fItorn write\fP), e.g. due to a
3240 power glitch or other electronics failure during the read/write
3241 .IP \(bu 2
3242 \fIDetection\fP: checksum mismatch on read
3243 .IP \(bu 2
3244 \fIRepair\fP: use another copy or rebuild from multiple blocks using some encoding
3245 scheme
3246 .UNINDENT
3247 .sp
3248 \fBThe flush command does not flush (2)\fP
3249 .sp
3250 This is perhaps the most serious problem and impossible to mitigate by
3251 filesystem without limitations and design restrictions. What could happen in
3252 the worst case is that writes from one generation bleed to another one, while
3253 still letting the filesystem consider the generations isolated. Crash at any
3254 point would leave data on the device in an inconsistent state without any hint
3255 what exactly got written, what is missing and leading to stale metadata link
3256 information.
3257 .sp
3258 Devices usually honor the flush command, but for performance reasons may do
3259 internal caching, where the flushed data are not yet persistently stored. A
3260 power failure could lead to a similar scenario as above, although it\(aqs less
3261 likely that later writes would be written before the cached ones. This is
3262 beyond what a filesystem can take into account. Devices or controllers are
3263 usually equipped with batteries or capacitors to write the cache contents even
3264 after power is cut. (\fIBattery backed write cache\fP)
3265 .sp
3266 \fBData get silently changed on write (3)\fP
3267 .sp
3268 Such thing should not happen frequently, but still can happen spuriously due
3269 the complex internal workings of devices or physical effects of the storage
3270 media itself.
3271 .INDENT 0.0
3272 .IP \(bu 2
3273 \fIProblem\fP: while the data are written atomically, the contents get changed
3274 .IP \(bu 2
3275 \fIDetection\fP: checksum mismatch on read
3276 .IP \(bu 2
3277 \fIRepair\fP: use another copy or rebuild from multiple blocks using some
3278 encoding scheme
3279 .UNINDENT
3280 .sp
3281 \fBData get silently written to another offset (3)\fP
3282 .sp
3283 This would be another serious problem as the filesystem has no information
3284 when it happens. For that reason the measures have to be done ahead of time.
3285 This problem is also commonly called \fIghost write\fP\&.
3286 .sp
3287 The metadata blocks have the checksum embedded in the blocks, so a correct
3288 atomic write would not corrupt the checksum. It\(aqs likely that after reading
3289 such block the data inside would not be consistent with the rest. To rule that
3290 out there\(aqs embedded block number in the metadata block. It\(aqs the logical
3291 block number because this is what the logical structure expects and verifies.
3292 .sp
3293 The following is based on information publicly available, user feedback,
3294 community discussions or bug report analyses. It\(aqs not complete and further
3295 research is encouraged when in doubt.
3296 .SS Main memory
3297 .sp
3298 The data structures and raw data blocks are temporarily stored in computer
3299 memory before they get written to the device. It is critical that memory is
3300 reliable because even simple bit flips can have vast consequences and lead to
3301 damaged structures, not only in the filesystem but in the whole operating
3302 system.
3303 .sp
3304 Based on experience in the community, memory bit flips are more common than one
3305 would think. When it happens, it\(aqs reported by the tree\-checker or by a checksum
3306 mismatch after reading blocks. There are some very obvious instances of bit
3307 flips that happen, e.g. in an ordered sequence of keys in metadata blocks. We can
3308 easily infer from the other data what values get damaged and how. However, fixing
3309 that is not straightforward and would require cross\-referencing data from the
3310 entire filesystem to see the scope.
3311 .sp
3312 If available, ECC memory should lower the chances of bit flips, but this
3313 type of memory is not available in all cases. A memory test should be performed
3314 in case there\(aqs a visible bit flip pattern, though this may not detect a faulty
3315 memory module because the actual load of the system could be the factor making
3316 the problems appear. In recent years attacks on how the memory modules operate
3317 have been demonstrated (\fIrowhammer\fP) achieving specific bits to be flipped.
3318 While these were targeted, this shows that a series of reads or writes can
3319 affect unrelated parts of memory.
3320 .sp
3321 Block group profiles with redundancy (like RAID1) will not protect against
3322 memory errors as the blocks are first stored in memory before they are written
3323 to the devices from the same source.
3324 .sp
3325 A filesystem mounted read\-only will not affect the underlying block device in
3326 almost 100% (with highly unlikely exceptions). The exception is a tree\-log that
3327 needs to be replayed during mount (and before the read\-only mount takes place),
3328 working memory is needed for that and that can be affected by bit flips.
3329 There\(aqs a theoretical case where bit flip changes the filesystem status from
3330 read\-only to read\-write.
3331 .sp
3332 Further reading:
3333 .INDENT 0.0
3334 .IP \(bu 2
3335 \X'tty: link https://en.wikipedia.org/wiki/Row_hammer'\fI\%https://en.wikipedia.org/wiki/Row_hammer\fP\X'tty: link'
3336 .IP \(bu 2
3337 memory overclocking, XMP, potential risks
3338 .UNINDENT
3339 .sp
3340 What to do:
3341 .INDENT 0.0
3342 .IP \(bu 2
3343 run \fImemtest\fP, note that sometimes memory errors happen only when the system
3344 is under heavy load that the default memtest cannot trigger
3345 .IP \(bu 2
3346 memory errors may appear as filesystem going read\-only due to \(dqpre write\(dq
3347 check, that verify meta data before they get written but fail some basic
3348 consistency checks
3349 .IP \(bu 2
3350 newly built systems should be tested before being put to production use,
3351 ideally start a IO/CPU load that will be run on such system later; namely
3352 systems that will utilize overclocking or special performance features
3353 .UNINDENT
3354 .SS Direct memory access (DMA)
3355 .sp
3356 Another class of errors is related to DMA (direct memory access) performed
3357 by device drivers. While this could be considered a software error, the
3358 data transfers that happen without CPU assistance may accidentally corrupt
3359 other pages. Storage devices utilize DMA for performance reasons, the
3360 filesystem structures and data pages are passed back and forth, making
3361 errors possible in case page life time is not properly tracked.
3362 .sp
3363 There are lots of quirks (device\-specific workarounds) in Linux kernel
3364 drivers (regarding not only DMA) that are added when found. The quirks
3365 may avoid specific errors or disable some features to avoid worse problems.
3366 .sp
3367 What to do:
3368 .INDENT 0.0
3369 .IP \(bu 2
3370 use up\-to\-date kernel (recent releases or maintained long term support versions)
3371 .IP \(bu 2
3372 as this may be caused by faulty drivers, keep the systems up\-to\-date
3373 .UNINDENT
3374 .SS Rotational disks (HDD)
3375 .sp
3376 Rotational HDDs typically fail at the level of individual sectors or small clusters.
3377 Read failures are caught on the levels below the filesystem and are returned to
3378 the user as \fIEIO \- Input/output error\fP\&. Reading the blocks repeatedly may
3379 return the data eventually, but this is better done by specialized tools and
3380 filesystem takes the result of the lower layers. Rewriting the sectors may
3381 trigger internal remapping but this inevitably leads to data loss.
3382 .sp
3383 Disk firmware is technically software but from the filesystem perspective is
3384 part of the hardware. IO requests are processed, and caching or various
3385 other optimizations are performed, which may lead to bugs under high load or
3386 unexpected physical conditions or unsupported use cases.
3387 .sp
3388 Disks are connected by cables with two ends, both of which can cause problems
3389 when not attached properly. Data transfers are protected by checksums and the
3390 lower layers try hard to transfer the data correctly or not at all. The errors
3391 from badly\-connecting cables may manifest as large amount of failed read or
3392 write requests, or as short error bursts depending on physical conditions.
3393 .sp
3394 What to do:
3395 .INDENT 0.0
3396 .IP \(bu 2
3397 check \fBsmartctl\fP for potential issues
3398 .UNINDENT
3399 .SS Solid state drives (SSD)
3400 .sp
3401 The mechanism of information storage is different from HDDs and this affects
3402 the failure mode as well. The data are stored in cells grouped in large blocks
3403 with limited number of resets and other write constraints. The firmware tries
3404 to avoid unnecessary resets and performs optimizations to maximize the storage
3405 media lifetime. The known techniques are deduplication (blocks with same
3406 fingerprint/hash are mapped to same physical block), compression or internal
3407 remapping and garbage collection of used memory cells. Due to the additional
3408 processing there are measures to verify the data e.g. by ECC codes.
3409 .sp
3410 The observations of failing SSDs show that the whole electronic fails at once
3411 or affects a lot of data (e.g. stored on one chip). Recovering such data
3412 may need specialized equipment and reading data repeatedly does not help as
3413 it\(aqs possible with HDDs.
3414 .sp
3415 There are several technologies of the memory cells with different
3416 characteristics and price. The lifetime is directly affected by the type and
3417 frequency of data written. Writing \(dqtoo much\(dq distinct data (e.g. encrypted)
3418 may render the internal deduplication ineffective and lead to a lot of rewrites
3419 and increased wear of the memory cells.
3420 .sp
3421 There are several technologies and manufacturers so it\(aqs hard to describe them
3422 but there are some that exhibit similar behaviour:
3423 .INDENT 0.0
3424 .IP \(bu 2
3425 expensive SSD will use more durable memory cells and is optimized for
3426 reliability and high load
3427 .IP \(bu 2
3428 cheap SSD is projected for a lower load (\(dqdesktop user\(dq) and is optimized for
3429 cost, it may employ the optimizations and/or extended error reporting
3430 partially or not at all
3431 .UNINDENT
3432 .sp
3433 It\(aqs not possible to reliably determine the expected lifetime of an SSD due to
3434 lack of information about how it works or due to lack of reliable stats provided
3435 by the device.
3436 .sp
3437 Metadata writes tend to be the biggest component of lifetime writes to a SSD,
3438 so there is some value in reducing them. Depending on the device class (high
3439 end/low end) the features like DUP block group profiles may affect the
3440 reliability in both ways:
3441 .INDENT 0.0
3442 .IP \(bu 2
3443 \fIhigh end\fP are typically more reliable and using \fIsingle\fP for data and
3444 metadata could be suitable to reduce device wear
3445 .IP \(bu 2
3446 \fIlow end\fP could lack ability to identify errors so an additional redundancy
3447 at the filesystem level (checksums, \fIDUP\fP) could help
3448 .UNINDENT
3449 .sp
3450 Only users who consume 50 to 100% of the SSD\(aqs actual lifetime writes need to be
3451 concerned by the write amplification of btrfs DUP metadata. Most users will be
3452 far below 50% of the actual lifetime, or will write the drive to death and
3453 discover how many writes 100% of the actual lifetime was. SSD firmware often
3454 adds its own write multipliers that can be arbitrary and unpredictable and
3455 dependent on application behavior, and these will typically have far greater
3456 effect on SSD lifespan than DUP metadata. It\(aqs more or less impossible to
3457 predict when a SSD will run out of lifetime writes to within a factor of two, so
3458 it\(aqs hard to justify wear reduction as a benefit.
3459 .sp
3460 Further reading:
3461 .INDENT 0.0
3462 .IP \(bu 2
3463 \X'tty: link https://www.snia.org/educational-library/ssd-and-deduplication-end-spinning-disk-2012'\fI\%https://www.snia.org/educational\-library/ssd\-and\-deduplication\-end\-spinning\-disk\-2012\fP\X'tty: link'
3464 .IP \(bu 2
3465 \X'tty: link https://www.snia.org/educational-library/realities-solid-state-storage-2013-2013'\fI\%https://www.snia.org/educational\-library/realities\-solid\-state\-storage\-2013\-2013\fP\X'tty: link'
3466 .IP \(bu 2
3467 \X'tty: link https://www.snia.org/educational-library/ssd-performance-primer-2013'\fI\%https://www.snia.org/educational\-library/ssd\-performance\-primer\-2013\fP\X'tty: link'
3468 .IP \(bu 2
3469 \X'tty: link https://www.snia.org/educational-library/how-controllers-maximize-ssd-life-2013'\fI\%https://www.snia.org/educational\-library/how\-controllers\-maximize\-ssd\-life\-2013\fP\X'tty: link'
3470 .UNINDENT
3471 .sp
3472 What to do:
3473 .INDENT 0.0
3474 .IP \(bu 2
3475 run \fBsmartctl\fP or self\-tests to look for potential issues
3476 .IP \(bu 2
3477 keep the firmware up\-to\-date
3478 .UNINDENT
3479 .SS NVM express, non\-volatile memory (NVMe)
3480 .sp
3481 NVMe is a type of persistent memory usually connected over a system bus (PCIe)
3482 or similar interface and the speeds are an order of magnitude faster than SSD.
3483 It is also a non\-rotating type of storage, and is not typically connected by a
3484 cable. It\(aqs not a SCSI type device either but rather a complete specification
3485 for logical device interface.
3486 .sp
3487 In a way the errors could be compared to a combination of SSD class and regular
3488 memory. Errors may exhibit as random bit flips or IO failures. There are tools
3489 to access the internal log (\fBnvme log\fP and \fBnvme\-cli\fP) for a more detailed
3490 analysis.
3491 .sp
3492 There are separate error detection and correction steps performed e.g. on the
3493 bus level and in most cases never making in to the filesystem level. Once this
3494 happens it could mean there\(aqs some systematic error like overheating or bad
3495 physical connection of the device. You may want to run self\-tests (using
3496 \fBsmartctl\fP).
3497 .INDENT 0.0
3498 .IP \(bu 2
3499 \X'tty: link https://en.wikipedia.org/wiki/NVM_Express'\fI\%https://en.wikipedia.org/wiki/NVM_Express\fP\X'tty: link'
3500 .IP \(bu 2
3501 \X'tty: link https://www.smartmontools.org/wiki/NVMe_Support'\fI\%https://www.smartmontools.org/wiki/NVMe_Support\fP\X'tty: link'
3502 .UNINDENT
3503 .SS Drive firmware
3504 .sp
3505 Firmware is technically still software but embedded into the hardware. As all
3506 software has bugs, so does firmware. Storage devices can update the firmware
3507 and fix known bugs. In some cases the it\(aqs possible to avoid certain bugs by
3508 quirks (device\-specific workarounds) in Linux kernel.
3509 .sp
3510 A faulty firmware can cause wide range of corruptions from small and localized
3511 to large affecting lots of data. Self\-repair capabilities may not be sufficient.
3512 .sp
3513 What to do:
3514 .INDENT 0.0
3515 .IP \(bu 2
3516 check for firmware updates in case there are known problems, note that
3517 updating firmware can be risky on itself
3518 .IP \(bu 2
3519 use up\-to\-date kernel (recent releases or maintained long term support versions)
3520 .UNINDENT
3521 .SS SD flash cards
3522 .sp
3523 There are a lot of devices with low power consumption and thus using storage
3524 media based on low power consumption too, typically flash memory stored on
3525 a chip enclosed in a detachable card package. An improperly inserted card may be
3526 damaged by electrical spikes when the device is turned on or off. The chips
3527 storing data in turn may be damaged permanently. All types of flash memory
3528 have a limited number of rewrites, so the data are internally translated by FTL
3529 (flash translation layer). This is implemented in firmware (technically a
3530 software) and prone to bugs that manifest as hardware errors.
3531 .sp
3532 Adding redundancy like using DUP profiles for both data and metadata can help
3533 in some cases but a full backup might be the best option once problems appear
3534 and replacing the card could be required as well.
3535 .SS Hardware as the main source of filesystem corruptions
3536 .sp
3537 \fBIf you use unreliable hardware and don\(aqt know about that, don\(aqt blame the
3538 filesystem when it tells you.\fP
3539 .SH SEE ALSO
3540 .sp
3541 \X'tty: link https://man7.org/linux/man-pages/man5/acl.5.html'\fI\%acl(5)\fP\X'tty: link',
3542 \fI\%btrfs(8)\fP,
3543 \X'tty: link https://man7.org/linux/man-pages/man1/chattr.1.html'\fI\%chattr(1)\fP\X'tty: link',
3544 \X'tty: link https://man7.org/linux/man-pages/man8/fstrim.8.html'\fI\%fstrim(8)\fP\X'tty: link',
3545 \X'tty: link https://man7.org/linux/man-pages/man2/ioctl.2.html'\fI\%ioctl(2)\fP\X'tty: link',
3546 \fI\%btrfs\-ioctl(2)\fP,
3547 \fI\%mkfs.btrfs(8)\fP,
3548 \X'tty: link https://man7.org/linux/man-pages/man8/mount.8.html'\fI\%mount(8)\fP\X'tty: link',
3549 \X'tty: link https://man7.org/linux/man-pages/man8/swapon.8.html'\fI\%swapon(8)\fP\X'tty: link'
3550 .\" Generated by docutils manpage writer.
3551 .