| 1 |
'\" t |
| 2 |
.\" Man page generated from reStructuredText |
| 3 |
.\" by the Docutils 0.22.4 manpage writer. |
| 4 |
. |
| 5 |
. |
| 6 |
.nr rst2man-indent-level 0 |
| 7 |
. |
| 8 |
.de1 rstReportMargin |
| 9 |
\\$1 \\n[an-margin] |
| 10 |
level \\n[rst2man-indent-level] |
| 11 |
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] |
| 12 |
- |
| 13 |
\\n[rst2man-indent0] |
| 14 |
\\n[rst2man-indent1] |
| 15 |
\\n[rst2man-indent2] |
| 16 |
.. |
| 17 |
.de1 INDENT |
| 18 |
.\" .rstReportMargin pre: |
| 19 |
. RS \\$1 |
| 20 |
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] |
| 21 |
. nr rst2man-indent-level +1 |
| 22 |
.\" .rstReportMargin post: |
| 23 |
.. |
| 24 |
.de UNINDENT |
| 25 |
. RE |
| 26 |
.\" indent \\n[an-margin] |
| 27 |
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] |
| 28 |
.nr rst2man-indent-level -1 |
| 29 |
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] |
| 30 |
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u |
| 31 |
.. |
| 32 |
.TH "BTRFS" "5" "May 10, 2026" "7.0" "BTRFS" |
| 33 |
.SH NAME |
| 34 |
btrfs \- topics about the BTRFS filesystem (mount options, supported file attributes and other) |
| 35 |
.SH DESCRIPTION |
| 36 |
.sp |
| 37 |
This document describes topics related to BTRFS that are not specific to the |
| 38 |
tools. Currently covers: |
| 39 |
.INDENT 0.0 |
| 40 |
.IP 1. 4 |
| 41 |
mount options |
| 42 |
.IP 2. 4 |
| 43 |
filesystem features |
| 44 |
.IP 3. 4 |
| 45 |
swapfile support |
| 46 |
.IP 4. 4 |
| 47 |
checksum algorithms |
| 48 |
.IP 5. 4 |
| 49 |
compression |
| 50 |
.IP 6. 4 |
| 51 |
sysfs interface |
| 52 |
.IP 7. 4 |
| 53 |
filesystem exclusive operations |
| 54 |
.IP 8. 4 |
| 55 |
filesystem limits |
| 56 |
.IP 9. 4 |
| 57 |
bootloader support |
| 58 |
.IP 10. 4 |
| 59 |
file attributes |
| 60 |
.IP 11. 4 |
| 61 |
zoned mode |
| 62 |
.IP 12. 4 |
| 63 |
control device |
| 64 |
.IP 13. 4 |
| 65 |
filesystems with multiple block group profiles |
| 66 |
.IP 14. 4 |
| 67 |
seeding device |
| 68 |
.IP 15. 4 |
| 69 |
RAID56 status and recommended practices |
| 70 |
.IP 16. 4 |
| 71 |
glossary |
| 72 |
.IP 17. 4 |
| 73 |
storage model, hardware considerations |
| 74 |
.UNINDENT |
| 75 |
.SH MOUNT OPTIONS |
| 76 |
.SS BTRFS SPECIFIC MOUNT OPTIONS |
| 77 |
.sp |
| 78 |
This section describes mount options specific to BTRFS. For the generic mount |
| 79 |
options please refer to mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> manual page and also see the section |
| 80 |
with BTRFS specifics below \%<btrfs-man5//#\:mount-options-generic>\&. The options are |
| 81 |
sorted alphabetically (discarding the \fIno\fP prefix). |
| 82 |
.sp |
| 83 |
\fBNote:\fP |
| 84 |
.INDENT 0.0 |
| 85 |
.INDENT 3.5 |
| 86 |
Most mount options apply to the whole filesystem and only options in the |
| 87 |
first mounted subvolume will take effect. This is due to lack of implementation |
| 88 |
and may change in the future. This means that (for example) you can\(aqt set |
| 89 |
per\-subvolume \fInodatacow\fP, \fInodatasum\fP, or \fIcompress\fP using mount options. This |
| 90 |
should eventually be fixed, but it has proved to be difficult to implement |
| 91 |
correctly within the Linux VFS framework. |
| 92 |
.UNINDENT |
| 93 |
.UNINDENT |
| 94 |
.sp |
| 95 |
Mount options are processed in order, only the last occurrence of an option |
| 96 |
takes effect and may disable other options due to constraints (see e.g. |
| 97 |
\fInodatacow\fP and \fIcompress\fP). The output of \fBmount\fP command shows which options |
| 98 |
have been applied. |
| 99 |
.INDENT 0.0 |
| 100 |
.TP |
| 101 |
.B acl, noacl |
| 102 |
(default: on) |
| 103 |
.sp |
| 104 |
Enable/disable support for POSIX Access Control Lists (ACLs). See the |
| 105 |
acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html> manual page for more information about ACLs. |
| 106 |
.sp |
| 107 |
The support for ACL is build\-time configurable (BTRFS_FS_POSIX_ACL) and |
| 108 |
mount fails if \fIacl\fP is requested but the feature is not compiled in. |
| 109 |
.UNINDENT |
| 110 |
.INDENT 0.0 |
| 111 |
.TP |
| 112 |
.B autodefrag, noautodefrag |
| 113 |
(since: 3.0, default: off) |
| 114 |
.sp |
| 115 |
Enable automatic file defragmentation. |
| 116 |
When enabled, small random writes into files (in a range of tens of kilobytes, |
| 117 |
currently it\(aqs 64KiB) are detected and queued up for the defragmentation process. |
| 118 |
May not be well suited for large database workloads. |
| 119 |
.sp |
| 120 |
The read latency may increase due to reading the adjacent blocks that make up the |
| 121 |
range for defragmentation, successive write will merge the blocks in the new |
| 122 |
location. |
| 123 |
.sp |
| 124 |
\fBWarning:\fP |
| 125 |
.INDENT 7.0 |
| 126 |
.INDENT 3.5 |
| 127 |
Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14\-rc2 as |
| 128 |
well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or |
| 129 |
≥ 3.13.4 will break up the reflinks of COW data (for example files |
| 130 |
copied with \fBcp \-\-reflink\fP, snapshots or de\-duplicated data). |
| 131 |
This may cause considerable increase of space usage depending on the |
| 132 |
broken up reflinks. |
| 133 |
.UNINDENT |
| 134 |
.UNINDENT |
| 135 |
.TP |
| 136 |
.B barrier, nobarrier |
| 137 |
(default: on) |
| 138 |
.sp |
| 139 |
Ensure that all IO write operations make it through the device cache and are stored |
| 140 |
permanently when the filesystem is at its consistency checkpoint. This |
| 141 |
typically means that a flush command is sent to the device that will |
| 142 |
synchronize all pending data and ordinary metadata blocks, then writes the |
| 143 |
superblock and issues another flush. |
| 144 |
.sp |
| 145 |
The write flushes incur a slight hit and also prevent the IO block |
| 146 |
scheduler to reorder requests in a more effective way. Disabling barriers gets |
| 147 |
rid of that penalty but will most certainly lead to a corrupted filesystem in |
| 148 |
case of a crash or power loss. The ordinary metadata blocks could be yet |
| 149 |
unwritten at the time the new superblock is stored permanently, expecting that |
| 150 |
the block pointers to metadata were stored permanently before. |
| 151 |
.sp |
| 152 |
On a device with a volatile battery\-backed write\-back cache, the \fInobarrier\fP |
| 153 |
option will not lead to filesystem corruption as the pending blocks are |
| 154 |
supposed to make it to the permanent storage. |
| 155 |
.TP |
| 156 |
.B clear_cache |
| 157 |
Force clearing and rebuilding of the free space cache if something |
| 158 |
has gone wrong. |
| 159 |
This option only takes effect on the first read\-write mount. |
| 160 |
.sp |
| 161 |
For free space cache \fIv1\fP, this only clears (and, unless \fInospace_cache\fP is |
| 162 |
used, rebuilds) the free space cache for block groups that are modified while |
| 163 |
the filesystem is mounted with that option. To actually clear an entire free |
| 164 |
space cache \fIv1\fP, see \fBbtrfs check \-\-clear\-space\-cache v1\fP\&. |
| 165 |
.sp |
| 166 |
For free space cache \fIv2\fP, this clears the entire free space cache. |
| 167 |
To do so without requiring to mounting the filesystem, see |
| 168 |
\fBbtrfs check \-\-clear\-space\-cache v2\fP\&. |
| 169 |
.sp |
| 170 |
See also: \fIspace_cache\fP\&. |
| 171 |
.TP |
| 172 |
.B commit=<seconds> |
| 173 |
(since: 3.12, default: 30) |
| 174 |
.sp |
| 175 |
Set the interval of periodic transaction commit when data are synchronized |
| 176 |
to permanent storage. Higher interval values lead to larger amount of unwritten |
| 177 |
data to accumulate in memory, which has obvious consequences when the |
| 178 |
system crashes. The upper bound is not forced, but a warning is |
| 179 |
printed if it\(aqs more than 300 seconds (5 minutes). Use with care. |
| 180 |
.sp |
| 181 |
The periodic commit is not the only mechanism to do the transaction commit, |
| 182 |
this can also happen by explicit \fBsync\fP or indirectly by other |
| 183 |
commands that affect the global filesystem state or internal kernel |
| 184 |
mechanisms that flush based on various thresholds or policies (e.g. cgroups). |
| 185 |
.TP |
| 186 |
.B compress, compress=<type[:level]>, compress\-force, compress\-force=<type[:level]> |
| 187 |
(default: off, level support since: 5.1) |
| 188 |
.sp |
| 189 |
Control BTRFS file data compression. Type may be specified as \fIzlib\fP, |
| 190 |
\fIlzo\fP, \fIzstd\fP or \fIno\fP (for no compression, used for remounting). If no type |
| 191 |
is specified, \fIzlib\fP is used. If \fIcompress\-force\fP is specified, |
| 192 |
then compression will always be attempted, but the data may end up uncompressed |
| 193 |
if the compression would make them larger. |
| 194 |
.sp |
| 195 |
Both \fIzlib\fP and \fIzstd\fP (since version 5.1) expose the compression level as a |
| 196 |
tunable knob with higher levels trading speed and memory (\fIzstd\fP) for higher |
| 197 |
compression ratios. This can be set by appending a colon and the desired level. |
| 198 |
ZLIB accepts the range [1, 9] and ZSTD accepts [1, 15]. If no level is set, |
| 199 |
both currently use a default level of 3. The value 0 is an alias for the |
| 200 |
default level. |
| 201 |
.sp |
| 202 |
Otherwise some simple heuristics are applied to detect an incompressible file. |
| 203 |
If the first blocks written to a file are not compressible, the whole file is |
| 204 |
permanently marked to skip compression. As this is too simple, the |
| 205 |
\fIcompress\-force\fP is a workaround that will compress most of the files at the |
| 206 |
cost of some wasted CPU cycles on failed attempts. |
| 207 |
Since kernel 4.15, a set of heuristic algorithms have been improved by using |
| 208 |
frequency sampling, repeated pattern detection and Shannon entropy calculation |
| 209 |
to avoid that. |
| 210 |
.sp |
| 211 |
\fBNote:\fP |
| 212 |
.INDENT 7.0 |
| 213 |
.INDENT 3.5 |
| 214 |
If compression is enabled, \fInodatacow\fP and \fInodatasum\fP are disabled. |
| 215 |
.UNINDENT |
| 216 |
.UNINDENT |
| 217 |
.TP |
| 218 |
.B datacow, nodatacow |
| 219 |
(default: on) |
| 220 |
.sp |
| 221 |
Enable data copy\-on\-write for newly created files. |
| 222 |
\fINodatacow\fP implies \fInodatasum\fP, and disables \fIcompression\fP\&. All files created |
| 223 |
under \fInodatacow\fP are also set the NOCOW file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>). |
| 224 |
.sp |
| 225 |
\fBNote:\fP |
| 226 |
.INDENT 7.0 |
| 227 |
.INDENT 3.5 |
| 228 |
If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled. |
| 229 |
.UNINDENT |
| 230 |
.UNINDENT |
| 231 |
.sp |
| 232 |
Updates in\-place improve performance for workloads that do frequent overwrites, |
| 233 |
at the cost of potential partial writes, in case the write is interrupted |
| 234 |
(system crash, device failure). |
| 235 |
.TP |
| 236 |
.B datasum, nodatasum |
| 237 |
(default: on) |
| 238 |
.sp |
| 239 |
Enable data checksumming for newly created files. |
| 240 |
\fIDatasum\fP implies \fIdatacow\fP, i.e. the normal mode of operation. All files created |
| 241 |
under \fInodatasum\fP inherit the \(dqno checksums\(dq property, however there\(aqs no |
| 242 |
corresponding file attribute (see chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>). |
| 243 |
.sp |
| 244 |
\fBNote:\fP |
| 245 |
.INDENT 7.0 |
| 246 |
.INDENT 3.5 |
| 247 |
If \fInodatacow\fP or \fInodatasum\fP are enabled, compression is disabled. |
| 248 |
.UNINDENT |
| 249 |
.UNINDENT |
| 250 |
.sp |
| 251 |
There is a slight performance gain when checksums are turned off, the |
| 252 |
corresponding metadata blocks holding the checksums do not need to updated. |
| 253 |
The cost of checksumming of the blocks in memory is much lower than the IO, |
| 254 |
modern CPUs feature hardware support of the checksumming algorithm. |
| 255 |
.UNINDENT |
| 256 |
.INDENT 0.0 |
| 257 |
.TP |
| 258 |
.B degraded |
| 259 |
(default: off) |
| 260 |
.sp |
| 261 |
Allow mounts with fewer devices than the RAID profile constraints |
| 262 |
require. A read\-write mount (or remount) may fail when there are too many devices |
| 263 |
missing, for example if a stripe member is completely missing from RAID0. |
| 264 |
.sp |
| 265 |
Since 4.14, the constraint checks have been improved and are verified on the |
| 266 |
chunk level, not at the device level. This allows degraded mounts of |
| 267 |
filesystems with mixed RAID profiles for data and metadata, even if the |
| 268 |
device number constraints would not be satisfied for some of the profiles. |
| 269 |
.sp |
| 270 |
Example: metadata \-\- raid1, data \-\- single, devices \-\- \fB/dev/sda\fP, \fB/dev/sdb\fP |
| 271 |
.sp |
| 272 |
Suppose the data are completely stored on \fIsda\fP, then missing \fIsdb\fP will not |
| 273 |
prevent the mount, even if 1 missing device would normally prevent (any) |
| 274 |
\fIsingle\fP profile to mount. In case some of the data chunks are stored on \fIsdb\fP, |
| 275 |
then the constraint of single/data is not satisfied and the filesystem |
| 276 |
cannot be mounted. |
| 277 |
.UNINDENT |
| 278 |
.INDENT 0.0 |
| 279 |
.TP |
| 280 |
.B device=<devicepath> |
| 281 |
Specify a path to a device that will be scanned for BTRFS filesystem during |
| 282 |
mount. This is usually done automatically by a device manager (like udev) or |
| 283 |
using the \fBbtrfs device scan\fP command (e.g. run from the initial ramdisk). In |
| 284 |
cases where this is not possible the \fIdevice\fP mount option can help. |
| 285 |
.sp |
| 286 |
\fBNote:\fP |
| 287 |
.INDENT 7.0 |
| 288 |
.INDENT 3.5 |
| 289 |
Booting e.g. a RAID1 system may fail even if all filesystem\(aqs \fIdevice\fP |
| 290 |
paths are provided as the actual device nodes may not be discovered by the |
| 291 |
system at that point. |
| 292 |
.UNINDENT |
| 293 |
.UNINDENT |
| 294 |
.TP |
| 295 |
.B discard, discard=sync, discard=async, nodiscard |
| 296 |
(default: async when devices support it since 6.2, async support since: 5.6) |
| 297 |
.sp |
| 298 |
Enable discarding of freed file blocks. This is useful for SSD/NVMe |
| 299 |
devices, thinly provisioned LUNs, or virtual machine images; however, |
| 300 |
every storage layer must support discard for it to work. |
| 301 |
.sp |
| 302 |
In the synchronous mode (\fIsync\fP or without option value), lack of asynchronous |
| 303 |
queued TRIM on the backing device TRIM can severely degrade performance, |
| 304 |
because a synchronous TRIM operation will be attempted instead. Queued TRIM |
| 305 |
requires SATA devices with chipsets revision newer than 3.1 and devices. |
| 306 |
.sp |
| 307 |
The asynchronous mode (\fIasync\fP) gathers extents in larger chunks before sending |
| 308 |
them to the devices for TRIM. The overhead and performance impact should be |
| 309 |
negligible compared to the previous mode and it\(aqs supposed to be the preferred |
| 310 |
mode if needed. |
| 311 |
.sp |
| 312 |
If it is not necessary to immediately discard freed blocks, then the \fBfstrim\fP |
| 313 |
tool can be used to discard all free blocks in a batch. Scheduling a TRIM |
| 314 |
during a period of low system activity will prevent latent interference with |
| 315 |
the performance of other operations. Also, a device may ignore the TRIM command |
| 316 |
if the range is too small, so running a batch discard has a greater probability |
| 317 |
of actually discarding the blocks. |
| 318 |
.TP |
| 319 |
.B enospc_debug, noenospc_debug |
| 320 |
(default: off) |
| 321 |
.sp |
| 322 |
Enable verbose output for some ENOSPC conditions. It\(aqs safe to use but can |
| 323 |
be noisy if the system reaches near\-full state. |
| 324 |
.TP |
| 325 |
.B fatal_errors=<action> |
| 326 |
(since: 3.4, default: bug) |
| 327 |
.sp |
| 328 |
Action to take when encountering a fatal error. |
| 329 |
.INDENT 7.0 |
| 330 |
.TP |
| 331 |
.B bug |
| 332 |
\fIBUG()\fP on a fatal error, the system will stay in the crashed state and may be |
| 333 |
still partially usable, but reboot is required for full operation |
| 334 |
.TP |
| 335 |
.B panic |
| 336 |
\fIpanic()\fP on a fatal error, depending on other system configuration, this may |
| 337 |
be followed by a reboot. Please refer to the documentation of kernel boot |
| 338 |
parameters, e.g. \fIpanic\fP, \fIoops\fP or \fIcrashkernel\fP\&. |
| 339 |
.UNINDENT |
| 340 |
.TP |
| 341 |
.B flushoncommit, noflushoncommit |
| 342 |
(default: off) |
| 343 |
.sp |
| 344 |
This option forces any data dirtied by a write in a prior transaction to commit |
| 345 |
as part of the current commit, effectively a full filesystem sync. |
| 346 |
.sp |
| 347 |
This makes the committed state a fully consistent view of the file system from |
| 348 |
the application\(aqs perspective (i.e. it includes all completed file system |
| 349 |
operations). This was previously the behavior only when a snapshot was |
| 350 |
created. |
| 351 |
.sp |
| 352 |
When off, the filesystem is consistent but buffered writes may last more than |
| 353 |
one transaction commit. |
| 354 |
.TP |
| 355 |
.B fragment=<type> |
| 356 |
(depends on compile\-time option CONFIG_BTRFS_DEBUG, since: 4.4, default: off) |
| 357 |
.sp |
| 358 |
A debugging helper to intentionally fragment given \fItype\fP of block groups. The |
| 359 |
type can be \fIdata\fP, \fImetadata\fP or \fIall\fP\&. This mount option should not be used |
| 360 |
outside of debugging environments and is not recognized if the kernel config |
| 361 |
option \fICONFIG_BTRFS_DEBUG\fP is not enabled. |
| 362 |
.TP |
| 363 |
.B nologreplay |
| 364 |
(default: off, even read\-only) |
| 365 |
.sp |
| 366 |
The tree\-log contains pending updates to the filesystem until the full commit. |
| 367 |
The log is replayed on next mount, this can be disabled by this option. See |
| 368 |
also \fItreelog\fP\&. Note that \fInologreplay\fP is the same as \fInorecovery\fP\&. |
| 369 |
.sp |
| 370 |
\fBWarning:\fP |
| 371 |
.INDENT 7.0 |
| 372 |
.INDENT 3.5 |
| 373 |
Currently, the tree log is replayed even with a read\-only mount! To |
| 374 |
disable that behaviour, mount also with \fInologreplay\fP\&. |
| 375 |
.UNINDENT |
| 376 |
.UNINDENT |
| 377 |
.TP |
| 378 |
.B max_inline=<bytes> |
| 379 |
(default: min(2048, page size) ) |
| 380 |
.sp |
| 381 |
Specify the maximum amount of space, that can be inlined in |
| 382 |
a metadata b\-tree leaf. The value is specified in bytes, optionally |
| 383 |
with a K suffix (case insensitive). In practice, this value |
| 384 |
is limited by the filesystem block size (named \fIsectorsize\fP at mkfs time), |
| 385 |
and memory page size of the system. In case of sectorsize limit, there\(aqs |
| 386 |
some space unavailable due to b\-tree leaf headers. For example, a 4KiB |
| 387 |
sectorsize, maximum size of inline data is about 3900 bytes. |
| 388 |
.sp |
| 389 |
Inlining can be completely turned off by specifying 0. This will increase data |
| 390 |
block slack if file sizes are much smaller than block size but will reduce |
| 391 |
metadata consumption in return. |
| 392 |
.sp |
| 393 |
\fBNote:\fP |
| 394 |
.INDENT 7.0 |
| 395 |
.INDENT 3.5 |
| 396 |
The default value has changed to 2048 in kernel 4.6. |
| 397 |
.UNINDENT |
| 398 |
.UNINDENT |
| 399 |
.TP |
| 400 |
.B metadata_ratio=<value> |
| 401 |
(default: 0, internal logic) |
| 402 |
.sp |
| 403 |
Specifies that 1 metadata chunk should be allocated after every \fIvalue\fP data |
| 404 |
chunks. Default behaviour depends on internal logic, some percent of unused |
| 405 |
metadata space is attempted to be maintained but is not always possible if |
| 406 |
there\(aqs not enough space left for chunk allocation. The option could be useful to |
| 407 |
override the internal logic in favor of the metadata allocation if the expected |
| 408 |
workload is supposed to be metadata intense (snapshots, reflinks, xattrs, |
| 409 |
inlined files). |
| 410 |
.TP |
| 411 |
.B norecovery |
| 412 |
(since: 4.5, default: off) |
| 413 |
.sp |
| 414 |
Do not attempt any data recovery at mount time. This will disable \fIlogreplay\fP |
| 415 |
and avoids other write operations. Note that this option is the same as |
| 416 |
\fInologreplay\fP\&. |
| 417 |
.sp |
| 418 |
\fBNote:\fP |
| 419 |
.INDENT 7.0 |
| 420 |
.INDENT 3.5 |
| 421 |
The opposite option \fIrecovery\fP used to have different meaning but was |
| 422 |
changed for consistency with other filesystems, where \fInorecovery\fP is used for |
| 423 |
skipping log replay. BTRFS does the same and in general will try to avoid any |
| 424 |
write operations. |
| 425 |
.UNINDENT |
| 426 |
.UNINDENT |
| 427 |
.UNINDENT |
| 428 |
.INDENT 0.0 |
| 429 |
.TP |
| 430 |
.B rescan_uuid_tree |
| 431 |
(since: 3.12, default: off) |
| 432 |
.sp |
| 433 |
Force check and rebuild procedure of the UUID tree. This should not |
| 434 |
normally be needed. Alternatively the tree can be cleared from |
| 435 |
userspace by command btrfs rescue clear\-uuid\-tree \%<#\:man-rescue-clear-uuid-tree> |
| 436 |
and then it will be automatically rebuilt in kernel (the mount option |
| 437 |
is not needed in that case). |
| 438 |
.TP |
| 439 |
.B rescue |
| 440 |
(since: 5.9) |
| 441 |
.sp |
| 442 |
Modes allowing mount with damaged filesystem structures, all requires |
| 443 |
the filesystem to be mounted read\-only and doesn\(aqt allow remount to read\-write. |
| 444 |
This is supposed to provide unified and more fine grained tuning of |
| 445 |
errors that affect filesystem operation. |
| 446 |
.INDENT 7.0 |
| 447 |
.IP \(bu 2 |
| 448 |
\fIusebackuproot\fP (since 5.9) |
| 449 |
.sp |
| 450 |
Try to use backup root slots inside super block. |
| 451 |
Replaces standalone option \fIusebackuproot\fP |
| 452 |
.IP \(bu 2 |
| 453 |
\fInologreplay\fP (since 5.9) |
| 454 |
.sp |
| 455 |
Do not replay any dirty logs. |
| 456 |
Replaces standalone option \fInologreplay\fP |
| 457 |
.IP \(bu 2 |
| 458 |
\fIignorebadroots\fP, \fIibadroots\fP (since: 5.11) |
| 459 |
.sp |
| 460 |
Ignore bad tree roots, greatly improve the chance for data salvage. |
| 461 |
.IP \(bu 2 |
| 462 |
\fIignoredatacsums\fP, \fIidatacsums\fP (since: 5.11) |
| 463 |
.sp |
| 464 |
Ignore data checksum verification. |
| 465 |
.IP \(bu 2 |
| 466 |
\fIignoremetacsums\fP, \fIimetacsums\fP (since 6.12) |
| 467 |
.sp |
| 468 |
Ignore metadata checksum verification, useful for interrupted checksum conversion. |
| 469 |
.IP \(bu 2 |
| 470 |
\fIall\fP (since: 5.9) |
| 471 |
.sp |
| 472 |
Enable all supported rescue options. |
| 473 |
.UNINDENT |
| 474 |
.TP |
| 475 |
.B skip_balance |
| 476 |
(since: 3.3, default: off) |
| 477 |
.sp |
| 478 |
Skip automatic resume of an interrupted balance operation. The operation can |
| 479 |
later be resumed with \fBbtrfs balance resume\fP, or the paused state can be |
| 480 |
removed with \fBbtrfs balance cancel\fP\&. The default behaviour is to resume an |
| 481 |
interrupted balance immediately after the filesystem is mounted. |
| 482 |
.TP |
| 483 |
.B space_cache, space_cache=<version>, nospace_cache |
| 484 |
(\fInospace_cache\fP since: 3.2, \fIspace_cache=v1\fP and \fIspace_cache=v2\fP since 4.5, default: \fIspace_cache=v2\fP) |
| 485 |
.sp |
| 486 |
Options to control the free space cache. The free space cache greatly improves |
| 487 |
performance when reading block group free space into memory. However, managing |
| 488 |
the space cache consumes some resources, including a small amount of disk |
| 489 |
space. |
| 490 |
.sp |
| 491 |
There are two implementations of the free space cache. The original |
| 492 |
one, referred to as \fIv1\fP, used to be a safe default but has been |
| 493 |
superseded by \fIv2\fP\&. The \fIv1\fP space cache can be disabled at mount time |
| 494 |
with \fInospace_cache\fP without clearing. |
| 495 |
.sp |
| 496 |
On very large filesystems (many terabytes) and certain workloads, the |
| 497 |
performance of the \fIv1\fP space cache may degrade drastically. The \fIv2\fP |
| 498 |
implementation, which adds a new b\-tree called the free space tree, addresses |
| 499 |
this issue. Once enabled, the \fIv2\fP space cache will always be used and cannot |
| 500 |
be disabled unless it is cleared. Use \fIclear_cache,space_cache=v1\fP or |
| 501 |
\fIclear_cache,nospace_cache\fP to do so. If \fIv2\fP is enabled, and \fIv1\fP space |
| 502 |
cache will be cleared (at the first mount) and kernels without \fIv2\fP |
| 503 |
support will only be able to mount the filesystem in read\-only mode. |
| 504 |
On an unmounted filesystem the caches (both versions) can be cleared by |
| 505 |
\(dqbtrfs check \-\-clear\-space\-cache\(dq. |
| 506 |
.sp |
| 507 |
The btrfs\-check(8) \%<> and \fI:doc:\(gamkfs.btrfs\fP commands have full \fIv2\fP free space |
| 508 |
cache support since v4.19. |
| 509 |
.sp |
| 510 |
If a version is not explicitly specified, the default implementation will be |
| 511 |
chosen, which is \fIv2\fP\&. |
| 512 |
.TP |
| 513 |
.B ssd, ssd_spread, nossd, nossd_spread |
| 514 |
(default: SSD autodetected) |
| 515 |
.sp |
| 516 |
Options to control SSD allocation schemes. By default, BTRFS will |
| 517 |
enable or disable SSD optimizations depending on status of a device with |
| 518 |
respect to rotational or non\-rotational type. This is determined by the |
| 519 |
contents of \fB/sys/block/DEV/queue/rotational\fP). If it is 0, the |
| 520 |
\fIssd\fP option is turned on. The option \fInossd\fP will disable the |
| 521 |
autodetection. |
| 522 |
.sp |
| 523 |
The optimizations make use of the absence of the seek penalty that\(aqs inherent |
| 524 |
for the rotational devices. The blocks can be typically written faster and |
| 525 |
are not offloaded to separate threads. |
| 526 |
.sp |
| 527 |
\fBNote:\fP |
| 528 |
.INDENT 7.0 |
| 529 |
.INDENT 3.5 |
| 530 |
Since 4.14, the block layout optimizations have been dropped. This used |
| 531 |
to help with first generations of SSD devices. Their FTL (flash translation |
| 532 |
layer) was not effective and the optimization was supposed to improve the wear |
| 533 |
by better aligning blocks. This is no longer true with modern SSD devices and |
| 534 |
the optimization had no real benefit. Furthermore it caused increased |
| 535 |
fragmentation. The layout tuning has been kept intact for the option |
| 536 |
\fIssd_spread\fP\&. |
| 537 |
.UNINDENT |
| 538 |
.UNINDENT |
| 539 |
.sp |
| 540 |
The \fIssd_spread\fP mount option attempts to allocate into bigger and aligned |
| 541 |
chunks of unused space, and may perform better on low\-end SSDs. \fIssd_spread\fP |
| 542 |
implies \fIssd\fP, enabling all other SSD heuristics as well. The option \fInossd\fP |
| 543 |
will disable all SSD options while \fInossd_spread\fP only disables \fIssd_spread\fP\&. |
| 544 |
.TP |
| 545 |
.B subvol=<path> |
| 546 |
Mount subvolume from \fIpath\fP rather than the toplevel subvolume. The |
| 547 |
\fIpath\fP is always treated as relative to the toplevel subvolume. |
| 548 |
This mount option overrides the default subvolume set for the given filesystem. |
| 549 |
.TP |
| 550 |
.B subvolid=<subvolid> |
| 551 |
Mount subvolume specified by a \fIsubvolid\fP number rather than the toplevel |
| 552 |
subvolume. You can use \fBbtrfs subvolume list\fP of \fBbtrfs subvolume show\fP to see |
| 553 |
subvolume ID numbers. |
| 554 |
This mount option overrides the default subvolume set for the given filesystem. |
| 555 |
.sp |
| 556 |
\fBNote:\fP |
| 557 |
.INDENT 7.0 |
| 558 |
.INDENT 3.5 |
| 559 |
If both \fIsubvolid\fP and \fIsubvol\fP are specified, they must point at the |
| 560 |
same subvolume, otherwise the mount will fail. |
| 561 |
.UNINDENT |
| 562 |
.UNINDENT |
| 563 |
.TP |
| 564 |
.B thread_pool=<number> |
| 565 |
(default: min(NRCPUS + 2, 8) ) |
| 566 |
.sp |
| 567 |
The number of worker threads to start. NRCPUS is number of on\-line CPUs |
| 568 |
detected at the time of mount. Small number leads to less parallelism in |
| 569 |
processing data and metadata, higher numbers could lead to a performance hit |
| 570 |
due to increased locking contention, process scheduling, cache\-line bouncing or |
| 571 |
costly data transfers between local CPU memories. |
| 572 |
.TP |
| 573 |
.B treelog, notreelog |
| 574 |
(default: on) |
| 575 |
.sp |
| 576 |
Enable the tree logging used for \fIfsync\fP and \fIO_SYNC\fP writes. The tree log |
| 577 |
stores changes without the need of a full filesystem sync. The log operations |
| 578 |
are flushed at sync and transaction commit. If the system crashes between two |
| 579 |
such syncs, the pending tree log operations are replayed during mount. |
| 580 |
.sp |
| 581 |
\fBWarning:\fP |
| 582 |
.INDENT 7.0 |
| 583 |
.INDENT 3.5 |
| 584 |
Currently, the tree log is replayed even with a read\-only mount! To |
| 585 |
disable that behaviour, also mount with \fInologreplay\fP\&. |
| 586 |
.UNINDENT |
| 587 |
.UNINDENT |
| 588 |
.sp |
| 589 |
The tree log could contain new files/directories, these would not exist on |
| 590 |
a mounted filesystem if the log is not replayed. |
| 591 |
.TP |
| 592 |
.B usebackuproot |
| 593 |
(since: 4.6, default: off) |
| 594 |
.sp |
| 595 |
Enable autorecovery attempts if a bad tree root is found at mount time. |
| 596 |
Currently this scans a backup list of several previous tree roots and tries to |
| 597 |
use the first readable. This can be used with read\-only mounts as well. |
| 598 |
.sp |
| 599 |
\fBNote:\fP |
| 600 |
.INDENT 7.0 |
| 601 |
.INDENT 3.5 |
| 602 |
This option has replaced \fIrecovery\fP which has been deprecated. |
| 603 |
.UNINDENT |
| 604 |
.UNINDENT |
| 605 |
.TP |
| 606 |
.B user_subvol_rm_allowed |
| 607 |
(default: off) |
| 608 |
.sp |
| 609 |
Allow subvolumes to be deleted by their respective owner. Otherwise, only the |
| 610 |
root user can do that. |
| 611 |
.sp |
| 612 |
\fBNote:\fP |
| 613 |
.INDENT 7.0 |
| 614 |
.INDENT 3.5 |
| 615 |
Historically, any user could create a snapshot even if he was not owner |
| 616 |
of the source subvolume, the subvolume deletion has been restricted for that |
| 617 |
reason. The subvolume creation has been restricted but this mount option is |
| 618 |
still required. This is a usability issue. |
| 619 |
Since 4.18, the rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an |
| 620 |
ordinary directory. Whether this is possible can be detected at runtime, see |
| 621 |
\fIrmdir_subvol\fP feature in \fIFILESYSTEM FEATURES\fP\&. |
| 622 |
.UNINDENT |
| 623 |
.UNINDENT |
| 624 |
.UNINDENT |
| 625 |
.SS DEPRECATED MOUNT OPTIONS |
| 626 |
.sp |
| 627 |
List of mount options that have been removed, kept for backward compatibility. |
| 628 |
.INDENT 0.0 |
| 629 |
.TP |
| 630 |
.B recovery |
| 631 |
(since: 3.2, default: off, deprecated since: 4.5) |
| 632 |
.sp |
| 633 |
\fBNote:\fP |
| 634 |
.INDENT 7.0 |
| 635 |
.INDENT 3.5 |
| 636 |
This option has been replaced by \fIusebackuproot\fP and should not be used |
| 637 |
but will work on 4.5+ kernels. |
| 638 |
.UNINDENT |
| 639 |
.UNINDENT |
| 640 |
.TP |
| 641 |
.B inode_cache, noinode_cache |
| 642 |
(removed in: 5.11, since: 3.0, default: off) |
| 643 |
.sp |
| 644 |
\fBNote:\fP |
| 645 |
.INDENT 7.0 |
| 646 |
.INDENT 3.5 |
| 647 |
The functionality has been removed in 5.11, any stale data created by |
| 648 |
previous use of the \fIinode_cache\fP option can be removed by |
| 649 |
btrfs rescue clear\-ino\-cache \%<#\:man-rescue-clear-ino-cache>\&. |
| 650 |
.UNINDENT |
| 651 |
.UNINDENT |
| 652 |
.TP |
| 653 |
.B check_int, check_int_data, check_int_print_mask=<value> |
| 654 |
(removed in: 6.7, since: 3.0, default: off) |
| 655 |
.sp |
| 656 |
These debugging options control the behavior of the integrity checking |
| 657 |
module (the BTRFS_FS_CHECK_INTEGRITY config option required). The main goal is |
| 658 |
to verify that all blocks from a given transaction period are properly linked. |
| 659 |
.sp |
| 660 |
\fIcheck_int\fP enables the integrity checker module, which examines all |
| 661 |
block write requests to ensure on\-disk consistency, at a large |
| 662 |
memory and CPU cost. |
| 663 |
.sp |
| 664 |
\fIcheck_int_data\fP includes extent data in the integrity checks, and |
| 665 |
implies the \fIcheck_int\fP option. |
| 666 |
.sp |
| 667 |
\fIcheck_int_print_mask\fP takes a bit mask of BTRFSIC_PRINT_MASK_* values |
| 668 |
as defined in \fIfs/btrfs/check\-integrity.c\fP, to control the integrity |
| 669 |
checker module behavior. |
| 670 |
.sp |
| 671 |
See comments at the top of \fIfs/btrfs/check\-integrity.c\fP |
| 672 |
for more information. |
| 673 |
.UNINDENT |
| 674 |
.SS NOTES ON GENERIC MOUNT OPTIONS |
| 675 |
.sp |
| 676 |
Some of the general mount options from mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html> that affect BTRFS and are |
| 677 |
worth mentioning. |
| 678 |
.INDENT 0.0 |
| 679 |
.TP |
| 680 |
.B context |
| 681 |
The context refers to the SELinux contexts and policy definitions passed |
| 682 |
as mount options. This works properly since version v6.8 (because the |
| 683 |
mount option parser of BTRFS was ported to new API that also understood |
| 684 |
the options). |
| 685 |
.TP |
| 686 |
.B noatime |
| 687 |
under read intensive work\-loads, specifying \fInoatime\fP significantly improves |
| 688 |
performance because no new access time information needs to be written. Without |
| 689 |
this option, the default is \fIrelatime\fP, which only reduces the number of |
| 690 |
inode atime updates in comparison to the traditional \fIstrictatime\fP\&. The worst |
| 691 |
case for atime updates under \fIrelatime\fP occurs when many files are read whose |
| 692 |
atime is older than 24 h and which are freshly snapshotted. In that case the |
| 693 |
atime is updated and COW happens \- for each file \- in bulk. See also |
| 694 |
\%<https://\:lwn\:.net/\:Articles/\:499293/> \- \fIAtime and btrfs: a bad combination? (LWN, 2012\-05\-31)\fP\&. |
| 695 |
.sp |
| 696 |
Note that \fInoatime\fP may break applications that rely on atime uptimes like |
| 697 |
the venerable Mutt (unless you use \fImaildir\fP mailboxes). |
| 698 |
.UNINDENT |
| 699 |
.SH FILESYSTEM FEATURES |
| 700 |
.sp |
| 701 |
The basic set of filesystem features gets extended over time. The backward |
| 702 |
compatibility is maintained and the features are optional, need to be |
| 703 |
explicitly asked for so accidental use will not create incompatibilities. |
| 704 |
.sp |
| 705 |
There are several classes and the respective tools to manage the features: |
| 706 |
.INDENT 0.0 |
| 707 |
.TP |
| 708 |
.B at mkfs time only |
| 709 |
This is namely for core structures, like the b\-tree nodesize or checksum |
| 710 |
algorithm, see mkfs.btrfs(8) \%<> for more details. |
| 711 |
.TP |
| 712 |
.B after mkfs, on an unmounted filesystem |
| 713 |
Features that may optimize internal structures or add new structures to support |
| 714 |
new functionality, see btrfstune(8) \%<>\&. The command |
| 715 |
\fBbtrfs inspect\-internal dump\-super /dev/sdx\fP |
| 716 |
will dump a superblock, you can map the value of |
| 717 |
\fIincompat_flags\fP to the features listed below |
| 718 |
.TP |
| 719 |
.B after mkfs, on a mounted filesystem |
| 720 |
The features of a filesystem (with a given UUID) are listed in |
| 721 |
\fB/sys/fs/btrfs/UUID/features/\fP, one file per feature. The status is stored |
| 722 |
inside the file. The value \fI1\fP is for enabled and active, while \fI0\fP means the |
| 723 |
feature was enabled at mount time but turned off afterwards. |
| 724 |
.sp |
| 725 |
Whether a particular feature can be turned on a mounted filesystem can be found |
| 726 |
in the directory \fB/sys/fs/btrfs/features/\fP, one file per feature. The value \fI1\fP |
| 727 |
means the feature can be enabled. |
| 728 |
.UNINDENT |
| 729 |
.sp |
| 730 |
List of features (see also mkfs.btrfs(8) \%<> section |
| 731 |
FILESYSTEM FEATURES \%<#\:man-mkfs-filesystem-features>): |
| 732 |
.INDENT 0.0 |
| 733 |
.TP |
| 734 |
.B big_metadata |
| 735 |
(since: 3.4) |
| 736 |
.sp |
| 737 |
the filesystem uses \fInodesize\fP for metadata blocks, this can be bigger than the |
| 738 |
page size |
| 739 |
.TP |
| 740 |
.B block_group_tree |
| 741 |
(since: 6.1) |
| 742 |
.sp |
| 743 |
block group item representation using a dedicated b\-tree, this can greatly |
| 744 |
reduce mount time for large filesystems |
| 745 |
.TP |
| 746 |
.B compress_lzo |
| 747 |
(since: 2.6.38) |
| 748 |
.sp |
| 749 |
the \fIlzo\fP compression has been used on the filesystem, either as a mount option |
| 750 |
or via \fBbtrfs filesystem defrag\fP\&. |
| 751 |
.TP |
| 752 |
.B compress_zstd |
| 753 |
(since: 4.14) |
| 754 |
.sp |
| 755 |
the \fIzstd\fP compression has been used on the filesystem, either as a mount option |
| 756 |
or via \fBbtrfs filesystem defrag\fP\&. |
| 757 |
.TP |
| 758 |
.B default_subvol |
| 759 |
(since: 2.6.34) |
| 760 |
.sp |
| 761 |
the default subvolume has been set on the filesystem |
| 762 |
.TP |
| 763 |
.B extended_iref |
| 764 |
(since: 3.7) |
| 765 |
.sp |
| 766 |
increased hardlink limit per file in a directory to 65536, older kernels |
| 767 |
supported a varying number of hardlinks depending on the sum of all file name |
| 768 |
sizes that can be stored into one metadata block |
| 769 |
.TP |
| 770 |
.B free_space_tree |
| 771 |
(since: 4.5) |
| 772 |
.sp |
| 773 |
free space representation using a dedicated b\-tree, successor of v1 space cache |
| 774 |
.TP |
| 775 |
.B metadata_uuid |
| 776 |
(since: 5.0) |
| 777 |
.sp |
| 778 |
the main filesystem UUID is the metadata_uuid, which stores the new UUID only |
| 779 |
in the superblock while all metadata blocks still have the UUID set at mkfs |
| 780 |
time, see btrfstune(8) \%<> for more |
| 781 |
.TP |
| 782 |
.B mixed_backref |
| 783 |
(since: 2.6.31) |
| 784 |
.sp |
| 785 |
the last major disk format change, improved backreferences, now default |
| 786 |
.TP |
| 787 |
.B mixed_groups |
| 788 |
(since: 2.6.37) |
| 789 |
.sp |
| 790 |
mixed data and metadata block groups, i.e. the data and metadata are not |
| 791 |
separated and occupy the same block groups, this mode is suitable for small |
| 792 |
volumes as there are no constraints how the remaining space should be used |
| 793 |
(compared to the split mode, where empty metadata space cannot be used for data |
| 794 |
and vice versa) |
| 795 |
.sp |
| 796 |
on the other hand, the final layout is quite unpredictable and possibly highly |
| 797 |
fragmented, which means worse performance |
| 798 |
.TP |
| 799 |
.B no_holes |
| 800 |
(since: 3.14) |
| 801 |
.sp |
| 802 |
improved representation of file extents where holes are not explicitly |
| 803 |
stored as an extent, saves a few percent of metadata if sparse files are used |
| 804 |
.TP |
| 805 |
.B raid1c34 |
| 806 |
(since: 5.5) |
| 807 |
.sp |
| 808 |
extended RAID1 mode with copies on 3 or 4 devices respectively |
| 809 |
.TP |
| 810 |
.B raid_stripe_tree |
| 811 |
(since: 6.7) |
| 812 |
.sp |
| 813 |
a separate tree for tracking file extents on RAID profiles |
| 814 |
.TP |
| 815 |
.B RAID56 |
| 816 |
(since: 3.9) |
| 817 |
.sp |
| 818 |
the filesystem contains or contained a RAID56 profile of block groups |
| 819 |
.TP |
| 820 |
.B rmdir_subvol |
| 821 |
(since: 4.18) |
| 822 |
.sp |
| 823 |
indicate that rmdir(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:rmdir\:.2\:.html> syscall can delete an empty subvolume just like an |
| 824 |
ordinary directory. Note that this feature only depends on the kernel version. |
| 825 |
.TP |
| 826 |
.B skinny_metadata |
| 827 |
(since: 3.10) |
| 828 |
.sp |
| 829 |
reduced\-size metadata for extent references, saves a few percent of metadata |
| 830 |
.TP |
| 831 |
.B send_stream_version |
| 832 |
(since: 5.10) |
| 833 |
.sp |
| 834 |
number of the highest supported send stream version |
| 835 |
.TP |
| 836 |
.B simple_quota |
| 837 |
(since: 6.7) |
| 838 |
.sp |
| 839 |
simplified quota accounting |
| 840 |
.TP |
| 841 |
.B supported_checksums |
| 842 |
(since: 5.5) |
| 843 |
.sp |
| 844 |
list of checksum algorithms supported by the kernel module, the respective |
| 845 |
modules or built\-in implementing the algorithms need to be present to mount |
| 846 |
the filesystem, see section CHECKSUM ALGORITHMS\&. |
| 847 |
.TP |
| 848 |
.B supported_sectorsizes |
| 849 |
(since: 5.13) |
| 850 |
.sp |
| 851 |
list of values that are accepted as sector sizes (\fBmkfs.btrfs \-\-sectorsize\fP) by |
| 852 |
the running kernel |
| 853 |
.TP |
| 854 |
.B supported_rescue_options |
| 855 |
(since: 5.11) |
| 856 |
.sp |
| 857 |
list of values for the mount option \fIrescue\fP that are supported by the running |
| 858 |
kernel, see btrfs(5) \%<> |
| 859 |
.TP |
| 860 |
.B zoned |
| 861 |
(since: 5.12) |
| 862 |
.sp |
| 863 |
zoned mode is allocation/write friendly to host\-managed zoned devices, |
| 864 |
allocation space is partitioned into fixed\-size zones that must be updated |
| 865 |
sequentially, see section ZONED MODE |
| 866 |
.UNINDENT |
| 867 |
.SH SWAPFILE SUPPORT |
| 868 |
.sp |
| 869 |
A swapfile, when active, is a file\-backed swap area. It is supported since kernel 5.0. |
| 870 |
Use swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html> to activate it, until then (respectively again after deactivating it |
| 871 |
with swapoff(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapoff\:.8\:.html>) it\(aqs just a normal file (with NODATACOW set), for which the special |
| 872 |
restrictions for active swapfiles don\(aqt apply. |
| 873 |
.sp |
| 874 |
There are some limitations of the implementation in BTRFS and Linux swap |
| 875 |
subsystem: |
| 876 |
.INDENT 0.0 |
| 877 |
.IP \(bu 2 |
| 878 |
filesystem \- must be only single device |
| 879 |
.IP \(bu 2 |
| 880 |
filesystem \- must have only \fIsingle\fP data profile |
| 881 |
.IP \(bu 2 |
| 882 |
subvolume \- cannot be snapshotted if it contains any active swapfiles |
| 883 |
.IP \(bu 2 |
| 884 |
swapfile \- must be preallocated (i.e. no holes) |
| 885 |
.IP \(bu 2 |
| 886 |
swapfile \- must be NODATACOW (i.e. also NODATASUM, no compression) |
| 887 |
.UNINDENT |
| 888 |
.sp |
| 889 |
The limitations come namely from the COW\-based design and mapping layer of |
| 890 |
blocks that allows the advanced features like relocation and multi\-device |
| 891 |
filesystems. However, the swap subsystem expects simpler mapping and no |
| 892 |
background changes of the file block location once they\(aqve been assigned to |
| 893 |
swap. The constraints mentioned above (single device and single profile) are |
| 894 |
related to the swapfile itself, i.e. the extents and their placement. It is |
| 895 |
possible to create swapfile on multi\-device filesystem as long as the extents |
| 896 |
are on one device but this cannot be affected by user and depends on free space |
| 897 |
fragmentation and available unused space for new chunks. |
| 898 |
.sp |
| 899 |
With active swapfiles, the following whole\-filesystem operations will skip |
| 900 |
some block groups or may fail: |
| 901 |
.INDENT 0.0 |
| 902 |
.IP \(bu 2 |
| 903 |
balance \- block groups with extents of any active swapfiles are skipped and |
| 904 |
reported through dmesg, the rest will be processed normally |
| 905 |
.IP \(bu 2 |
| 906 |
scrub \- ditto |
| 907 |
.IP \(bu 2 |
| 908 |
resize grow \- unaffected |
| 909 |
.IP \(bu 2 |
| 910 |
resize shrink \- works as long as the extents of any active swapfiles are |
| 911 |
outside of the shrunk range |
| 912 |
.IP \(bu 2 |
| 913 |
device add \- if the new devices do not interfere with any already active swapfiles |
| 914 |
this operation will work, though no new swapfile can be activated |
| 915 |
afterwards |
| 916 |
.IP \(bu 2 |
| 917 |
device delete \- if the device has been added as above, it can be also deleted |
| 918 |
.IP \(bu 2 |
| 919 |
device replace \- ditto |
| 920 |
.UNINDENT |
| 921 |
.sp |
| 922 |
The above limitations will significantly affect btrfs maintenance routines, |
| 923 |
namely balance and scrub. Balance and scrub will skip those block groups which |
| 924 |
have extents of an active swapfile on them, and those block groups may also |
| 925 |
contain extents belonging to other files. This means swapfiles will have an |
| 926 |
amplified impact, much larger than the size of the swapfile itself. |
| 927 |
.sp |
| 928 |
For balance, it means data block groups may not be properly balanced, causing |
| 929 |
unbalanced data/metadata usage, which can lead to early ENOSPC errors. |
| 930 |
.sp |
| 931 |
For scrub, it means data block groups may not be verified, degrading the |
| 932 |
integrity of data. |
| 933 |
.sp |
| 934 |
Users should be aware of the impact when using swapfiles alongside other data. |
| 935 |
This is especially discouraged for root filesystem. |
| 936 |
.sp |
| 937 |
When there are no active swapfiles and a whole\-filesystem exclusive operation |
| 938 |
is running (e.g. balance, device delete, shrink), the swapfiles cannot be |
| 939 |
temporarily activated. The operation must finish first. |
| 940 |
.sp |
| 941 |
To create and activate a swapfile run the following commands: |
| 942 |
.INDENT 0.0 |
| 943 |
.INDENT 3.5 |
| 944 |
.sp |
| 945 |
.EX |
| 946 |
# truncate \-s 0 swapfile |
| 947 |
# chattr +C swapfile |
| 948 |
# fallocate \-l 2G swapfile |
| 949 |
# chmod 0600 swapfile |
| 950 |
# mkswap swapfile |
| 951 |
# swapon swapfile |
| 952 |
.EE |
| 953 |
.UNINDENT |
| 954 |
.UNINDENT |
| 955 |
.sp |
| 956 |
Since version 6.1 it\(aqs possible to create the swapfile in a single command |
| 957 |
(except the activation): |
| 958 |
.INDENT 0.0 |
| 959 |
.INDENT 3.5 |
| 960 |
.sp |
| 961 |
.EX |
| 962 |
# btrfs filesystem mkswapfile \-\-size 2G swapfile |
| 963 |
# swapon swapfile |
| 964 |
.EE |
| 965 |
.UNINDENT |
| 966 |
.UNINDENT |
| 967 |
.sp |
| 968 |
Please note that the UUID returned by the \fImkswap\fP utility identifies the swap |
| 969 |
\(dqfilesystem\(dq and because it\(aqs stored in a file, it\(aqs not generally visible and |
| 970 |
usable as an identifier unlike if it was on a block device. |
| 971 |
.sp |
| 972 |
Once activated the file will appear in \fB/proc/swaps\fP: |
| 973 |
.INDENT 0.0 |
| 974 |
.INDENT 3.5 |
| 975 |
.sp |
| 976 |
.EX |
| 977 |
# cat /proc/swaps |
| 978 |
Filename Type Size Used Priority |
| 979 |
/path/swapfile file 2097152 0 \-2 |
| 980 |
.EE |
| 981 |
.UNINDENT |
| 982 |
.UNINDENT |
| 983 |
.sp |
| 984 |
The swapfile can be created as one\-time operation or, once properly created, |
| 985 |
activated on each boot by the \fBswapon \-a\fP command (usually started by the |
| 986 |
service manager). Add the following entry to \fI/etc/fstab\fP, assuming the |
| 987 |
filesystem that provides the \fI/path\fP has been already mounted at this point. |
| 988 |
Additional mount options relevant for the swapfile can be set too (like |
| 989 |
priority, not the BTRFS mount options). |
| 990 |
.INDENT 0.0 |
| 991 |
.INDENT 3.5 |
| 992 |
.sp |
| 993 |
.EX |
| 994 |
/path/swapfile none swap defaults 0 0 |
| 995 |
.EE |
| 996 |
.UNINDENT |
| 997 |
.UNINDENT |
| 998 |
.sp |
| 999 |
From now on the subvolume with the active swapfile cannot be snapshotted until |
| 1000 |
the swapfile is deactivated again by \fBswapoff\fP\&. Then the swapfile is a |
| 1001 |
regular file and the subvolume can be snapshotted again, though this would prevent |
| 1002 |
another activation any swapfile that has been snapshotted. New swapfiles (not |
| 1003 |
snapshotted) can be created and activated. |
| 1004 |
.sp |
| 1005 |
Otherwise, an inactive swapfile does not affect the containing subvolume. Activation |
| 1006 |
creates a temporary in\-memory status and prevents some file operations, but is |
| 1007 |
not stored permanently. |
| 1008 |
.SH HIBERNATION |
| 1009 |
.sp |
| 1010 |
A swapfile can be used for hibernation but it\(aqs not straightforward. Before |
| 1011 |
hibernation a resume offset must be written to file \fI/sys/power/resume_offset\fP |
| 1012 |
or the kernel command line parameter \fIresume_offset\fP must be set. |
| 1013 |
.sp |
| 1014 |
The value is the physical offset on the device. Note that \fBthis is not the same |
| 1015 |
value that\fP \fBfilefrag\fP \fBprints as physical offset!\fP |
| 1016 |
.sp |
| 1017 |
Btrfs filesystem uses mapping between logical and physical addresses but here |
| 1018 |
the physical can still map to one or more device\-specific physical block |
| 1019 |
addresses. It\(aqs the device\-specific physical offset that is suitable as resume |
| 1020 |
offset. |
| 1021 |
.sp |
| 1022 |
Since version 6.1 there\(aqs a command btrfs inspect\-internal map\-swapfile \%<#\:man-inspect-map-swapfile> |
| 1023 |
that will print the device physical offset and the adjusted value for |
| 1024 |
\fB/sys/power/resume_offset\fP\&. Note that the value is divided by page size, i.e. |
| 1025 |
it\(aqs not the offset itself. |
| 1026 |
.INDENT 0.0 |
| 1027 |
.INDENT 3.5 |
| 1028 |
.sp |
| 1029 |
.EX |
| 1030 |
# btrfs filesystem mkswapfile swapfile |
| 1031 |
# btrfs inspect\-internal map\-swapfile swapfile |
| 1032 |
Physical start: 811511726080 |
| 1033 |
Resume offset: 198122980 |
| 1034 |
.EE |
| 1035 |
.UNINDENT |
| 1036 |
.UNINDENT |
| 1037 |
.sp |
| 1038 |
For scripting and convenience the option \fI\-r\fP will print just the offset: |
| 1039 |
.INDENT 0.0 |
| 1040 |
.INDENT 3.5 |
| 1041 |
.sp |
| 1042 |
.EX |
| 1043 |
# btrfs inspect\-internal map\-swapfile \-r swapfile |
| 1044 |
198122980 |
| 1045 |
.EE |
| 1046 |
.UNINDENT |
| 1047 |
.UNINDENT |
| 1048 |
.sp |
| 1049 |
The command \fBmap\-swapfile\fP also verifies all the requirements, i.e. no holes, |
| 1050 |
single device, etc. |
| 1051 |
.SH TROUBLESHOOTING |
| 1052 |
.sp |
| 1053 |
If the swapfile activation fails please verify that you followed all the steps |
| 1054 |
above or check the system log (e.g. \fBdmesg\fP or \fBjournalctl\fP) for more |
| 1055 |
information. |
| 1056 |
.sp |
| 1057 |
Notably, the \fBswapon\fP utility exits with a message that does not say what |
| 1058 |
failed: |
| 1059 |
.INDENT 0.0 |
| 1060 |
.INDENT 3.5 |
| 1061 |
.sp |
| 1062 |
.EX |
| 1063 |
# swapon /path/swapfile |
| 1064 |
swapon: /path/swapfile: swapon failed: Invalid argument |
| 1065 |
.EE |
| 1066 |
.UNINDENT |
| 1067 |
.UNINDENT |
| 1068 |
.sp |
| 1069 |
The specific reason is likely to be printed to the system log by the btrfs |
| 1070 |
module: |
| 1071 |
.INDENT 0.0 |
| 1072 |
.INDENT 3.5 |
| 1073 |
.sp |
| 1074 |
.EX |
| 1075 |
# journalctl \-t kernel | grep swapfile |
| 1076 |
kernel: BTRFS warning (device sda): swapfile must have single data profile |
| 1077 |
.EE |
| 1078 |
.UNINDENT |
| 1079 |
.UNINDENT |
| 1080 |
.SH CHECKSUM ALGORITHMS |
| 1081 |
.sp |
| 1082 |
Data and metadata are checksummed by default. The checksum is calculated before |
| 1083 |
writing and verified after reading the blocks from devices. The whole metadata |
| 1084 |
block has an inline checksum stored in the b\-tree node header. Each data block |
| 1085 |
has a detached checksum stored in the checksum tree. |
| 1086 |
.sp |
| 1087 |
\fBNote:\fP |
| 1088 |
.INDENT 0.0 |
| 1089 |
.INDENT 3.5 |
| 1090 |
Since a data checksum is calculated just before submitting to the block |
| 1091 |
device, btrfs has a strong requirement that the corresponding data block must |
| 1092 |
not be modified until the writeback is finished. |
| 1093 |
.sp |
| 1094 |
This requirement is met for a buffered write as btrfs has the full control on |
| 1095 |
its page cache, but a direct write (\fBO_DIRECT\fP) bypasses page cache, and |
| 1096 |
btrfs can not control the direct IO buffer (as it can be in user space memory). |
| 1097 |
Thus it\(aqs possible that a user space program modifies its direct write buffer |
| 1098 |
before the buffer is fully written back, and this can lead to a data |
| 1099 |
checksum mismatch. |
| 1100 |
.sp |
| 1101 |
To avoid this, kernel starting with version 6.14 will force a direct |
| 1102 |
write to fall back to buffered, if the inode requires a data checksum. |
| 1103 |
This will bring a small performance penalty. If you require true zero\-copy |
| 1104 |
direct writes, then set the \fBNODATASUM\fP flag for the inode and make |
| 1105 |
sure the direct IO buffer is fully aligned to block size. |
| 1106 |
.UNINDENT |
| 1107 |
.UNINDENT |
| 1108 |
.sp |
| 1109 |
There are several checksum algorithms supported. The default and backward |
| 1110 |
compatible algorithm is \fIcrc32c\fP\&. Since kernel 5.5 there are three more with different |
| 1111 |
characteristics and trade\-offs regarding speed and strength. The following list |
| 1112 |
may help you to decide which one to select. |
| 1113 |
.INDENT 0.0 |
| 1114 |
.TP |
| 1115 |
.B CRC32C (32 bits digest) |
| 1116 |
Default, best backward compatibility. Very fast, modern CPUs have |
| 1117 |
instruction\-level support, not collision\-resistant but still good error |
| 1118 |
detection capabilities. |
| 1119 |
.TP |
| 1120 |
.B XXHASH (64 bits digest) |
| 1121 |
Can be used as CRC32C successor. Very fast, optimized for modern CPUs utilizing |
| 1122 |
instruction pipelining, good collision resistance and error detection. |
| 1123 |
.TP |
| 1124 |
.B SHA256 (256 bits digest) |
| 1125 |
Cryptographic\-strength hash. Relatively slow but with possible CPU |
| 1126 |
instruction acceleration or specialized hardware cards. FIPS certified and |
| 1127 |
in wide use. |
| 1128 |
.TP |
| 1129 |
.B BLAKE2b (256 bits digest) |
| 1130 |
Cryptographic\-strength hash. Relatively fast, with possible CPU acceleration |
| 1131 |
using SIMD extensions. Not standardized but based on BLAKE which was a SHA3 |
| 1132 |
finalist, in wide use. The algorithm used is BLAKE2b\-256 that\(aqs optimized for |
| 1133 |
64\-bit platforms. |
| 1134 |
.UNINDENT |
| 1135 |
.sp |
| 1136 |
The \fIdigest size\fP affects overall size of data block checksums stored in the |
| 1137 |
filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so |
| 1138 |
there\(aqs no increase. Each data block has a separate checksum stored, with |
| 1139 |
additional overhead of the b\-tree leaves. |
| 1140 |
.sp |
| 1141 |
Approximate relative performance of the algorithms, measured against CRC32C |
| 1142 |
using implementations on a 11th gen 3.6GHz intel CPU: |
| 1143 |
.TS |
| 1144 |
box center; |
| 1145 |
l|l|l|l. |
| 1146 |
T{ |
| 1147 |
Digest |
| 1148 |
T} T{ |
| 1149 |
Cycles/4KiB |
| 1150 |
T} T{ |
| 1151 |
Ratio |
| 1152 |
T} T{ |
| 1153 |
Implementation |
| 1154 |
T} |
| 1155 |
_ |
| 1156 |
T{ |
| 1157 |
CRC32C |
| 1158 |
T} T{ |
| 1159 |
470 |
| 1160 |
T} T{ |
| 1161 |
1.00 |
| 1162 |
T} T{ |
| 1163 |
CPU instruction, PCL combination |
| 1164 |
T} |
| 1165 |
_ |
| 1166 |
T{ |
| 1167 |
XXHASH |
| 1168 |
T} T{ |
| 1169 |
870 |
| 1170 |
T} T{ |
| 1171 |
1.9 |
| 1172 |
T} T{ |
| 1173 |
reference impl. |
| 1174 |
T} |
| 1175 |
_ |
| 1176 |
T{ |
| 1177 |
SHA256 |
| 1178 |
T} T{ |
| 1179 |
7600 |
| 1180 |
T} T{ |
| 1181 |
16 |
| 1182 |
T} T{ |
| 1183 |
libgcrypt |
| 1184 |
T} |
| 1185 |
_ |
| 1186 |
T{ |
| 1187 |
SHA256 |
| 1188 |
T} T{ |
| 1189 |
8500 |
| 1190 |
T} T{ |
| 1191 |
18 |
| 1192 |
T} T{ |
| 1193 |
openssl |
| 1194 |
T} |
| 1195 |
_ |
| 1196 |
T{ |
| 1197 |
SHA256 |
| 1198 |
T} T{ |
| 1199 |
8700 |
| 1200 |
T} T{ |
| 1201 |
18 |
| 1202 |
T} T{ |
| 1203 |
botan |
| 1204 |
T} |
| 1205 |
_ |
| 1206 |
T{ |
| 1207 |
SHA256 |
| 1208 |
T} T{ |
| 1209 |
32000 |
| 1210 |
T} T{ |
| 1211 |
68 |
| 1212 |
T} T{ |
| 1213 |
builtin, CPU instruction |
| 1214 |
T} |
| 1215 |
_ |
| 1216 |
T{ |
| 1217 |
SHA256 |
| 1218 |
T} T{ |
| 1219 |
37000 |
| 1220 |
T} T{ |
| 1221 |
78 |
| 1222 |
T} T{ |
| 1223 |
libsodium |
| 1224 |
T} |
| 1225 |
_ |
| 1226 |
T{ |
| 1227 |
SHA256 |
| 1228 |
T} T{ |
| 1229 |
78000 |
| 1230 |
T} T{ |
| 1231 |
166 |
| 1232 |
T} T{ |
| 1233 |
builtin, reference impl. |
| 1234 |
T} |
| 1235 |
_ |
| 1236 |
T{ |
| 1237 |
BLAKE2b |
| 1238 |
T} T{ |
| 1239 |
10000 |
| 1240 |
T} T{ |
| 1241 |
21 |
| 1242 |
T} T{ |
| 1243 |
builtin/AVX2 |
| 1244 |
T} |
| 1245 |
_ |
| 1246 |
T{ |
| 1247 |
BLAKE2b |
| 1248 |
T} T{ |
| 1249 |
10900 |
| 1250 |
T} T{ |
| 1251 |
23 |
| 1252 |
T} T{ |
| 1253 |
libgcrypt |
| 1254 |
T} |
| 1255 |
_ |
| 1256 |
T{ |
| 1257 |
BLAKE2b |
| 1258 |
T} T{ |
| 1259 |
13500 |
| 1260 |
T} T{ |
| 1261 |
29 |
| 1262 |
T} T{ |
| 1263 |
builtin/SSE41 |
| 1264 |
T} |
| 1265 |
_ |
| 1266 |
T{ |
| 1267 |
BLAKE2b |
| 1268 |
T} T{ |
| 1269 |
13700 |
| 1270 |
T} T{ |
| 1271 |
29 |
| 1272 |
T} T{ |
| 1273 |
libsodium |
| 1274 |
T} |
| 1275 |
_ |
| 1276 |
T{ |
| 1277 |
BLAKE2b |
| 1278 |
T} T{ |
| 1279 |
14100 |
| 1280 |
T} T{ |
| 1281 |
30 |
| 1282 |
T} T{ |
| 1283 |
openssl |
| 1284 |
T} |
| 1285 |
_ |
| 1286 |
T{ |
| 1287 |
BLAKE2b |
| 1288 |
T} T{ |
| 1289 |
14500 |
| 1290 |
T} T{ |
| 1291 |
31 |
| 1292 |
T} T{ |
| 1293 |
kcapi |
| 1294 |
T} |
| 1295 |
_ |
| 1296 |
T{ |
| 1297 |
BLAKE2b |
| 1298 |
T} T{ |
| 1299 |
14500 |
| 1300 |
T} T{ |
| 1301 |
34 |
| 1302 |
T} T{ |
| 1303 |
builtin, reference impl. |
| 1304 |
T} |
| 1305 |
.TE |
| 1306 |
.sp |
| 1307 |
Many kernels are configured with SHA256 as built\-in and not as a module. |
| 1308 |
Up to kernel v6.15 the accelerated versions are however provided by the |
| 1309 |
modules and must be loaded |
| 1310 |
explicitly (\fBmodprobe sha256\fP) before mounting the filesystem to make use of |
| 1311 |
them. You can check in \fB/sys/fs/btrfs/FSID/checksum\fP which one is used. If you |
| 1312 |
see \fIsha256\-generic\fP, then you may want to unmount and mount the filesystem |
| 1313 |
again. Changing that on a mounted filesystem is not possible. |
| 1314 |
.sp |
| 1315 |
Since kernel v6.16 the accelerated implementation is always used if available. |
| 1316 |
.sp |
| 1317 |
Check the file \fB/proc/crypto\fP, when the implementation is built\-in, you\(aqd find: |
| 1318 |
.INDENT 0.0 |
| 1319 |
.INDENT 3.5 |
| 1320 |
.sp |
| 1321 |
.EX |
| 1322 |
name : sha256 |
| 1323 |
driver : sha256\-generic |
| 1324 |
module : kernel |
| 1325 |
priority : 100 |
| 1326 |
\&... |
| 1327 |
.EE |
| 1328 |
.UNINDENT |
| 1329 |
.UNINDENT |
| 1330 |
.sp |
| 1331 |
While accelerated implementation is e.g.: |
| 1332 |
.INDENT 0.0 |
| 1333 |
.INDENT 3.5 |
| 1334 |
.sp |
| 1335 |
.EX |
| 1336 |
name : sha256 |
| 1337 |
driver : sha256\-avx2 |
| 1338 |
module : sha256_ssse3 |
| 1339 |
priority : 170 |
| 1340 |
\&... |
| 1341 |
.EE |
| 1342 |
.UNINDENT |
| 1343 |
.UNINDENT |
| 1344 |
.SH COMPRESSION |
| 1345 |
.sp |
| 1346 |
Btrfs supports transparent file compression. There are three algorithms |
| 1347 |
available: ZLIB, LZO and ZSTD (since v4.14), with various levels. |
| 1348 |
The compression happens on the level of file extents and the algorithm is |
| 1349 |
selected by file property, mount option or by a defrag command. |
| 1350 |
You can have a single btrfs mount point that has some files that are |
| 1351 |
uncompressed, some that are compressed with LZO, some with ZLIB, for instance |
| 1352 |
(though you may not want it that way, it is supported). |
| 1353 |
.sp |
| 1354 |
Once the compression is set, all newly written data will be compressed, i.e. |
| 1355 |
existing data are untouched. Data are split into smaller chunks (128KiB) before |
| 1356 |
compression to make random rewrites possible without a high performance hit. Due |
| 1357 |
to the increased number of extents the metadata consumption is higher. The |
| 1358 |
chunks are compressed in parallel. |
| 1359 |
.sp |
| 1360 |
The algorithms can be characterized as follows regarding the speed/ratio |
| 1361 |
trade\-offs: |
| 1362 |
.INDENT 0.0 |
| 1363 |
.TP |
| 1364 |
.B ZLIB |
| 1365 |
.INDENT 7.0 |
| 1366 |
.IP \(bu 2 |
| 1367 |
slower, higher compression ratio |
| 1368 |
.IP \(bu 2 |
| 1369 |
levels: 1 to 9, mapped directly, default level is 3 |
| 1370 |
.IP \(bu 2 |
| 1371 |
good backward compatibility |
| 1372 |
.UNINDENT |
| 1373 |
.TP |
| 1374 |
.B LZO |
| 1375 |
.INDENT 7.0 |
| 1376 |
.IP \(bu 2 |
| 1377 |
faster compression and decompression than ZLIB, worse compression ratio, designed to be fast |
| 1378 |
.IP \(bu 2 |
| 1379 |
no levels |
| 1380 |
.IP \(bu 2 |
| 1381 |
good backward compatibility |
| 1382 |
.UNINDENT |
| 1383 |
.TP |
| 1384 |
.B ZSTD |
| 1385 |
.INDENT 7.0 |
| 1386 |
.IP \(bu 2 |
| 1387 |
compression comparable to ZLIB with higher compression/decompression speeds and different ratio |
| 1388 |
.IP \(bu 2 |
| 1389 |
levels: \-15..15, mapped directly, default is 3 |
| 1390 |
.IP \(bu 2 |
| 1391 |
support since 4.14 |
| 1392 |
.IP \(bu 2 |
| 1393 |
levels 1..15 supported since 5.1 |
| 1394 |
.IP \(bu 2 |
| 1395 |
levels \-15..\-1 supported since 6.15 |
| 1396 |
.UNINDENT |
| 1397 |
.UNINDENT |
| 1398 |
.sp |
| 1399 |
The differences depend on the actual data set and cannot be expressed by a |
| 1400 |
single number or recommendation. Higher levels consume more CPU time and may |
| 1401 |
not bring a significant improvement, lower levels are close to real time. |
| 1402 |
.SH HOW TO ENABLE COMPRESSION |
| 1403 |
.sp |
| 1404 |
Typically the compression can be enabled on the whole filesystem, specified for |
| 1405 |
the mount point. Note that the compression mount options are shared among all |
| 1406 |
mounts of the same filesystem, either bind mounts or subvolume mounts. |
| 1407 |
Please refer to btrfs(5) \%<> section |
| 1408 |
MOUNT OPTIONS\&. |
| 1409 |
.INDENT 0.0 |
| 1410 |
.INDENT 3.5 |
| 1411 |
.sp |
| 1412 |
.EX |
| 1413 |
$ mount \-o compress=zstd /dev/sdx /mnt |
| 1414 |
.EE |
| 1415 |
.UNINDENT |
| 1416 |
.UNINDENT |
| 1417 |
.sp |
| 1418 |
This will enable the \fBzstd\fP algorithm on the default level (which is 3). |
| 1419 |
The level can be specified manually too like \fBzstd:3\fP\&. Higher levels compress |
| 1420 |
better at the cost of time. This in turn may cause increased write latency, low |
| 1421 |
levels are suitable for real\-time compression and on reasonably fast CPU don\(aqt |
| 1422 |
cause noticeable performance drops. |
| 1423 |
.INDENT 0.0 |
| 1424 |
.INDENT 3.5 |
| 1425 |
.sp |
| 1426 |
.EX |
| 1427 |
$ btrfs filesystem defrag \-czstd file |
| 1428 |
.EE |
| 1429 |
.UNINDENT |
| 1430 |
.UNINDENT |
| 1431 |
.sp |
| 1432 |
The command above will start defragmentation of the whole \fIfile\fP and apply |
| 1433 |
the compression, regardless of the mount option. The compression level can be |
| 1434 |
also specified with the \fB\-\-level\fP or \fB\-L\fP argument as of version \fB6.14\fP\&. |
| 1435 |
The compression algorithm is not persistent and applies only |
| 1436 |
to the defragmentation command, for any other writes other compression settings |
| 1437 |
apply. |
| 1438 |
.sp |
| 1439 |
Persistent settings on a per\-file basis can be set in two ways: |
| 1440 |
.INDENT 0.0 |
| 1441 |
.INDENT 3.5 |
| 1442 |
.sp |
| 1443 |
.EX |
| 1444 |
$ chattr +c file |
| 1445 |
$ btrfs property set file compression zstd |
| 1446 |
.EE |
| 1447 |
.UNINDENT |
| 1448 |
.UNINDENT |
| 1449 |
.sp |
| 1450 |
The first command is using legacy interface of file attributes inherited from |
| 1451 |
ext2 filesystem and is not flexible, so by default the \fIzlib\fP compression is |
| 1452 |
set. The other command sets a property on the file with the given algorithm. |
| 1453 |
(Note: setting level that way is not yet implemented.) |
| 1454 |
.SH COMPRESSION LEVELS |
| 1455 |
.sp |
| 1456 |
The level support of ZLIB has been added in v4.14, LZO does not support levels |
| 1457 |
(the kernel implementation provides only one), ZSTD level support has been added |
| 1458 |
in v5.1 and the negative levels in v6.15. |
| 1459 |
.sp |
| 1460 |
There are 9 levels of ZLIB supported (1 to 9), mapping 1:1 from the mount option |
| 1461 |
to the algorithm defined level. The default is level 3, which provides the |
| 1462 |
reasonably good compression ratio and is still reasonably fast. The difference |
| 1463 |
in compression gain of levels 7, 8 and 9 is comparable but the higher levels |
| 1464 |
take longer. |
| 1465 |
.sp |
| 1466 |
The ZSTD support includes levels \-15..15, a subset of full range of what ZSTD |
| 1467 |
provides. Levels \-15..\-1 are real\-time with worse compression ratio, levels |
| 1468 |
1..3 are near real\-time with good compression, 4..8 are slower with improved |
| 1469 |
compression and 9..15 try even harder though the resulting size may not be |
| 1470 |
significantly improved. Higher levels also require more memory and as they need |
| 1471 |
more CPU the system performance is affected. |
| 1472 |
.sp |
| 1473 |
Level 0 always maps to the default. The compression level does not affect |
| 1474 |
compatibility. |
| 1475 |
.SH EXCEPTIONS |
| 1476 |
.sp |
| 1477 |
Any file that has been touched by the \fIfallocate\fP system call will always be |
| 1478 |
excepted from compression even if \fIforce\-compress\fP mount option is used. |
| 1479 |
.sp |
| 1480 |
The reason for this is that a successful \fIfallocate\fP call must guarantee that |
| 1481 |
future writes to the allocated range will not fail because of lack of space. |
| 1482 |
This is difficult to guarantee in a COW filesystem. To reduce the chances of |
| 1483 |
it happening, btrfs preallocates space and disables compression for the file. |
| 1484 |
.sp |
| 1485 |
As a workaround, one can trigger a compressed rewrite for such a file using the |
| 1486 |
\fIbtrfs defrag\fP command. Be aware that if the file is touched again by the |
| 1487 |
\fIfallocate\fP system call, it will be excepted again from compression for all the |
| 1488 |
new data written to it. |
| 1489 |
.SH INCOMPRESSIBLE DATA |
| 1490 |
.sp |
| 1491 |
Files with already compressed data or with data that won\(aqt compress well with |
| 1492 |
the CPU and memory constraints of the kernel implementations are using a simple |
| 1493 |
decision logic. If the first portion of data being compressed is not smaller |
| 1494 |
than the original, the compression of the whole file is disabled. Unless the |
| 1495 |
filesystem is mounted with \fIcompress\-force\fP in which case btrfs will try |
| 1496 |
compressing every block, falling back to storing the uncompressed version for |
| 1497 |
each block that ends up larger after compression. This is not optimal and |
| 1498 |
subject to optimizations and further development. |
| 1499 |
.sp |
| 1500 |
If a file is identified as incompressible, a flag is set (\fINOCOMPRESS\fP) and it\(aqs |
| 1501 |
sticky. On that file compression won\(aqt be performed unless forced. The flag |
| 1502 |
can be also set by \fBchattr +m\fP (since e2fsprogs 1.46.2) or by properties with |
| 1503 |
value \fIno\fP or \fInone\fP\&. Empty value will reset it to the default that\(aqs currently |
| 1504 |
applicable on the mounted filesystem. |
| 1505 |
.sp |
| 1506 |
There are two ways to detect incompressible data: |
| 1507 |
.INDENT 0.0 |
| 1508 |
.IP \(bu 2 |
| 1509 |
actual compression attempt \- data are compressed, if the result is not smaller, |
| 1510 |
it\(aqs discarded, so this depends on the algorithm and level |
| 1511 |
.IP \(bu 2 |
| 1512 |
pre\-compression heuristics \- a quick statistical evaluation on the data is |
| 1513 |
performed and based on the result either compression is performed or skipped, |
| 1514 |
the NOCOMPRESS bit is not set just by the heuristic, only if the compression |
| 1515 |
algorithm does not make an improvement |
| 1516 |
.UNINDENT |
| 1517 |
.INDENT 0.0 |
| 1518 |
.INDENT 3.5 |
| 1519 |
.sp |
| 1520 |
.EX |
| 1521 |
$ lsattr file |
| 1522 |
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-m file |
| 1523 |
.EE |
| 1524 |
.UNINDENT |
| 1525 |
.UNINDENT |
| 1526 |
.sp |
| 1527 |
Using the forcing compression is not recommended, the heuristics are |
| 1528 |
supposed to decide that and compression algorithms internally detect |
| 1529 |
incompressible data too. |
| 1530 |
.SH PRE-COMPRESSION HEURISTICS |
| 1531 |
.sp |
| 1532 |
The heuristics aim to do a few quick statistical tests on the compressed data |
| 1533 |
in order to avoid probably costly compression that would turn out to be |
| 1534 |
inefficient. Compression algorithms could have internal detection of |
| 1535 |
incompressible data too but this leads to more overhead as the compression is |
| 1536 |
done in another thread and has to write the data anyway. The heuristic is |
| 1537 |
read\-only and can utilize cached memory. |
| 1538 |
.sp |
| 1539 |
The tests performed based on the following: data sampling, long repeated |
| 1540 |
pattern detection, byte frequency, Shannon entropy. |
| 1541 |
.SH COMPATIBILITY |
| 1542 |
.sp |
| 1543 |
Compression requires both data checksums and COW, so either \fInodatasum\fP or |
| 1544 |
\fInodatasum\fP mount option/inode flag will result in no compression. |
| 1545 |
.sp |
| 1546 |
Direct IO reads of compressed data will always fallback to buffered reads. |
| 1547 |
.sp |
| 1548 |
Direct IO write behavior depends on the inode flag. |
| 1549 |
For inodes with data checksum, direct IO writes always fallback to buffered |
| 1550 |
writes, thus can generate compressed data if the mount option/inode flags |
| 1551 |
allows that. |
| 1552 |
.sp |
| 1553 |
For inodes without data checksums, direct IO writes will not populate page cache, |
| 1554 |
and since the inode has no data checksums, no compressed data will be generated |
| 1555 |
anyway. |
| 1556 |
.sp |
| 1557 |
The compression algorithms have been added over time so the version |
| 1558 |
compatibility should be also considered, together with other tools that may |
| 1559 |
access the compressed data like bootloaders. |
| 1560 |
.SH SYSFS INTERFACE |
| 1561 |
.sp |
| 1562 |
Btrfs has a sysfs interface to provide extra knobs. |
| 1563 |
.sp |
| 1564 |
The top level path is \fB/sys/fs/btrfs/\fP, and the main directory layout is the following: |
| 1565 |
.TS |
| 1566 |
box center; |
| 1567 |
l|l|l. |
| 1568 |
T{ |
| 1569 |
Relative Path |
| 1570 |
T} T{ |
| 1571 |
Description |
| 1572 |
T} T{ |
| 1573 |
Version |
| 1574 |
T} |
| 1575 |
_ |
| 1576 |
T{ |
| 1577 |
features/ |
| 1578 |
T} T{ |
| 1579 |
All supported features |
| 1580 |
T} T{ |
| 1581 |
3.14 |
| 1582 |
T} |
| 1583 |
_ |
| 1584 |
T{ |
| 1585 |
<UUID>/ |
| 1586 |
T} T{ |
| 1587 |
Mounted fs UUID |
| 1588 |
T} T{ |
| 1589 |
3.14 |
| 1590 |
T} |
| 1591 |
_ |
| 1592 |
T{ |
| 1593 |
<UUID>/allocation/ |
| 1594 |
T} T{ |
| 1595 |
Space allocation info |
| 1596 |
T} T{ |
| 1597 |
3.14 |
| 1598 |
T} |
| 1599 |
_ |
| 1600 |
T{ |
| 1601 |
<UUID>/bdi/ |
| 1602 |
T} T{ |
| 1603 |
Backing device info (writeback) |
| 1604 |
T} T{ |
| 1605 |
5.9 |
| 1606 |
T} |
| 1607 |
_ |
| 1608 |
T{ |
| 1609 |
<UUID>/devices/<DEVID>/ |
| 1610 |
T} T{ |
| 1611 |
Symlink to each block device sysfs |
| 1612 |
T} T{ |
| 1613 |
5.6 |
| 1614 |
T} |
| 1615 |
_ |
| 1616 |
T{ |
| 1617 |
<UUID>/devinfo/<DEVID>/ |
| 1618 |
T} T{ |
| 1619 |
Btrfs specific info for each device |
| 1620 |
T} T{ |
| 1621 |
5.6 |
| 1622 |
T} |
| 1623 |
_ |
| 1624 |
T{ |
| 1625 |
<UUID>/discard/ |
| 1626 |
T} T{ |
| 1627 |
Discard stats and tunables |
| 1628 |
T} T{ |
| 1629 |
6.1 |
| 1630 |
T} |
| 1631 |
_ |
| 1632 |
T{ |
| 1633 |
<UUID>/features/ |
| 1634 |
T} T{ |
| 1635 |
Features of the filesystem |
| 1636 |
T} T{ |
| 1637 |
3.14 |
| 1638 |
T} |
| 1639 |
_ |
| 1640 |
T{ |
| 1641 |
<UUID>/qgroups/ |
| 1642 |
T} T{ |
| 1643 |
Global qgroup info |
| 1644 |
T} T{ |
| 1645 |
5.9 |
| 1646 |
T} |
| 1647 |
_ |
| 1648 |
T{ |
| 1649 |
<UUID>/qgroups/<LEVEL>_<ID>/ |
| 1650 |
T} T{ |
| 1651 |
Info for each qgroup |
| 1652 |
T} T{ |
| 1653 |
5.9 |
| 1654 |
T} |
| 1655 |
.TE |
| 1656 |
.sp |
| 1657 |
For \fB/sys/fs/btrfs/features/\fP directory, each file means a supported feature |
| 1658 |
of the current kernel. Most files have value 0. Otherwise it depends on the file, |
| 1659 |
value \fI1\fP typically means the feature can be turned on a mounted filesystem. |
| 1660 |
.sp |
| 1661 |
For \fB/sys/fs/btrfs/<UUID>/features/\fP directory, each file means an enabled |
| 1662 |
feature on the mounted filesystem. |
| 1663 |
.sp |
| 1664 |
The features share the same name in section |
| 1665 |
FILESYSTEM FEATURES\&. |
| 1666 |
.SS UUID |
| 1667 |
.sp |
| 1668 |
Files in \fB/sys/fs/btrfs/<UUID>/\fP directory are: |
| 1669 |
.INDENT 0.0 |
| 1670 |
.TP |
| 1671 |
.B bg_reclaim_threshold |
| 1672 |
(RW, since: 5.19) |
| 1673 |
.sp |
| 1674 |
Used space percentage of total device space to start auto block group claim. |
| 1675 |
Mostly for zoned devices. |
| 1676 |
.TP |
| 1677 |
.B checksum |
| 1678 |
(RO, since: 5.5) |
| 1679 |
.sp |
| 1680 |
The checksum used for the mounted filesystem. |
| 1681 |
This includes both the checksum type (see section |
| 1682 |
CHECKSUM ALGORITHMS) |
| 1683 |
and the implemented driver (mostly shows if it\(aqs hardware accelerated). |
| 1684 |
.TP |
| 1685 |
.B clone_alignment |
| 1686 |
(RO, since: 3.16) |
| 1687 |
.sp |
| 1688 |
The bytes alignment for \fIclone\fP and \fIdedupe\fP ioctls. |
| 1689 |
.TP |
| 1690 |
.B commit_stats |
| 1691 |
(RW, since: 6.0) |
| 1692 |
.sp |
| 1693 |
The performance statistics for btrfs transaction commit since the first |
| 1694 |
mount. Mostly for debugging purposes. |
| 1695 |
.sp |
| 1696 |
Writing into this file will reset the maximum commit duration |
| 1697 |
(\fImax_commit_ms\fP) to 0. The file looks like: |
| 1698 |
.INDENT 7.0 |
| 1699 |
.INDENT 3.5 |
| 1700 |
.sp |
| 1701 |
.EX |
| 1702 |
commits 70649 |
| 1703 |
last_commit_ms 2 |
| 1704 |
max_commit_ms 131 |
| 1705 |
total_commit_ms 170840 |
| 1706 |
.EE |
| 1707 |
.UNINDENT |
| 1708 |
.UNINDENT |
| 1709 |
.INDENT 7.0 |
| 1710 |
.IP \(bu 2 |
| 1711 |
\fIcommits\fP \- number of transaction commits since the first mount |
| 1712 |
.IP \(bu 2 |
| 1713 |
\fIlast_commit_ms\fP \- duration in milliseconds of the last commit |
| 1714 |
.IP \(bu 2 |
| 1715 |
\fImax_commit_ms\fP \- maximum time a transaction commit took since first mount or last reset |
| 1716 |
.IP \(bu 2 |
| 1717 |
\fItotal_commit_ms\fP \- sum of all transaction commit times |
| 1718 |
.UNINDENT |
| 1719 |
.TP |
| 1720 |
.B exclusive_operation |
| 1721 |
(RO, since: 5.10) |
| 1722 |
.sp |
| 1723 |
Shows the running exclusive operation. |
| 1724 |
Check section |
| 1725 |
FILESYSTEM EXCLUSIVE OPERATIONS |
| 1726 |
for details. |
| 1727 |
.TP |
| 1728 |
.B generation |
| 1729 |
(RO, since: 5.11) |
| 1730 |
.sp |
| 1731 |
Show the generation of the mounted filesystem. |
| 1732 |
.TP |
| 1733 |
.B label |
| 1734 |
(RW, since: 3.14) |
| 1735 |
.sp |
| 1736 |
Show the current label of the mounted filesystem. |
| 1737 |
.TP |
| 1738 |
.B metadata_uuid |
| 1739 |
(RO, since: 5.0) |
| 1740 |
.sp |
| 1741 |
Shows the metadata UUID of the mounted filesystem. |
| 1742 |
Check \fImetadata_uuid\fP feature for more details. |
| 1743 |
.TP |
| 1744 |
.B nodesize |
| 1745 |
(RO, since: 3.14) |
| 1746 |
.sp |
| 1747 |
Show the nodesize of the mounted filesystem. |
| 1748 |
.TP |
| 1749 |
.B quota_override |
| 1750 |
(RW, since: 4.13) |
| 1751 |
.sp |
| 1752 |
Shows the current quota override status. |
| 1753 |
0 means no quota override. |
| 1754 |
1 means quota override, quota can ignore the existing limit settings. |
| 1755 |
.TP |
| 1756 |
.B read_policy |
| 1757 |
(RW, since: 5.11) |
| 1758 |
.sp |
| 1759 |
Shows the current balance policy for reads. |
| 1760 |
Currently only \fBpid\fP (balance using the process id (pid) value) is |
| 1761 |
supported. More balancing policies are available in experimental |
| 1762 |
build, namely round\-robin. |
| 1763 |
.TP |
| 1764 |
.B sectorsize |
| 1765 |
(RO, since: 3.14) |
| 1766 |
.sp |
| 1767 |
Shows the sectorsize of the mounted filesystem. |
| 1768 |
.TP |
| 1769 |
.B temp_fsid |
| 1770 |
(RO, since 6.7) |
| 1771 |
.sp |
| 1772 |
Indicate that this filesystem got assigned a temporary FSID at mount time, |
| 1773 |
making possible to mount devices with the same FSID. |
| 1774 |
.UNINDENT |
| 1775 |
.SS UUID/allocations |
| 1776 |
.sp |
| 1777 |
Files and directories in \fB/sys/fs/btrfs/<UUID>/allocations\fP directory are: |
| 1778 |
.INDENT 0.0 |
| 1779 |
.TP |
| 1780 |
.B global_rsv_reserved |
| 1781 |
(RO, since: 3.14) |
| 1782 |
.sp |
| 1783 |
The used bytes of the global reservation. |
| 1784 |
.TP |
| 1785 |
.B global_rsv_size |
| 1786 |
(RO, since: 3.14) |
| 1787 |
.sp |
| 1788 |
The total size of the global reservation. |
| 1789 |
.TP |
| 1790 |
.B \fIdata/\fP, \fImetadata/\fP and \fIsystem/\fP directories |
| 1791 |
(RO, since: 5.14) |
| 1792 |
.sp |
| 1793 |
Space info accounting for the 3 block group types. |
| 1794 |
.UNINDENT |
| 1795 |
.SS UUID/allocations/{data,metadata,system} |
| 1796 |
.sp |
| 1797 |
Files in \fB/sys/fs/btrfs/<UUID>/allocations/\fP\fIdata,metadata,system\fP directory are: |
| 1798 |
.INDENT 0.0 |
| 1799 |
.TP |
| 1800 |
.B bg_reclaim_threshold |
| 1801 |
(RW, since: 5.19) |
| 1802 |
.sp |
| 1803 |
Reclaimable space percentage of block group\(aqs size (excluding |
| 1804 |
permanently unusable space) to reclaim the block group. |
| 1805 |
Can be used on regular or zoned devices. |
| 1806 |
.TP |
| 1807 |
.B bytes_* |
| 1808 |
(RO) |
| 1809 |
.sp |
| 1810 |
Values of the corresponding data structures for the given block group |
| 1811 |
type and profile that are used internally and may change rapidly depending |
| 1812 |
on the load. |
| 1813 |
.sp |
| 1814 |
Complete list: bytes_may_use, bytes_pinned, bytes_readonly, |
| 1815 |
bytes_reserved, bytes_used, bytes_zone_unusable |
| 1816 |
.TP |
| 1817 |
.B chunk_size |
| 1818 |
(RW, since: 6.0) |
| 1819 |
.sp |
| 1820 |
Shows the chunk size. Can be changed for data and metadata (independently) |
| 1821 |
and cannot be set for system block group type. |
| 1822 |
Cannot be set for zoned devices as it depends on the fixed device zone size. |
| 1823 |
Upper bound is 10% of the filesystem size, the value must be multiple of 256MiB |
| 1824 |
and greater than 0. |
| 1825 |
.TP |
| 1826 |
.B size_classes |
| 1827 |
(RO, since: 6.3) |
| 1828 |
.sp |
| 1829 |
Numbers of block groups of a given classes based on heuristics that |
| 1830 |
measure extent length, age and fragmentation. |
| 1831 |
.INDENT 7.0 |
| 1832 |
.INDENT 3.5 |
| 1833 |
.sp |
| 1834 |
.EX |
| 1835 |
none 136 |
| 1836 |
small 374 |
| 1837 |
medium 282 |
| 1838 |
large 93 |
| 1839 |
.EE |
| 1840 |
.UNINDENT |
| 1841 |
.UNINDENT |
| 1842 |
.UNINDENT |
| 1843 |
.SS UUID/bdi |
| 1844 |
.sp |
| 1845 |
Symlink to the sysfs directory of the backing device info (BDI), which is |
| 1846 |
related to writeback process and infrastructure. |
| 1847 |
.SS UUID/devices |
| 1848 |
.sp |
| 1849 |
Files in \fB/sys/fs/btrfs/<UUID>/devices\fP directory are symlinks named |
| 1850 |
after device nodes (e.g. sda, dm\-0) and pointing to their sysfs directory. |
| 1851 |
.SS UUID/devinfo |
| 1852 |
.sp |
| 1853 |
The directory contains subdirectories named after device ids (numeric values). Each |
| 1854 |
subdirectory has information about the device of the given \fIdevid\fP\&. |
| 1855 |
.SS UUID/devinfo/DEVID |
| 1856 |
.sp |
| 1857 |
Files in \fB/sys/fs/btrfs/<UUID>/devinfo/<DEVID>\fP directory are: |
| 1858 |
.INDENT 0.0 |
| 1859 |
.TP |
| 1860 |
.B error_stats: |
| 1861 |
(RO, since: 5.14) |
| 1862 |
.sp |
| 1863 |
Shows device stats of this device, same as \fBbtrfs device stats\fP (btrfs\-device(8) \%<>). |
| 1864 |
.INDENT 7.0 |
| 1865 |
.INDENT 3.5 |
| 1866 |
.sp |
| 1867 |
.EX |
| 1868 |
write_errs 0 |
| 1869 |
read_errs 0 |
| 1870 |
flush_errs 0 |
| 1871 |
corruption_errs 0 |
| 1872 |
generation_errs 0 |
| 1873 |
.EE |
| 1874 |
.UNINDENT |
| 1875 |
.UNINDENT |
| 1876 |
.TP |
| 1877 |
.B fsid: |
| 1878 |
(RO, since: 5.17) |
| 1879 |
.sp |
| 1880 |
Shows the fsid which the device belongs to. |
| 1881 |
It can be different than the \fBUUID\fP if it\(aqs a seed device. |
| 1882 |
.TP |
| 1883 |
.B in_fs_metadata |
| 1884 |
(RO, since: 5.6) |
| 1885 |
.sp |
| 1886 |
Shows whether we have found the device. |
| 1887 |
Should always be 1, as if this turns to 0, the \fBDEVID\fP directory |
| 1888 |
would get removed automatically. |
| 1889 |
.TP |
| 1890 |
.B missing |
| 1891 |
(RO, since: 5.6) |
| 1892 |
.sp |
| 1893 |
Shows whether the device is considered missing by the kernel module. |
| 1894 |
.TP |
| 1895 |
.B replace_target |
| 1896 |
(RO, since: 5.6) |
| 1897 |
.sp |
| 1898 |
Shows whether the device is the replace target. |
| 1899 |
If no device replace is running, this value is 0. |
| 1900 |
.TP |
| 1901 |
.B scrub_speed_max |
| 1902 |
(RW, since: 5.14) |
| 1903 |
.sp |
| 1904 |
Shows the scrub speed limit for this device. The unit is Bytes/s. |
| 1905 |
0 means no limit. The value can be set but is not persistent. |
| 1906 |
.TP |
| 1907 |
.B writeable |
| 1908 |
(RO, since: 5.6) |
| 1909 |
.sp |
| 1910 |
Show if the device is writeable. |
| 1911 |
.UNINDENT |
| 1912 |
.SS UUID/qgroups |
| 1913 |
.sp |
| 1914 |
Files in \fB/sys/fs/btrfs/<UUID>/qgroups/\fP directory are: |
| 1915 |
.INDENT 0.0 |
| 1916 |
.TP |
| 1917 |
.B enabled |
| 1918 |
(RO, since: 6.1) |
| 1919 |
.sp |
| 1920 |
Shows if qgroup is enabled. |
| 1921 |
Also, if qgroup is disabled, the \fBqgroups\fP directory will |
| 1922 |
be removed automatically. |
| 1923 |
.TP |
| 1924 |
.B inconsistent |
| 1925 |
(RO, since: 6.1) |
| 1926 |
.sp |
| 1927 |
Shows if the qgroup numbers are inconsistent. |
| 1928 |
If 1, it\(aqs recommended to do a qgroup rescan. |
| 1929 |
.TP |
| 1930 |
.B drop_subtree_threshold |
| 1931 |
(RW, since: 6.1) |
| 1932 |
.sp |
| 1933 |
Shows the subtree drop threshold to automatically mark qgroup inconsistent. |
| 1934 |
.sp |
| 1935 |
When dropping large subvolumes with qgroup enabled, there would be a huge |
| 1936 |
load for qgroup accounting. |
| 1937 |
If we have a subtree whose level is larger than or equal to this value, |
| 1938 |
we will not trigger qgroup account at all, but mark qgroup inconsistent to |
| 1939 |
avoid the huge workload. |
| 1940 |
.sp |
| 1941 |
Default value is 3, which means that trees of low height will be accounted |
| 1942 |
properly as this is sufficiently fast. The value was 8 until 6.13 where |
| 1943 |
no subtree drop can trigger qgroup rescan making it less useful. |
| 1944 |
.sp |
| 1945 |
Lower value can reduce qgroup workload, at the cost of extra qgroup rescan |
| 1946 |
to re\-calculate the numbers. |
| 1947 |
.UNINDENT |
| 1948 |
.SS UUID/qgroups/LEVEL_ID |
| 1949 |
.sp |
| 1950 |
Files in each \fB/sys/fs/btrfs/<UUID>/qgroups/<LEVEL>_<ID>/\fP directory are: |
| 1951 |
.INDENT 0.0 |
| 1952 |
.TP |
| 1953 |
.B exclusive |
| 1954 |
(RO, since: 5.9) |
| 1955 |
.sp |
| 1956 |
Shows the exclusively owned bytes of the qgroup. |
| 1957 |
.TP |
| 1958 |
.B limit_flags |
| 1959 |
(RO, since: 5.9) |
| 1960 |
.sp |
| 1961 |
Shows the numeric value of the limit flags. |
| 1962 |
If 0, means no limit implied. |
| 1963 |
.TP |
| 1964 |
.B max_exclusive |
| 1965 |
(RO, since: 5.9) |
| 1966 |
.sp |
| 1967 |
Shows the limits on exclusively owned bytes. |
| 1968 |
.TP |
| 1969 |
.B max_referenced |
| 1970 |
(RO, since: 5.9) |
| 1971 |
.sp |
| 1972 |
Shows the limits on referenced bytes. |
| 1973 |
.TP |
| 1974 |
.B referenced |
| 1975 |
(RO, since: 5.9) |
| 1976 |
.sp |
| 1977 |
Shows the referenced bytes of the qgroup. |
| 1978 |
.TP |
| 1979 |
.B rsv_data |
| 1980 |
(RO, since: 5.9) |
| 1981 |
.sp |
| 1982 |
Shows the reserved bytes for data. |
| 1983 |
.TP |
| 1984 |
.B rsv_meta_pertrans |
| 1985 |
(RO, since: 5.9) |
| 1986 |
.sp |
| 1987 |
Shows the reserved bytes for per transaction metadata. |
| 1988 |
.TP |
| 1989 |
.B rsv_meta_prealloc |
| 1990 |
(RO, since: 5.9) |
| 1991 |
.sp |
| 1992 |
Shows the reserved bytes for preallocated metadata. |
| 1993 |
.UNINDENT |
| 1994 |
.SS UUID/discard |
| 1995 |
.sp |
| 1996 |
Files in \fB/sys/fs/btrfs/<UUID>/discard/\fP directory are: |
| 1997 |
.INDENT 0.0 |
| 1998 |
.TP |
| 1999 |
.B discardable_bytes |
| 2000 |
(RO, since: 6.1) |
| 2001 |
.sp |
| 2002 |
Shows amount of bytes that can be discarded in the async discard and |
| 2003 |
nodiscard mode. |
| 2004 |
.TP |
| 2005 |
.B discardable_extents |
| 2006 |
(RO, since: 6.1) |
| 2007 |
.sp |
| 2008 |
Shows number of extents to be discarded in the async discard and |
| 2009 |
nodiscard mode. |
| 2010 |
.TP |
| 2011 |
.B discard_bitmap_bytes |
| 2012 |
(RO, since: 6.1) |
| 2013 |
.sp |
| 2014 |
Shows amount of discarded bytes from data tracked as bitmaps. |
| 2015 |
.TP |
| 2016 |
.B discard_extent_bytes |
| 2017 |
(RO, since: 6.1) |
| 2018 |
.sp |
| 2019 |
Shows amount of discarded extents from data tracked as bitmaps. |
| 2020 |
.TP |
| 2021 |
.B discard_bytes_saved |
| 2022 |
(RO, since: 6.1) |
| 2023 |
.sp |
| 2024 |
Shows the amount of bytes that were reallocated without being discarded. |
| 2025 |
.TP |
| 2026 |
.B kbps_limit |
| 2027 |
(RW, since: 6.1) |
| 2028 |
.sp |
| 2029 |
Tunable limit of kilobytes per second issued as discard IO in the async |
| 2030 |
discard mode. |
| 2031 |
.TP |
| 2032 |
.B iops_limit |
| 2033 |
(RW, since: 6.1) |
| 2034 |
.sp |
| 2035 |
Tunable limit of number of discard IO operations to be issued in the |
| 2036 |
async discard mode. |
| 2037 |
.TP |
| 2038 |
.B max_discard_size |
| 2039 |
(RW, since: 6.1) |
| 2040 |
.sp |
| 2041 |
Tunable limit for size of one IO discard request. |
| 2042 |
.UNINDENT |
| 2043 |
.SH FILESYSTEM EXCLUSIVE OPERATIONS |
| 2044 |
.sp |
| 2045 |
There are several operations that affect the whole filesystem and cannot be run |
| 2046 |
in parallel. Attempt to start one while another is running will fail (see |
| 2047 |
exceptions below). |
| 2048 |
.sp |
| 2049 |
Since kernel 5.10 the currently running operation can be obtained from |
| 2050 |
\fB/sys/fs/UUID/exclusive_operation\fP with following values and operations: |
| 2051 |
.INDENT 0.0 |
| 2052 |
.IP \(bu 2 |
| 2053 |
balance |
| 2054 |
.IP \(bu 2 |
| 2055 |
balance paused (since 5.17) |
| 2056 |
.IP \(bu 2 |
| 2057 |
device add |
| 2058 |
.IP \(bu 2 |
| 2059 |
device delete |
| 2060 |
.IP \(bu 2 |
| 2061 |
device replace |
| 2062 |
.IP \(bu 2 |
| 2063 |
resize |
| 2064 |
.IP \(bu 2 |
| 2065 |
swapfile activate |
| 2066 |
.IP \(bu 2 |
| 2067 |
none |
| 2068 |
.UNINDENT |
| 2069 |
.sp |
| 2070 |
Enqueuing is supported for several btrfs subcommands so they can be started |
| 2071 |
at once and then serialized. |
| 2072 |
.sp |
| 2073 |
There\(aqs an exception when a paused balance allows to start a device add |
| 2074 |
operation as they don\(aqt really collide and this can be used to add more space |
| 2075 |
for the balance to finish. |
| 2076 |
.SH FILESYSTEM LIMITS |
| 2077 |
.INDENT 0.0 |
| 2078 |
.TP |
| 2079 |
.B maximum file name length |
| 2080 |
255 |
| 2081 |
.sp |
| 2082 |
This limit is imposed by Linux VFS, the structures of BTRFS could store |
| 2083 |
larger file names. |
| 2084 |
.TP |
| 2085 |
.B maximum symlink target length |
| 2086 |
depends on the \fInodesize\fP value, for 4KiB it\(aqs 3949 bytes, for larger nodesize |
| 2087 |
it\(aqs 4095 due to the system limit PATH_MAX |
| 2088 |
.sp |
| 2089 |
The symlink target may not be a valid path, i.e. the path name components |
| 2090 |
can exceed the limits (NAME_MAX), there\(aqs no content validation at symlink(3) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man3/\:symlink\:.3\:.html> |
| 2091 |
creation. |
| 2092 |
.TP |
| 2093 |
.B maximum number of inodes |
| 2094 |
2\s-2\u64\d\s0 but depends on the available metadata space as the inodes are created |
| 2095 |
dynamically |
| 2096 |
.sp |
| 2097 |
Each subvolume is an independent namespace of inodes and thus their |
| 2098 |
numbers, so the limit is per subvolume, not for the whole filesystem. |
| 2099 |
.TP |
| 2100 |
.B inode numbers |
| 2101 |
minimum number: 256 (for subvolumes), regular files and directories: 257, |
| 2102 |
maximum number: (2\s-2\u64\d\s0 \- 256) |
| 2103 |
.sp |
| 2104 |
The inode numbers that can be assigned to user created files are from |
| 2105 |
the whole 64bit space except first 256 and last 256 in that range that |
| 2106 |
are reserved for internal b\-tree identifiers. |
| 2107 |
.TP |
| 2108 |
.B maximum file length |
| 2109 |
inherent limit of BTRFS is 2\s-2\u64\d\s0 (16 EiB) but the practical |
| 2110 |
limit of Linux VFS is 2\s-2\u63\d\s0 (8 EiB) |
| 2111 |
.TP |
| 2112 |
.B maximum number of subvolumes |
| 2113 |
the subvolume ids can go up to 2\s-2\u48\d\s0 but the number of actual subvolumes |
| 2114 |
depends on the available metadata space |
| 2115 |
.sp |
| 2116 |
The space consumed by all subvolume metadata includes bookkeeping of |
| 2117 |
shared extents can be large (MiB, GiB). The range is not the full 64bit |
| 2118 |
range because of qgroups that use the upper 16 bits for another |
| 2119 |
purposes. |
| 2120 |
.TP |
| 2121 |
.B maximum number of hardlinks of a file in a directory |
| 2122 |
65536 when the \fIextref\fP feature is turned on during mkfs (default), roughly |
| 2123 |
100 otherwise and depends on file name length that fits into one metadata node |
| 2124 |
.TP |
| 2125 |
.B minimum filesystem size |
| 2126 |
the minimal size of each device depends on the \fImixed\-bg\fP feature, without that |
| 2127 |
(the default) it\(aqs about 109MiB, with mixed\-bg it\(aqs is 16MiB |
| 2128 |
.UNINDENT |
| 2129 |
.SH BOOTLOADER SUPPORT |
| 2130 |
.sp |
| 2131 |
GRUB2 (\%<https://\:www\:.gnu\:.org/\:software/\:grub>) has the most advanced support of |
| 2132 |
booting from BTRFS with respect to features. |
| 2133 |
.sp |
| 2134 |
U\-Boot (\%<https://\:www\:.denx\:.de/\:wiki/\:U-Boot/>) has decent support for booting but |
| 2135 |
not all BTRFS features are implemented, check the documentation. |
| 2136 |
.sp |
| 2137 |
In general, the first 1MiB on each device is unused with the exception of |
| 2138 |
primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be |
| 2139 |
freely used by bootloaders or for other system information. Note that booting |
| 2140 |
from a filesystem on zoned device \%<> is not supported. |
| 2141 |
.SH FILE ATTRIBUTES |
| 2142 |
.sp |
| 2143 |
The btrfs filesystem supports setting file attributes or flags. Note there are |
| 2144 |
old and new interfaces, with confusing names. The following list should clarify |
| 2145 |
that: |
| 2146 |
.INDENT 0.0 |
| 2147 |
.IP \(bu 2 |
| 2148 |
\fIattributes\fP: chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> or lsattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:lsattr\:.1\:.html> utilities (the ioctls are |
| 2149 |
FS_IOC_GETFLAGS and FS_IOC_SETFLAGS), due to the ioctl names the attributes |
| 2150 |
are also called flags |
| 2151 |
.IP \(bu 2 |
| 2152 |
\fIxflags\fP: to distinguish from the previous, it\(aqs extended flags, with tunable |
| 2153 |
bits similar to the attributes but extensible and new bits will be added in |
| 2154 |
the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR but they |
| 2155 |
are not related to extended attributes that are also called xattrs), there\(aqs |
| 2156 |
no standard tool to change the bits, there\(aqs support in xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> as |
| 2157 |
command \fBxfs_io \-c chattr\fP |
| 2158 |
.UNINDENT |
| 2159 |
.sp |
| 2160 |
Attributes have constraints associated and not all combinations can be set, the |
| 2161 |
order of setting them also matters. Most attributes apply to files and |
| 2162 |
directories but the semantics may differ. For directories the attribute may |
| 2163 |
only mean to set this attribute to all new files (\fIinheritable\fP in the list |
| 2164 |
below). Some attributes need root privileges to be set. |
| 2165 |
.SS Attributes |
| 2166 |
.INDENT 0.0 |
| 2167 |
.TP |
| 2168 |
.B a |
| 2169 |
(file, dir, root) |
| 2170 |
\fIappend only\fP, new writes are always written at the end of the file |
| 2171 |
.TP |
| 2172 |
.B A |
| 2173 |
(file, dir) |
| 2174 |
\fIno atime updates\fP |
| 2175 |
.TP |
| 2176 |
.B c |
| 2177 |
(file, dir, inherited) |
| 2178 |
\fIcompress data\fP, all data written after this attribute is set will be compressed. |
| 2179 |
Please note that compression is also affected by the mount options or the parent |
| 2180 |
directory attributes. |
| 2181 |
.sp |
| 2182 |
When set on a directory, all newly created files will inherit this attribute. |
| 2183 |
This attribute cannot be set with \(aqm\(aq at the same time. |
| 2184 |
.TP |
| 2185 |
.B C |
| 2186 |
(file, dir, inherited) |
| 2187 |
\fIno copy\-on\-write\fP, file data modifications are done in\-place |
| 2188 |
.sp |
| 2189 |
When set on a directory, all newly created files will inherit this attribute. |
| 2190 |
.sp |
| 2191 |
\fBNote:\fP |
| 2192 |
.INDENT 7.0 |
| 2193 |
.INDENT 3.5 |
| 2194 |
Due to implementation limitations, this flag can be set/unset only on |
| 2195 |
empty files. |
| 2196 |
.UNINDENT |
| 2197 |
.UNINDENT |
| 2198 |
.TP |
| 2199 |
.B d |
| 2200 |
(file) |
| 2201 |
\fIno dump\fP, makes sense with 3rd party tools like dump(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:dump\:.8\:.html>, on BTRFS the |
| 2202 |
attribute can be set/unset but no other special handling is done |
| 2203 |
.TP |
| 2204 |
.B D |
| 2205 |
(dir) |
| 2206 |
\fIsynchronous directory updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP |
| 2207 |
and \fIO_DSYNC\fP |
| 2208 |
.TP |
| 2209 |
.B i |
| 2210 |
(file, dir, root) |
| 2211 |
\fIimmutable\fP, no file data and metadata changes allowed even to the root user as |
| 2212 |
long as this attribute is set (obviously the exception is unsetting the attribute) |
| 2213 |
.TP |
| 2214 |
.B m |
| 2215 |
(file, dir) |
| 2216 |
\fIno compression\fP, permanently turn off compression on the given file. Any |
| 2217 |
compression mount options will not affect this file. (chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> support added in |
| 2218 |
1.46.2) |
| 2219 |
.sp |
| 2220 |
When set on a directory, all newly created files will inherit this attribute. |
| 2221 |
This attribute cannot be set with \fIc\fP at the same time. |
| 2222 |
.TP |
| 2223 |
.B S |
| 2224 |
(file) |
| 2225 |
\fIsynchronous updates\fP, for more details search open(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:open\:.2\:.html> for \fIO_SYNC\fP and |
| 2226 |
\fIO_DSYNC\fP |
| 2227 |
.TP |
| 2228 |
.B V |
| 2229 |
(file, read\-only) |
| 2230 |
\fIfs\-verity enabled\fP on the file |
| 2231 |
.UNINDENT |
| 2232 |
.sp |
| 2233 |
No other attributes are supported. For the complete list please refer to the |
| 2234 |
chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html> manual page. |
| 2235 |
.SS XFLAGS |
| 2236 |
.sp |
| 2237 |
There\(aqs an overlap of letters assigned to the bits with the attributes, this list |
| 2238 |
refers to what xfs_io(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:xfs_io\:.8\:.html> provides: |
| 2239 |
.INDENT 0.0 |
| 2240 |
.TP |
| 2241 |
.B i |
| 2242 |
\fIimmutable\fP, same as the attribute |
| 2243 |
.TP |
| 2244 |
.B a |
| 2245 |
\fIappend only\fP, same as the attribute |
| 2246 |
.TP |
| 2247 |
.B s |
| 2248 |
\fIsynchronous updates\fP, same as the attribute \fIS\fP |
| 2249 |
.TP |
| 2250 |
.B A |
| 2251 |
\fIno atime updates\fP, same as the attribute |
| 2252 |
.TP |
| 2253 |
.B d |
| 2254 |
\fIno dump\fP, same as the attribute |
| 2255 |
.UNINDENT |
| 2256 |
.SH ZONED MODE |
| 2257 |
.sp |
| 2258 |
Since version 5.12 btrfs supports so called \fIzoned mode\fP\&. This is a special |
| 2259 |
on\-disk format and allocation/write strategy that\(aqs friendly to zoned devices. |
| 2260 |
In short, a device is partitioned into fixed\-size zones and each zone can be |
| 2261 |
updated by append\-only manner, or reset. As btrfs has no fixed data structures, |
| 2262 |
except the super blocks, the zoned mode only requires block placement that |
| 2263 |
follows the device constraints. You can learn about the whole architecture at |
| 2264 |
\%<https://\:zonedstorage\:.io> . |
| 2265 |
.sp |
| 2266 |
The devices are also called SMR/ZBC/ZNS, in \fIhost\-managed\fP mode. Note that |
| 2267 |
there are devices that appear as non\-zoned but actually are, this is |
| 2268 |
\fIdrive\-managed\fP and using zoned mode won\(aqt help. |
| 2269 |
.sp |
| 2270 |
The zone size depends on the device, typical sizes are 256MiB or 1GiB. In |
| 2271 |
general it must be a power of two. Emulated zoned devices like \fInull_blk\fP allow |
| 2272 |
to set various zone sizes. |
| 2273 |
.SS Requirements, limitations |
| 2274 |
.INDENT 0.0 |
| 2275 |
.IP \(bu 2 |
| 2276 |
all devices must have the same zone size |
| 2277 |
.IP \(bu 2 |
| 2278 |
maximum zone size is 8GiB |
| 2279 |
.IP \(bu 2 |
| 2280 |
minimum zone size is 4MiB |
| 2281 |
.IP \(bu 2 |
| 2282 |
mixing zoned and non\-zoned devices is possible, the zone writes are emulated, |
| 2283 |
but this is namely for testing |
| 2284 |
.IP \(bu 2 |
| 2285 |
the super block is handled in a special way and is at different locations than on a non\-zoned filesystem: |
| 2286 |
.INDENT 2.0 |
| 2287 |
.IP \(bu 2 |
| 2288 |
primary: 0B (and the next two zones) |
| 2289 |
.IP \(bu 2 |
| 2290 |
secondary: 512GiB (and the next two zones) |
| 2291 |
.IP \(bu 2 |
| 2292 |
tertiary: 4TiB (4096GiB, and the next two zones) |
| 2293 |
.UNINDENT |
| 2294 |
.UNINDENT |
| 2295 |
.SS Incompatible features |
| 2296 |
.sp |
| 2297 |
The main constraint of the zoned devices is lack of in\-place update of the data. |
| 2298 |
This is inherently incompatible with some features: |
| 2299 |
.INDENT 0.0 |
| 2300 |
.IP \(bu 2 |
| 2301 |
NODATACOW \- overwrite in\-place, cannot create such files |
| 2302 |
.IP \(bu 2 |
| 2303 |
fallocate \- preallocating space for in\-place first write |
| 2304 |
.IP \(bu 2 |
| 2305 |
mixed\-bg \- unordered writes to data and metadata, fixing that means using |
| 2306 |
separate data and metadata block groups |
| 2307 |
.IP \(bu 2 |
| 2308 |
booting \- the zone at offset 0 contains superblock, resetting the zone would |
| 2309 |
destroy the bootloader data |
| 2310 |
.UNINDENT |
| 2311 |
.sp |
| 2312 |
Initial support lacks some features but they\(aqre planned: |
| 2313 |
.INDENT 0.0 |
| 2314 |
.IP \(bu 2 |
| 2315 |
only single (data, metadata) and DUP (metadata) profile is supported |
| 2316 |
.IP \(bu 2 |
| 2317 |
fstrim \- due to dependency on free space cache v1 |
| 2318 |
.UNINDENT |
| 2319 |
.SS Super block |
| 2320 |
.sp |
| 2321 |
As said above, super block is handled in a special way. In order to be crash |
| 2322 |
safe, at least one zone in a known location must contain a valid superblock. |
| 2323 |
This is implemented as a ring buffer in two consecutive zones, starting from |
| 2324 |
known offsets 0B, 512GiB and 4TiB. |
| 2325 |
.sp |
| 2326 |
The values are different than on non\-zoned devices. Each new super block is |
| 2327 |
appended to the end of the zone, once it\(aqs filled, the zone is reset and writes |
| 2328 |
continue to the next one. Looking up the latest super block needs to read |
| 2329 |
offsets of both zones and determine the last written version. |
| 2330 |
.sp |
| 2331 |
The amount of space reserved for super block depends on the zone size. The |
| 2332 |
secondary and tertiary copies are at distant offsets as the capacity of the |
| 2333 |
devices is expected to be large, tens of terabytes. Maximum zone size supported |
| 2334 |
is 8GiB, which would mean that e.g. offset 0\-16GiB would be reserved just for |
| 2335 |
the super block on a hypothetical device of that zone size. This is wasteful |
| 2336 |
but required to guarantee crash safety. |
| 2337 |
.SS Zone reclaim, garbage collection |
| 2338 |
.sp |
| 2339 |
As the zones are append\-only, overwriting data or COW changes in metadata |
| 2340 |
make parts of the zones used but not connected to the filesystem structures. |
| 2341 |
This makes the space unusable and grows over time. Once the ratio hits a |
| 2342 |
(configurable) threshold a background reclaim process is started and relocates |
| 2343 |
the remaining blocks in use to a new zone. The old one is reset and can be used |
| 2344 |
again. |
| 2345 |
.sp |
| 2346 |
This process may take some time depending on other background work or |
| 2347 |
amount of new data written. It is possible to hit an intermittent ENOSPC. |
| 2348 |
Some devices also limit number of active zones. |
| 2349 |
.SS Devices |
| 2350 |
.SS Real hardware |
| 2351 |
.sp |
| 2352 |
The WD Ultrastar series 600 advertises HM\-SMR, i.e. the host\-managed zoned |
| 2353 |
mode. There are two more: DA (device managed, no zoned information exported to |
| 2354 |
the system), HA (host aware, can be used as regular disk but zoned writes |
| 2355 |
improve performance). There are not many devices available at the moment, the |
| 2356 |
information about exact zoned mode is hard to find, check data sheets or |
| 2357 |
community sources gathering information from real devices. |
| 2358 |
.sp |
| 2359 |
Note: zoned mode won\(aqt work with DM\-SMR disks. |
| 2360 |
.INDENT 0.0 |
| 2361 |
.IP \(bu 2 |
| 2362 |
Ultrastar® DC ZN540 NVMe ZNS SSD (product |
| 2363 |
brief \%<https://\:documents\:.westerndigital\:.com/\:content/\:dam/\:doc-library/\:en_us/\:assets/\:public/\:western-digital/\:collateral/\:product-brief/\:product-brief-ultrastar-dc-zn540\:.pdf>) |
| 2364 |
.UNINDENT |
| 2365 |
.SS Emulated: null_blk |
| 2366 |
.sp |
| 2367 |
The driver \fInull_blk\fP provides memory backed device and is suitable for |
| 2368 |
testing. There are some quirks setting up the devices. The module must be |
| 2369 |
loaded with \fInr_devices=0\fP or the numbering of device nodes will be offset. The |
| 2370 |
\fIconfigfs\fP must be mounted at \fI/sys/kernel/config\fP and the administration of |
| 2371 |
the null_blk devices is done in \fI/sys/kernel/config/nullb\fP\&. The device nodes |
| 2372 |
are named like \fB/dev/nullb0\fP and are numbered sequentially. NOTE: the device |
| 2373 |
name may be different than the named directory in sysfs! |
| 2374 |
.sp |
| 2375 |
Setup: |
| 2376 |
.INDENT 0.0 |
| 2377 |
.INDENT 3.5 |
| 2378 |
.sp |
| 2379 |
.EX |
| 2380 |
modprobe\ configfs |
| 2381 |
modprobe\ null_blk\ nr_devices=0 |
| 2382 |
.EE |
| 2383 |
.UNINDENT |
| 2384 |
.UNINDENT |
| 2385 |
.sp |
| 2386 |
Create a device \fImydev\fP, assuming no other previously created devices, size is |
| 2387 |
2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal |
| 2388 |
example taking defaults: |
| 2389 |
.INDENT 0.0 |
| 2390 |
.INDENT 3.5 |
| 2391 |
.sp |
| 2392 |
.EX |
| 2393 |
cd\ /sys/kernel/config/nullb/ |
| 2394 |
mkdir\ mydev |
| 2395 |
cd\ mydev |
| 2396 |
echo\ 2048\ >\ size |
| 2397 |
echo\ 1\ >\ zoned |
| 2398 |
echo\ 1\ >\ memory_backed |
| 2399 |
echo\ 256\ >\ zone_size |
| 2400 |
echo\ 1\ >\ power |
| 2401 |
.EE |
| 2402 |
.UNINDENT |
| 2403 |
.UNINDENT |
| 2404 |
.sp |
| 2405 |
This will create a device \fB/dev/nullb0\fP and the value of file \fIindex\fP will |
| 2406 |
match the ending number of the device node. |
| 2407 |
.sp |
| 2408 |
Remove the device: |
| 2409 |
.INDENT 0.0 |
| 2410 |
.INDENT 3.5 |
| 2411 |
.sp |
| 2412 |
.EX |
| 2413 |
rmdir\ /sys/kernel/config/nullb/mydev |
| 2414 |
.EE |
| 2415 |
.UNINDENT |
| 2416 |
.UNINDENT |
| 2417 |
.sp |
| 2418 |
Then continue with \fBmkfs.btrfs /dev/nullb0\fP, the zoned mode is auto\-detected. |
| 2419 |
.sp |
| 2420 |
For convenience, there\(aqs a script wrapping the basic null_blk management operations |
| 2421 |
\%<https://\:github\:.com/\:kdave/\:nullb\:.git>, the above commands become: |
| 2422 |
.INDENT 0.0 |
| 2423 |
.INDENT 3.5 |
| 2424 |
.sp |
| 2425 |
.EX |
| 2426 |
nullb setup |
| 2427 |
nullb create \-s 2g \-z 256 |
| 2428 |
mkfs.btrfs /dev/nullb0 |
| 2429 |
\&... |
| 2430 |
nullb rm nullb0 |
| 2431 |
.EE |
| 2432 |
.UNINDENT |
| 2433 |
.UNINDENT |
| 2434 |
.SS Emulated: TCMU runner |
| 2435 |
.sp |
| 2436 |
TCMU is a framework to emulate SCSI devices in userspace, providing various |
| 2437 |
backends for the storage, with zoned support as well. A file\-backed zoned |
| 2438 |
device can provide more options for larger storage and zone size. Please follow |
| 2439 |
the instructions at \%<https://\:zonedstorage\:.io/\:projects/\:tcmu-runner/> . |
| 2440 |
.SS Compatibility, incompatibility |
| 2441 |
.INDENT 0.0 |
| 2442 |
.IP \(bu 2 |
| 2443 |
the feature sets an incompat bit and requires new kernel to access the |
| 2444 |
filesystem (for both read and write) |
| 2445 |
.IP \(bu 2 |
| 2446 |
superblock needs to be handled in a special way, there are still 3 copies |
| 2447 |
but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a |
| 2448 |
ring buffer of the superblocks, finding the latest one needs reading it from |
| 2449 |
the write pointer or do a full scan of the zones |
| 2450 |
.IP \(bu 2 |
| 2451 |
mixing zoned and non zoned devices is possible (zones are emulated) but is |
| 2452 |
recommended only for testing |
| 2453 |
.IP \(bu 2 |
| 2454 |
mixing zoned devices with different zone sizes is not possible |
| 2455 |
.IP \(bu 2 |
| 2456 |
zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB |
| 2457 |
or 1GiB, larger size is expected, maximum zone size supported by btrfs is |
| 2458 |
8GiB |
| 2459 |
.UNINDENT |
| 2460 |
.SS Status, stability, reporting bugs |
| 2461 |
.sp |
| 2462 |
The zoned mode has been released in 5.12 and there are still some rough edges |
| 2463 |
and corner cases one can hit during testing. Please report bugs to |
| 2464 |
\%<https://\:github\:.com/\:naota/\:linux/\:issues/> . |
| 2465 |
.SS References |
| 2466 |
.INDENT 0.0 |
| 2467 |
.IP \(bu 2 |
| 2468 |
\%<https://\:zonedstorage\:.io> |
| 2469 |
.INDENT 2.0 |
| 2470 |
.IP \(bu 2 |
| 2471 |
\%<https://\:zonedstorage\:.io/\:projects/\:libzbc/> \-\- \fIlibzbc\fP is library and set |
| 2472 |
of tools to directly manipulate devices with ZBC/ZAC support |
| 2473 |
.IP \(bu 2 |
| 2474 |
\%<https://\:zonedstorage\:.io/\:projects/\:libzbd/> \-\- \fIlibzbd\fP uses the kernel |
| 2475 |
provided zoned block device interface based on the ioctl() system calls |
| 2476 |
.UNINDENT |
| 2477 |
.IP \(bu 2 |
| 2478 |
\%<https://\:hddscan\:.com/\:blog/\:2020/\:hdd-wd-smr\:.html> \-\- some details about exact device types |
| 2479 |
.IP \(bu 2 |
| 2480 |
\%<https://\:lwn\:.net/\:Articles/\:853308/> \-\- \fIBtrfs on zoned block devices\fP |
| 2481 |
.IP \(bu 2 |
| 2482 |
\%<https://\:www\:.usenix\:.org/\:conference/\:vault20/\:presentation/\:bjorling> \-\- Zone |
| 2483 |
Append: A New Way of Writing to Zoned Storage |
| 2484 |
.UNINDENT |
| 2485 |
.SH CONTROL DEVICE |
| 2486 |
.sp |
| 2487 |
There\(aqs a character special device \fB/dev/btrfs\-control\fP with major and minor |
| 2488 |
numbers 10 and 234 (the device can be found under the \fImisc\fP category). |
| 2489 |
.INDENT 0.0 |
| 2490 |
.INDENT 3.5 |
| 2491 |
.sp |
| 2492 |
.EX |
| 2493 |
$ ls \-l /dev/btrfs\-control |
| 2494 |
crw\-\-\-\-\-\-\- 1 root root 10, 234 Jan 1 12:00 /dev/btrfs\-control |
| 2495 |
.EE |
| 2496 |
.UNINDENT |
| 2497 |
.UNINDENT |
| 2498 |
.sp |
| 2499 |
The device accepts some ioctl calls that can perform following actions on the |
| 2500 |
filesystem module: |
| 2501 |
.INDENT 0.0 |
| 2502 |
.IP \(bu 2 |
| 2503 |
scan devices for btrfs filesystem (i.e. to let multi\-device filesystems mount |
| 2504 |
automatically) and register them with the kernel module |
| 2505 |
.IP \(bu 2 |
| 2506 |
similar to scan, but also wait until the device scanning process is finished |
| 2507 |
for a given filesystem |
| 2508 |
.IP \(bu 2 |
| 2509 |
get the supported features (can be also found under \fB/sys/fs/btrfs/features\fP) |
| 2510 |
.UNINDENT |
| 2511 |
.sp |
| 2512 |
The device is created when btrfs is initialized, either as a module or a |
| 2513 |
built\-in functionality and makes sense only in connection with that. Running |
| 2514 |
e.g. mkfs without the module loaded will not register the device and will |
| 2515 |
probably warn about that. |
| 2516 |
.sp |
| 2517 |
In rare cases when the module is loaded but the device is not present (most |
| 2518 |
likely accidentally deleted), it\(aqs possible to recreate it by |
| 2519 |
.INDENT 0.0 |
| 2520 |
.INDENT 3.5 |
| 2521 |
.sp |
| 2522 |
.EX |
| 2523 |
# mknod \-\-mode=600 /dev/btrfs\-control c 10 234 |
| 2524 |
.EE |
| 2525 |
.UNINDENT |
| 2526 |
.UNINDENT |
| 2527 |
.sp |
| 2528 |
or (since 5.11) by a convenience command |
| 2529 |
.INDENT 0.0 |
| 2530 |
.INDENT 3.5 |
| 2531 |
.sp |
| 2532 |
.EX |
| 2533 |
# btrfs rescue create\-control\-device |
| 2534 |
.EE |
| 2535 |
.UNINDENT |
| 2536 |
.UNINDENT |
| 2537 |
.sp |
| 2538 |
The control device is not strictly required but the device scanning will not |
| 2539 |
work and a workaround would need to be used to mount a multi\-device filesystem. |
| 2540 |
The mount option \fIdevice\fP can trigger the device scanning during mount, see |
| 2541 |
also \fBbtrfs device scan\fP\&. |
| 2542 |
.SH FILESYSTEM WITH MULTIPLE PROFILES |
| 2543 |
.sp |
| 2544 |
It is possible that a btrfs filesystem contains multiple block group profiles |
| 2545 |
of the same type. This could happen when a profile conversion using balance |
| 2546 |
filters is interrupted (see btrfs\-balance(8) \%<>). Some |
| 2547 |
\fBbtrfs\fP commands perform |
| 2548 |
a test to detect this kind of condition and print a warning like this: |
| 2549 |
.INDENT 0.0 |
| 2550 |
.INDENT 3.5 |
| 2551 |
.sp |
| 2552 |
.EX |
| 2553 |
WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq. |
| 2554 |
WARNING: Data: single, raid1 |
| 2555 |
WARNING: Metadata: single, raid1 |
| 2556 |
.EE |
| 2557 |
.UNINDENT |
| 2558 |
.UNINDENT |
| 2559 |
.sp |
| 2560 |
The corresponding output of \fBbtrfs filesystem df\fP might look like: |
| 2561 |
.INDENT 0.0 |
| 2562 |
.INDENT 3.5 |
| 2563 |
.sp |
| 2564 |
.EX |
| 2565 |
WARNING: Multiple block group profiles detected, see \(aqman btrfs(5)\(aq. |
| 2566 |
WARNING: Data: single, raid1 |
| 2567 |
WARNING: Metadata: single, raid1 |
| 2568 |
Data, RAID1: total=832.00MiB, used=0.00B |
| 2569 |
Data, single: total=1.63GiB, used=0.00B |
| 2570 |
System, single: total=4.00MiB, used=16.00KiB |
| 2571 |
Metadata, single: total=8.00MiB, used=112.00KiB |
| 2572 |
Metadata, RAID1: total=64.00MiB, used=32.00KiB |
| 2573 |
GlobalReserve, single: total=16.25MiB, used=0.00B |
| 2574 |
.EE |
| 2575 |
.UNINDENT |
| 2576 |
.UNINDENT |
| 2577 |
.sp |
| 2578 |
There\(aqs more than one line for type \fIData\fP and \fIMetadata\fP, while the profiles |
| 2579 |
are \fIsingle\fP and \fIRAID1\fP\&. |
| 2580 |
.sp |
| 2581 |
This state of the filesystem OK but most likely needs the user/administrator to |
| 2582 |
take an action and finish the interrupted tasks. This cannot be easily done |
| 2583 |
automatically, also the user knows the expected final profiles. |
| 2584 |
.sp |
| 2585 |
In the example above, the filesystem started as a single device and \fIsingle\fP |
| 2586 |
block group profile. Then another device was added, followed by balance with |
| 2587 |
\fIconvert=raid1\fP but for some reason hasn\(aqt finished. Restarting the balance |
| 2588 |
with \fIconvert=raid1\fP will continue and end up with filesystem with all block |
| 2589 |
group profiles \fIRAID1\fP\&. |
| 2590 |
.sp |
| 2591 |
\fBNote:\fP |
| 2592 |
.INDENT 0.0 |
| 2593 |
.INDENT 3.5 |
| 2594 |
If you\(aqre familiar with balance filters, you can use |
| 2595 |
\fIconvert=raid1,profiles=single,soft\fP, which will take only the unconverted |
| 2596 |
\fIsingle\fP profiles and convert them to \fIraid1\fP\&. This may speed up the conversion |
| 2597 |
as it would not try to rewrite the already convert \fIraid1\fP profiles. |
| 2598 |
.UNINDENT |
| 2599 |
.UNINDENT |
| 2600 |
.sp |
| 2601 |
Having just one profile is desired as this also clearly defines the profile of |
| 2602 |
newly allocated block groups, otherwise this depends on internal allocation |
| 2603 |
policy. When there are multiple profiles present, the order of selection is |
| 2604 |
RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are |
| 2605 |
satisfied. |
| 2606 |
.sp |
| 2607 |
Commands that print the warning were chosen so they\(aqre brought to user |
| 2608 |
attention when the filesystem state is being changed in that regard. This is: |
| 2609 |
\fBdevice add\fP, \fBdevice delete\fP, \fBbalance cancel\fP, \fBbalance pause\fP\&. Commands |
| 2610 |
that report space usage: \fBfilesystem df\fP, \fBdevice usage\fP\&. The command |
| 2611 |
\fBfilesystem usage\fP provides a line in the overall summary: |
| 2612 |
.INDENT 0.0 |
| 2613 |
.INDENT 3.5 |
| 2614 |
.sp |
| 2615 |
.EX |
| 2616 |
Multiple profiles: yes (data, metadata) |
| 2617 |
.EE |
| 2618 |
.UNINDENT |
| 2619 |
.UNINDENT |
| 2620 |
.SH SEEDING DEVICE |
| 2621 |
.sp |
| 2622 |
The COW mechanism and multiple devices under one hood enable an interesting |
| 2623 |
concept, called a seeding device: extending a read\-only filesystem on a |
| 2624 |
device with another device that captures all writes. For example |
| 2625 |
imagine an immutable golden image of an operating system enhanced with another |
| 2626 |
device that allows to use the data from the golden image and normal operation. |
| 2627 |
This idea originated on CD\-ROMs with base OS and allowing to use them for live |
| 2628 |
systems, but this became obsolete. There are technologies providing similar |
| 2629 |
functionality, like unionmount \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Union_mount>, |
| 2630 |
overlayfs \%<https://\:en\:.wikipedia\:.org/\:wiki/\:OverlayFS> or |
| 2631 |
qcow2 \%<https://\:en\:.wikipedia\:.org/\:wiki/\:Qcow#\:qcow2> image snapshot. |
| 2632 |
.sp |
| 2633 |
The seeding device starts as a normal filesystem, once the contents is ready, |
| 2634 |
\fBbtrfstune \-S 1\fP is used to flag it as a seeding device. Mounting such device |
| 2635 |
will not allow any writes, except adding a new device by \fBbtrfs device add\fP\&. |
| 2636 |
Then the filesystem can be remounted as read\-write. |
| 2637 |
.sp |
| 2638 |
Given that the filesystem on the seeding device is always recognized as |
| 2639 |
read\-only, it can be used to seed multiple filesystems from one device at the |
| 2640 |
same time. The UUID that is normally attached to a device is automatically |
| 2641 |
changed to a random UUID on each mount. |
| 2642 |
.sp |
| 2643 |
\fBNote:\fP |
| 2644 |
.INDENT 0.0 |
| 2645 |
.INDENT 3.5 |
| 2646 |
Before v6.17 kernel, a seed device could have been mounted |
| 2647 |
independently along with sprouted filesystems. |
| 2648 |
But since 6.17 kernel, a seed device can only be mounted either through |
| 2649 |
a sprouted filesystem, or the seed device itself, not both at the same time. |
| 2650 |
.sp |
| 2651 |
This is to ensure a block device to have only a single filesystem bound |
| 2652 |
to it, so that runtime device missing events can be properly handled. |
| 2653 |
.UNINDENT |
| 2654 |
.UNINDENT |
| 2655 |
.sp |
| 2656 |
Once the seeding device is mounted, it needs the writable device. After adding |
| 2657 |
it, unmounting and mounting with \fBumount /path; mount /dev/writable |
| 2658 |
/path\fP or remounting read\-write with \fBremount \-o remount,rw\fP makes the |
| 2659 |
filesystem at \fB/path\fP ready for use. |
| 2660 |
.sp |
| 2661 |
\fBNote:\fP |
| 2662 |
.INDENT 0.0 |
| 2663 |
.INDENT 3.5 |
| 2664 |
There was a known bug with using remount to make the mount writeable: |
| 2665 |
remount will leave the filesystem in a state where it is unable to |
| 2666 |
clean deleted snapshots, so it will leak space until it is unmounted |
| 2667 |
and mounted properly. |
| 2668 |
.sp |
| 2669 |
That bug has been fixed in 5.11 and newer kernels. |
| 2670 |
.UNINDENT |
| 2671 |
.UNINDENT |
| 2672 |
.sp |
| 2673 |
Furthermore, deleting the seeding device from the filesystem can turn it into |
| 2674 |
a normal filesystem, provided that the writable device can also contain all the |
| 2675 |
data from the seeding device. |
| 2676 |
.sp |
| 2677 |
The seeding device flag can be cleared again by \fBbtrfstune \-f \-S 0\fP, e.g. |
| 2678 |
allowing to update with newer data but please note that this will invalidate |
| 2679 |
all existing filesystems that use this particular seeding device. This works |
| 2680 |
for some use cases, not for others, and the forcing flag to the command is |
| 2681 |
mandatory to avoid accidental mistakes. |
| 2682 |
.sp |
| 2683 |
Example how to create and use one seeding device: |
| 2684 |
.INDENT 0.0 |
| 2685 |
.INDENT 3.5 |
| 2686 |
.sp |
| 2687 |
.EX |
| 2688 |
# mkfs.btrfs /dev/sda |
| 2689 |
# mount /dev/sda /mnt/mnt1 |
| 2690 |
\&... fill mnt1 with data |
| 2691 |
# umount /mnt/mnt1 |
| 2692 |
|
| 2693 |
# btrfstune \-S 1 /dev/sda |
| 2694 |
|
| 2695 |
# mount /dev/sda /mnt/mnt1 |
| 2696 |
# btrfs device add /dev/sdb /mnt/mnt1 |
| 2697 |
# umount /mnt/mnt1 |
| 2698 |
# mount /dev/sdb /mnt/mnt1 |
| 2699 |
\&... /mnt/mnt1 is now writable |
| 2700 |
.EE |
| 2701 |
.UNINDENT |
| 2702 |
.UNINDENT |
| 2703 |
.sp |
| 2704 |
Now \fB/mnt/mnt1\fP can be used normally. The device \fB/dev/sda\fP can be mounted |
| 2705 |
again with a another writable device: |
| 2706 |
.INDENT 0.0 |
| 2707 |
.INDENT 3.5 |
| 2708 |
.sp |
| 2709 |
.EX |
| 2710 |
# mount /dev/sda /mnt/mnt2 |
| 2711 |
# btrfs device add /dev/sdc /mnt/mnt2 |
| 2712 |
# umount /mnt/mnt2 |
| 2713 |
# mount /dev/sdc /mnt/mnt2 |
| 2714 |
\&... /mnt/mnt2 is now writable |
| 2715 |
.EE |
| 2716 |
.UNINDENT |
| 2717 |
.UNINDENT |
| 2718 |
.sp |
| 2719 |
The writable device (file:\fI/dev/sdb\fP) can be decoupled from the seeding device and |
| 2720 |
used independently: |
| 2721 |
.INDENT 0.0 |
| 2722 |
.INDENT 3.5 |
| 2723 |
.sp |
| 2724 |
.EX |
| 2725 |
# btrfs device delete /dev/sda /mnt/mnt1 |
| 2726 |
.EE |
| 2727 |
.UNINDENT |
| 2728 |
.UNINDENT |
| 2729 |
.sp |
| 2730 |
As the contents originated in the seeding device, it\(aqs possible to turn |
| 2731 |
\fB/dev/sdb\fP to a seeding device again and repeat the whole process. |
| 2732 |
.sp |
| 2733 |
A few things to note: |
| 2734 |
.INDENT 0.0 |
| 2735 |
.IP \(bu 2 |
| 2736 |
it\(aqs recommended to use only single device for the seeding device, it works |
| 2737 |
for multiple devices but the \fIsingle\fP profile must be used in order to make |
| 2738 |
the seeding device deletion work |
| 2739 |
.IP \(bu 2 |
| 2740 |
block group profiles \fIsingle\fP and \fIdup\fP support the use cases above |
| 2741 |
.IP \(bu 2 |
| 2742 |
the label is copied from the seeding device and can be changed by \fBbtrfs filesystem label\fP |
| 2743 |
.IP \(bu 2 |
| 2744 |
each new mount of the seeding device gets a new random UUID |
| 2745 |
.IP \(bu 2 |
| 2746 |
\fBumount /path; mount /dev/writable /path\fP can be replaced with |
| 2747 |
\fBmount \-o remount,rw /path\fP |
| 2748 |
but it won\(aqt reclaim space of deleted subvolumes until the seeding device |
| 2749 |
is mounted read\-write again before making it seeding again |
| 2750 |
.UNINDENT |
| 2751 |
.SS Chained seeding devices |
| 2752 |
.sp |
| 2753 |
Though it\(aqs not recommended and is rather an obscure and untested use case, |
| 2754 |
chaining seeding devices is possible. In the first example, the writable device |
| 2755 |
\fB/dev/sdb\fP can be turned onto another seeding device again, depending on the |
| 2756 |
unchanged seeding device \fB/dev/sda\fP\&. Then using \fB/dev/sdb\fP as the primary |
| 2757 |
seeding device it can be extended with another writable device, say \fB/dev/sdd\fP, |
| 2758 |
and it continues as before as a simple tree structure on devices. |
| 2759 |
.INDENT 0.0 |
| 2760 |
.INDENT 3.5 |
| 2761 |
.sp |
| 2762 |
.EX |
| 2763 |
# mkfs.btrfs /dev/sda |
| 2764 |
# mount /dev/sda /mnt/mnt1 |
| 2765 |
\&... fill mnt1 with data |
| 2766 |
# umount /mnt/mnt1 |
| 2767 |
|
| 2768 |
# btrfstune \-S 1 /dev/sda |
| 2769 |
|
| 2770 |
# mount /dev/sda /mnt/mnt1 |
| 2771 |
# btrfs device add /dev/sdb /mnt/mnt1 |
| 2772 |
# mount \-o remount,rw /mnt/mnt1 |
| 2773 |
\&... /mnt/mnt1 is now writable |
| 2774 |
# umount /mnt/mnt1 |
| 2775 |
|
| 2776 |
# btrfstune \-S 1 /dev/sdb |
| 2777 |
|
| 2778 |
# mount /dev/sdb /mnt/mnt1 |
| 2779 |
# btrfs device add /dev/sdc /mnt |
| 2780 |
# mount \-o remount,rw /mnt/mnt1 |
| 2781 |
\&... /mnt/mnt1 is now writable |
| 2782 |
# umount /mnt/mnt1 |
| 2783 |
.EE |
| 2784 |
.UNINDENT |
| 2785 |
.UNINDENT |
| 2786 |
.sp |
| 2787 |
As a result we have: |
| 2788 |
.INDENT 0.0 |
| 2789 |
.IP \(bu 2 |
| 2790 |
\fIsda\fP is a single seeding device, with its initial contents |
| 2791 |
.IP \(bu 2 |
| 2792 |
\fIsdb\fP is a seeding device but requires \fIsda\fP, the contents are from the time |
| 2793 |
when \fIsdb\fP is made seeding, i.e. contents of \fIsda\fP with any later changes |
| 2794 |
.IP \(bu 2 |
| 2795 |
\fIsdc\fP last writable, can be made a seeding one the same way as was \fIsdb\fP, |
| 2796 |
preserving its contents and depending on \fIsda\fP and \fIsdb\fP |
| 2797 |
.UNINDENT |
| 2798 |
.sp |
| 2799 |
As long as the seeding devices are unmodified and available, they can be used |
| 2800 |
to start another branch. |
| 2801 |
.SH RAID56 STATUS AND RECOMMENDED PRACTICES |
| 2802 |
.sp |
| 2803 |
The RAID56 feature provides striping and parity over several devices, same as |
| 2804 |
the traditional RAID5/6. There are some implementation and design deficiencies |
| 2805 |
that make it unreliable for some corner cases and the feature \fBshould not be |
| 2806 |
used in production, only for evaluation or testing\fP\&. The power failure safety |
| 2807 |
for metadata with RAID56 is not 100%. |
| 2808 |
.SS Metadata |
| 2809 |
.sp |
| 2810 |
Do not use \fIraid5\fP nor \fIraid6\fP for metadata. Use \fIraid1\fP or \fIraid1c3\fP |
| 2811 |
respectively. |
| 2812 |
.sp |
| 2813 |
The substitute profiles provide the same guarantees against loss of 1 or 2 |
| 2814 |
devices, and in some respect can be an improvement. Recovering from one |
| 2815 |
missing device will only need to access the remaining 1st or 2nd copy, that in |
| 2816 |
general may be stored on some other devices due to the way RAID1 works on |
| 2817 |
btrfs, unlike on a striped profile (similar to \fIraid0\fP) that would need all |
| 2818 |
devices all the time. |
| 2819 |
.sp |
| 2820 |
The space allocation pattern and consumption is different (e.g. on N devices): |
| 2821 |
for \fIraid5\fP as an example, a 1GiB chunk is reserved on each device, while with |
| 2822 |
\fIraid1\fP there\(aqs each 1GiB chunk stored on 2 devices. The consumption of each |
| 2823 |
1GiB of used metadata is then \fIN * 1GiB\fP for vs \fI2 * 1GiB\fP\&. Using \fIraid1\fP |
| 2824 |
is also more convenient for balancing/converting to other profile due to lower |
| 2825 |
requirement on the available chunk space. |
| 2826 |
.SS Missing/incomplete support |
| 2827 |
.sp |
| 2828 |
When RAID56 is on the same filesystem with different raid profiles, the space |
| 2829 |
reporting is inaccurate, e.g. \fBdf\fP, \fBbtrfs filesystem df\fP or |
| 2830 |
\fBbtrfs filesystem usage\fP\&. When there\(aqs only a one profile per block |
| 2831 |
group type (e.g. RAID5 for data) the reporting is accurate. |
| 2832 |
.sp |
| 2833 |
When scrub is started on a RAID56 filesystem, it\(aqs started on all devices that |
| 2834 |
degrade the performance. The workaround is to start it on each device |
| 2835 |
separately. Due to that the device stats may not match the actual state and |
| 2836 |
some errors might get reported multiple times. |
| 2837 |
.sp |
| 2838 |
The \fIwrite hole\fP problem. An unclean shutdown could leave a partially written |
| 2839 |
stripe in a state where the some stripe ranges and the parity are from the old |
| 2840 |
writes and some are new. The information which is which is not tracked. Write |
| 2841 |
journal is not implemented. Alternatively a full read\-modify\-write would make |
| 2842 |
sure that a full stripe is always written, avoiding the write hole completely, |
| 2843 |
but performance in that case turned out to be too bad for use. |
| 2844 |
.sp |
| 2845 |
The striping happens on all available devices (at the time the chunks were |
| 2846 |
allocated), so in case a new device is added it may not be utilized |
| 2847 |
immediately and would require a rebalance. A fixed configured stripe width is |
| 2848 |
not implemented. |
| 2849 |
.SH GLOSSARY |
| 2850 |
.sp |
| 2851 |
Terms in \fIitalics\fP also appear in this glossary. |
| 2852 |
.INDENT 0.0 |
| 2853 |
.TP |
| 2854 |
.B allocator |
| 2855 |
Usually \fIallocator\fP means the \fIblock\fP allocator, i.e. the logic |
| 2856 |
inside the filesystem which decides where to place newly allocated blocks |
| 2857 |
in order to maintain several constraints (like data locality, low |
| 2858 |
fragmentation). |
| 2859 |
.sp |
| 2860 |
In btrfs, allocator may also refer to \fIchunk\fP allocator, i.e. the |
| 2861 |
logic behind placing chunks on devices. |
| 2862 |
.TP |
| 2863 |
.B balance |
| 2864 |
An operation that can be done to a btrfs filesystem, for example |
| 2865 |
through \fBbtrfs balance /path\fP\&. A |
| 2866 |
balance passes all data in the filesystem through the \fIallocator\fP |
| 2867 |
again. It is primarily intended to rebalance the data in the filesystem |
| 2868 |
across the \fIdevices\fP when a device is added or removed. A balance |
| 2869 |
will regenerate missing copies for the redundant \fIRAID\fP levels, if a |
| 2870 |
device has failed. As of Linux kernel 3.3, a balance operation can be |
| 2871 |
made selective about which parts of the filesystem are rewritten |
| 2872 |
using filters \%<#\:man-balance-filters>\&. |
| 2873 |
.TP |
| 2874 |
.B barrier |
| 2875 |
An instruction to the underlying hardware to ensure that everything before |
| 2876 |
the barrier is physically written to permanent storage before anything |
| 2877 |
after it. Used in btrfs\(aqs \fIcopy on write\fP approach to ensure |
| 2878 |
filesystem consistency. |
| 2879 |
.TP |
| 2880 |
.B block |
| 2881 |
A single physically and logically contiguous piece of storage on a |
| 2882 |
device, of size e.g. 4K. In some contexts also referred to as \fIsector\fP, |
| 2883 |
though the term \fIblock\fP is preferred. |
| 2884 |
.TP |
| 2885 |
.B block group |
| 2886 |
The unit of allocation of space in btrfs. A block group is laid out on |
| 2887 |
the disk by the btrfs \fIallocator\fP, and will consist of one or more |
| 2888 |
\fIchunks\fP, each stored on a different \fIdevice\fP\&. The number of chunks |
| 2889 |
used in a block group will depend on its \fIRAID\fP level. |
| 2890 |
.TP |
| 2891 |
.B B\-tree |
| 2892 |
The fundamental storage data structure used in btrfs. Except for the |
| 2893 |
\fIsuperblocks\fP, all of btrfs \fImetadata\fP is stored in one of several |
| 2894 |
B\-trees on disk. B\-trees store key/item pairs. While the same code is |
| 2895 |
used to implement all of the B\-trees, there are a few different |
| 2896 |
categories of B\-tree. The name \fIbtrfs\fP refers to its use of B\-trees. |
| 2897 |
.TP |
| 2898 |
.B btrfsck, fsck, btrfs\-check |
| 2899 |
Tool in \fIbtrfs\-progs\fP that checks an unmounted filesystem (\fIoffline\fP) |
| 2900 |
and reports on any errors in the filesystem structures it finds. By |
| 2901 |
default the tool runs in read\-only mode as fixing errors is potentially |
| 2902 |
dangerous. See also \fIscrub\fP\&. |
| 2903 |
.TP |
| 2904 |
.B btrfs\-progs |
| 2905 |
User mode tools to manage btrfs\-specific features. Maintained at |
| 2906 |
\%<http://\:github\:.com/\:kdave/\:btrfs-progs\:.git> . The main frontend to btrfs |
| 2907 |
features is the standalone tool \fIbtrfs\fP, although |
| 2908 |
other tools such as \fImkfs.btrfs\fP and \fIbtrfstune\fP are also part of |
| 2909 |
btrfs\-progs. |
| 2910 |
.TP |
| 2911 |
.B chunk |
| 2912 |
A part of a \fIblock group\fP\&. Chunks are either 1 GiB in size (for data) |
| 2913 |
or 256 MiB (for \fImetadata\fP), depending on the overall filesystem size. |
| 2914 |
.TP |
| 2915 |
.B chunk tree |
| 2916 |
A layer that keeps information about mapping between physical and |
| 2917 |
logical block addresses. It\(aqs stored within the \fIsystem\fP group. |
| 2918 |
.TP |
| 2919 |
.B cleaner |
| 2920 |
Usually referred to in context of deleted subvolumes. It\(aqs a background |
| 2921 |
process that removes the actual data once a subvolume has been deleted. |
| 2922 |
Cleaning can involve lots of IO and CPU activity depending on the |
| 2923 |
fragmentation and amount of shared data with other subvolumes. |
| 2924 |
.sp |
| 2925 |
The cleaner kernel thread also processes defragmentation triggered by the |
| 2926 |
\fIautodefrag\fP mount option, removing of empty blocks groups and some |
| 2927 |
other finalization tasks. |
| 2928 |
.TP |
| 2929 |
.B copy\-on\-write, COW |
| 2930 |
Also known as \fICOW\fP\&. The method that btrfs uses for modifying data. |
| 2931 |
Instead of directly overwriting data in place, btrfs takes a copy of |
| 2932 |
the data, alters it, and then writes the modified data back to a |
| 2933 |
different (unused) location on the disk. It then updates the \fImetadata\fP |
| 2934 |
to reflect the new location of the data. In order to update the |
| 2935 |
metadata, the affected metadata blocks are also treated in the same |
| 2936 |
way. In COW filesystems, files tend to fragment as they are modified. |
| 2937 |
Copy\-on\-write is also used in the implementation of \fIsnapshots\fP and |
| 2938 |
\fIreflink copies\fP\&. A copy\-on\-write filesystem is, in theory, |
| 2939 |
\fIalways\fP consistent, provided the underlying hardware supports |
| 2940 |
\fIbarriers\fP\&. |
| 2941 |
.TP |
| 2942 |
.B default subvolume |
| 2943 |
The \fIsubvolume\fP in a btrfs filesystem which is mounted when mounting |
| 2944 |
the filesystem without using the \fBsubvol=\fP mount option. |
| 2945 |
.TP |
| 2946 |
.B device |
| 2947 |
A Linux block device, e.g. a whole disk, partition, LVM logical volume, |
| 2948 |
loopback device, or network block device. A btrfs filesystem can reside |
| 2949 |
on one or more devices. |
| 2950 |
.TP |
| 2951 |
.B df |
| 2952 |
A standard Unix tool for reporting the amount of space used and free in |
| 2953 |
a filesystem. The standard tool does not give accurate results, but the |
| 2954 |
\fIbtrfs\fP command from \fIbtrfs\-progs\fP has |
| 2955 |
an implementation of \fIdf\fP which shows space available in more detail. See |
| 2956 |
the |
| 2957 |
[[FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F|FAQ]] |
| 2958 |
for a more detailed explanation of btrfs free space accounting. |
| 2959 |
.TP |
| 2960 |
.B DUP |
| 2961 |
A form of \(dq\fIRAID\fP\(dq which stores two copies of each piece of data on |
| 2962 |
the same \fIdevice\fP\&. This is similar to \fIRAID1\fP, and protects |
| 2963 |
against \fIblock\fP\-level errors on the device, but does not provide any |
| 2964 |
guarantees if the entire device fails. By default, btrfs uses \fIDUP\fP |
| 2965 |
profile for metadata on single device filesystem.s |
| 2966 |
.TP |
| 2967 |
.B ENOSPC |
| 2968 |
Error code returned by the OS to a user program when the filesystem |
| 2969 |
cannot allocate enough data to fulfill the user request. In most |
| 2970 |
filesystems, it indicates there is no free space available in the |
| 2971 |
filesystem. Due to the additional space requirements from btrfs\(aqs |
| 2972 |
\fICOW\fP behaviour, btrfs can sometimes return ENOSPC when there is |
| 2973 |
apparently (in terms of \fIdf\fP) a large amount of space free. This is |
| 2974 |
effectively a bug in btrfs, and (if it is repeatable), using the mount |
| 2975 |
option \fBenospc_debug\fP may give a report |
| 2976 |
that will help the btrfs developers. See the |
| 2977 |
[[FAQ#if_your_device_is_large_.28.3E16GiB.29|FAQ entry]] on free space. |
| 2978 |
.TP |
| 2979 |
.B extent |
| 2980 |
Contiguous sequence of bytes on disk that holds file data. It\(aqs a compact |
| 2981 |
representation that tracks the start and length of the byte range, so the |
| 2982 |
logic behind allocating blocks (\fIdelayed allocation\fP) strives for |
| 2983 |
maximizing the length before writing the extents to the devices. |
| 2984 |
.TP |
| 2985 |
.B extent buffer |
| 2986 |
An abstraction of a \fIb\-tree\fP metadata block storing item keys and item |
| 2987 |
data. The underlying related structures are physical device block and a |
| 2988 |
CPU memory page. |
| 2989 |
.TP |
| 2990 |
.B fallocate |
| 2991 |
Command line tool in util\-linux, and a syscall, that reserves space in |
| 2992 |
the filesystem for a file, without actually writing any file data to |
| 2993 |
the filesystem. First data write will turn the preallocated extents |
| 2994 |
into regular ones. See fallocate(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:fallocate\:.1\:.html> and fallocate(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fallocate\:.2\:.html> manual pages |
| 2995 |
for more details. |
| 2996 |
.TP |
| 2997 |
.B filefrag |
| 2998 |
A tool to show the number of extents in a file, and hence the amount of |
| 2999 |
fragmentation in the file. It is usually part of the e2fsprogs package |
| 3000 |
on most Linux distributions. While initially developed for the ext2 |
| 3001 |
filesystem, it works on Btrfs as well. It uses the \fIFIEMAP\fP ioctl. |
| 3002 |
.TP |
| 3003 |
.B free space cache |
| 3004 |
Also known as \(dqspace cache v1\(dq. A separate cache tracking free space as |
| 3005 |
btrfs only tracks the allocated space. The free space is by definition |
| 3006 |
any hole between allocated ranges. Finding the free ranges can be |
| 3007 |
I/O intensive so the cache stores a condensed representation of it. |
| 3008 |
It is updated every \fItransaction\fP commit. |
| 3009 |
.sp |
| 3010 |
The v1 free space cache has been superseded by free space tree. |
| 3011 |
.TP |
| 3012 |
.B free space tree |
| 3013 |
Successor of \fIfree space cache\fP, also known as \(dqspace cache v2\(dq and now |
| 3014 |
default. The free space is tracked in a better way and using COW |
| 3015 |
unlike a custom mechanism of v1. |
| 3016 |
.TP |
| 3017 |
.B fsync |
| 3018 |
On Unix and Unix\-like operating systems (of which Linux is the latter), |
| 3019 |
the fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> system call causes all buffered file |
| 3020 |
descriptor related data changes to be flushed to the underlying block |
| 3021 |
device. When a file is modified on a modern operating system the |
| 3022 |
changes are generally not written to the disk immediately but rather |
| 3023 |
those changes are buffered in memory for performance reasons, |
| 3024 |
calling fsync(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:fsync\:.2\:.html> causes any in\-memory changes to be written |
| 3025 |
to disk. |
| 3026 |
.TP |
| 3027 |
.B generation |
| 3028 |
An internal counter which updates for each \fItransaction\fP\&. When a |
| 3029 |
\fImetadata\fP block is written (using \fIcopy on write\fP), current |
| 3030 |
generation is stored in the block, so that blocks which are too new |
| 3031 |
(and hence possibly inconsistent) can be identified. |
| 3032 |
.TP |
| 3033 |
.B key |
| 3034 |
A fixed sized tuple used to identify and sort items in a \fIB\-tree\fP\&. |
| 3035 |
The key is broken up into 3 parts: \fIobjectid\fP, \fItype\fP, and |
| 3036 |
\fIoffset\fP\&. The \fItype\fP field indicates how each of the other two |
| 3037 |
fields should be used, and what to expect to find in the item. |
| 3038 |
.TP |
| 3039 |
.B item |
| 3040 |
A variable sized structure stored in B\-tree leaves. Items hold |
| 3041 |
different types of data depending on key type. |
| 3042 |
.TP |
| 3043 |
.B log tree |
| 3044 |
A b\-tree that temporarily tracks ongoing metadata updates until a full |
| 3045 |
transaction commit is done. It\(aqs a performance optimization of |
| 3046 |
\fIfsync\fP\&. The log tracked in the tree are replayed if the filesystem |
| 3047 |
is not unmounted cleanly. |
| 3048 |
.TP |
| 3049 |
.B metadata |
| 3050 |
Data about data. In btrfs, this includes all of the internal data |
| 3051 |
structures of the filesystem, including directory structures, |
| 3052 |
filenames, file permissions, checksums, and the location of each file\(aqs |
| 3053 |
\fIextents\fP\&. All btrfs metadata is stored in \fIB\-trees\fP\&. |
| 3054 |
.TP |
| 3055 |
.B mkfs.btrfs |
| 3056 |
The tool (from \fIbtrfs\-progs\fP) to create a btrfs filesystem. |
| 3057 |
.TP |
| 3058 |
.B offline |
| 3059 |
A filesystem which is not mounted is offline. Some tools (e.g. |
| 3060 |
\fIbtrfsck\fP) will only work on offline filesystems. Compare \fIonline\fP\&. |
| 3061 |
.TP |
| 3062 |
.B online |
| 3063 |
A filesystem which is mounted is online. Most btrfs tools will only |
| 3064 |
work on online filesystems. Compare \fIoffline\fP\&. |
| 3065 |
.TP |
| 3066 |
.B orphan |
| 3067 |
A file that\(aqs still in use (opened by a running process) but all |
| 3068 |
directory entries of that file have been removed. |
| 3069 |
.TP |
| 3070 |
.B RAID |
| 3071 |
A class of different methods for writing some additional redundant data |
| 3072 |
across multiple \fIdevices\fP so that if one device fails, the missing |
| 3073 |
data can be reconstructed from the remaining ones. See \fIRAID0\fP, |
| 3074 |
\fIRAID1\fP, \fIRAID5\fP, \fIRAID6\fP, \fIRAID10\fP, \fIDUP\fP and |
| 3075 |
\fIsingle\fP\&. Traditional RAID methods operate across multiple devices of |
| 3076 |
equal size, whereas btrfs\(aq RAID implementation works inside \fIblock |
| 3077 |
groups\fP\&. |
| 3078 |
.TP |
| 3079 |
.B RAID0 |
| 3080 |
A form of \fIRAID\fP which provides no guarantees of error recovery, but |
| 3081 |
stripes a single copy of data across multiple devices for performance |
| 3082 |
purposes. The stripe size is fixed to 64KB for now. |
| 3083 |
.TP |
| 3084 |
.B RAID1, RAID1C3, RAID1C4 |
| 3085 |
A form of \fIRAID\fP which stores two/three/four complete copies of each |
| 3086 |
piece of data. Each copy is stored on a different \fIdevice\fP\&. btrfs |
| 3087 |
requires a minimum of two devices to use RAID\-1 or three/four respectively. |
| 3088 |
This is the default block group profile for btrfs\(aqs \fImetadata\fP on more |
| 3089 |
than one device. |
| 3090 |
.TP |
| 3091 |
.B RAID5 |
| 3092 |
A form of \fIRAID\fP which stripes a single copy of data across multiple |
| 3093 |
\fIdevices\fP, including one device\(aqs worth of additional parity data. |
| 3094 |
Can be used to recover from a single device failure. |
| 3095 |
.TP |
| 3096 |
.B RAID6 |
| 3097 |
A form of \fIRAID\fP which stripes a single copy of data across multiple |
| 3098 |
\fIdevices\fP, including two device\(aqs worth of additional parity data. Can |
| 3099 |
be used to recover from the failure of two devices. |
| 3100 |
.TP |
| 3101 |
.B RAID10 |
| 3102 |
A form of \fIRAID\fP which stores two complete copies of each piece of |
| 3103 |
data, and also stripes each copy across multiple devices for |
| 3104 |
performance. |
| 3105 |
.TP |
| 3106 |
.B reflink |
| 3107 |
Commonly used as a reference to a shallow copy of file extents that share |
| 3108 |
the extents until the first change. Reflinked files (e.g. by the \fBcp\fP) |
| 3109 |
are different files but point to the same extents, any change will be |
| 3110 |
detected and new copy of the data created, keeping the files independent. |
| 3111 |
Related to that is extent range cloning, that works on a range of a file. |
| 3112 |
.TP |
| 3113 |
.B relocation |
| 3114 |
The process of moving block groups within the filesystem while |
| 3115 |
maintaining full filesystem integrity and consistency. This |
| 3116 |
functionality is underlying \fIbalance\fP and \fIdevice\fP removing features. |
| 3117 |
.TP |
| 3118 |
.B scrub \%<> |
| 3119 |
An \fIonline\fP filesystem checking tool. Reads all the data and metadata |
| 3120 |
on the filesystem, verifies \fIchecksums\fP and eventually uses redundant |
| 3121 |
copies from \fIRAID\fP or \fIDUP\fP repair any corrupt data/metadata. |
| 3122 |
.TP |
| 3123 |
.B seed device \%<> |
| 3124 |
A readonly device can be used as a filesystem seed or template (e.g. a |
| 3125 |
CD\-ROM containing an OS image). Read/write devices can be added to |
| 3126 |
store modifications (using \fIcopy on write\fP), changes to the writable |
| 3127 |
devices are persistent across reboots. The original device remains |
| 3128 |
unchanged and can be removed at any time (after Btrfs has been |
| 3129 |
instructed to copy over all missing blocks). Multiple read/write file |
| 3130 |
systems can be built from the same seed. |
| 3131 |
.TP |
| 3132 |
.B single |
| 3133 |
A block group profile storing a single copy of each piece of data. |
| 3134 |
.TP |
| 3135 |
.B snapshot \%<> |
| 3136 |
A \fIsubvolume\fP which is a \fIcopy on write\fP copy of another subvolume. The |
| 3137 |
two subvolumes share all of their common (unmodified) data, which means |
| 3138 |
that snapshots can be used to keep the historical state of a filesystem |
| 3139 |
very cheaply. After the snapshot is made, the original subvolume and |
| 3140 |
the snapshot are of equal status: the original does not \(dqown\(dq the |
| 3141 |
snapshot, and either one can be deleted without affecting the other |
| 3142 |
one. |
| 3143 |
.TP |
| 3144 |
.B subvolume \%<> |
| 3145 |
A tree of files and directories inside a btrfs that can be mounted as |
| 3146 |
if it were an independent filesystem. A subvolume is created by taking |
| 3147 |
a reference on the root of another subvolume. Each btrfs filesystem has |
| 3148 |
at least one subvolume, the \fItop\-level subvolume\fP, which contains |
| 3149 |
everything else in the filesystem. Additional subvolumes can be created |
| 3150 |
and deleted with the \fIbtrfs<\fP tool. All subvolumes share the same pool |
| 3151 |
of free space in the filesystem. See also \fIdefault subvolume\fP\&. |
| 3152 |
.TP |
| 3153 |
.B super block |
| 3154 |
A special metadata block that is a main access point of the filesystem |
| 3155 |
structures. It\(aqs size is fixed and there are fixed locations on the devices |
| 3156 |
used for detecting and opening the filesystem. Updating the superblock |
| 3157 |
defines one \fItransaction\fP\&. The super blocks contains filesystem |
| 3158 |
identification (UUID), checksum type, block pointers to fundamental |
| 3159 |
trees, features and creation parameters. |
| 3160 |
.TP |
| 3161 |
.B system array |
| 3162 |
A technical term for \fIsuper block\fP metadata describing how to assemble a |
| 3163 |
filesystem from multiple device, storing information about chunks and devices that are |
| 3164 |
required to be scanned/registered at the time the mount happens. |
| 3165 |
Scanning is done by command \fBbtrfs device scan\fP, alternatively |
| 3166 |
all the required devices can be specified by a mount option \fIdevice=/path\fP\&. |
| 3167 |
.TP |
| 3168 |
.B top\-level subvolume |
| 3169 |
The \fIsubvolume\fP at the very top of the filesystem. This is the only |
| 3170 |
subvolume present in a newly\-created btrfs filesystem, and internally has ID 5, |
| 3171 |
otherwise could be referenced as 0 (e.g. within the \fIset\-default\fP subcommand of |
| 3172 |
\fIbtrfs\fP). |
| 3173 |
.TP |
| 3174 |
.B transaction |
| 3175 |
A consistent set of changes. To avoid generating very large amounts of |
| 3176 |
disk activity, btrfs caches changes in RAM for up to 30 seconds |
| 3177 |
(sometimes more often if the filesystem is running short on space or |
| 3178 |
doing a lot of \fIfsync*s), and then writes (commits) these changes out |
| 3179 |
to disk in one go (using *copy on write\fP behaviour). This period of |
| 3180 |
caching is called a transaction. Only one transaction is active on the |
| 3181 |
filesystem at any one time. |
| 3182 |
.TP |
| 3183 |
.B transid |
| 3184 |
An alternative term for \fIgeneration\fP\&. |
| 3185 |
.TP |
| 3186 |
.B writeback |
| 3187 |
\fIWriteback\fP in the context of the Linux kernel can be defined as the |
| 3188 |
process of writing \(dqdirty\(dq memory from the page cache to the disk, |
| 3189 |
when certain conditions are met (timeout, number of dirty pages over a |
| 3190 |
ratio). |
| 3191 |
.UNINDENT |
| 3192 |
.SH STORAGE MODEL, HARDWARE CONSIDERATIONS |
| 3193 |
.SS Storage model |
| 3194 |
.sp |
| 3195 |
\fIA storage model is a model that captures key physical aspects of data |
| 3196 |
structure in a data store. A filesystem is the logical structure organizing |
| 3197 |
data on top of the storage device.\fP |
| 3198 |
.sp |
| 3199 |
The filesystem assumes several features or limitations of the storage device |
| 3200 |
and utilizes them or applies measures to guarantee reliability. BTRFS in |
| 3201 |
particular is based on a COW (copy on write) mode of writing, i.e. not updating |
| 3202 |
data in place but rather writing a new copy to a different location and then |
| 3203 |
atomically switching the pointers. |
| 3204 |
.sp |
| 3205 |
In an ideal world, the device does what it promises. The filesystem assumes |
| 3206 |
that this may not be true so additional mechanisms are applied to either detect |
| 3207 |
misbehaving hardware or get valid data by other means. The devices may (and do) |
| 3208 |
apply their own detection and repair mechanisms but we won\(aqt assume any. |
| 3209 |
.sp |
| 3210 |
The following assumptions about storage devices are considered (sorted by |
| 3211 |
importance, numbers are for further reference): |
| 3212 |
.INDENT 0.0 |
| 3213 |
.IP 1. 3 |
| 3214 |
atomicity of reads and writes of blocks/sectors (the smallest unit of data |
| 3215 |
the device presents to the upper layers) |
| 3216 |
.IP 2. 3 |
| 3217 |
there\(aqs a flush command that instructs the device to forcibly order writes |
| 3218 |
before and after the command; alternatively there\(aqs a barrier command that |
| 3219 |
facilitates the ordering but may not flush the data |
| 3220 |
.IP 3. 3 |
| 3221 |
data sent to write to a given device offset will be written without further |
| 3222 |
changes to the data and to the offset |
| 3223 |
.IP 4. 3 |
| 3224 |
writes can be reordered by the device, unless explicitly serialized by the |
| 3225 |
flush command |
| 3226 |
.IP 5. 3 |
| 3227 |
reads and writes can be freely reordered and interleaved |
| 3228 |
.UNINDENT |
| 3229 |
.sp |
| 3230 |
The consistency model of BTRFS builds on these assumptions. The logical data |
| 3231 |
updates are grouped, into a generation, written on the device, serialized by |
| 3232 |
the flush command and then the super block is written ending the generation. |
| 3233 |
All logical links among metadata comprising a consistent view of the data may |
| 3234 |
not cross the generation boundary. |
| 3235 |
.SS When things go wrong |
| 3236 |
.sp |
| 3237 |
\fBNo or partial atomicity of block reads/writes (1)\fP |
| 3238 |
.INDENT 0.0 |
| 3239 |
.IP \(bu 2 |
| 3240 |
\fIProblem\fP: a partial block contents is written (\fItorn write\fP), e.g. due to a |
| 3241 |
power glitch or other electronics failure during the read/write |
| 3242 |
.IP \(bu 2 |
| 3243 |
\fIDetection\fP: checksum mismatch on read |
| 3244 |
.IP \(bu 2 |
| 3245 |
\fIRepair\fP: use another copy or rebuild from multiple blocks using some encoding |
| 3246 |
scheme |
| 3247 |
.UNINDENT |
| 3248 |
.sp |
| 3249 |
\fBThe flush command does not flush (2)\fP |
| 3250 |
.sp |
| 3251 |
This is perhaps the most serious problem and impossible to mitigate by |
| 3252 |
filesystem without limitations and design restrictions. What could happen in |
| 3253 |
the worst case is that writes from one generation bleed to another one, while |
| 3254 |
still letting the filesystem consider the generations isolated. Crash at any |
| 3255 |
point would leave data on the device in an inconsistent state without any hint |
| 3256 |
what exactly got written, what is missing and leading to stale metadata link |
| 3257 |
information. |
| 3258 |
.sp |
| 3259 |
Devices usually honor the flush command, but for performance reasons may do |
| 3260 |
internal caching, where the flushed data are not yet persistently stored. A |
| 3261 |
power failure could lead to a similar scenario as above, although it\(aqs less |
| 3262 |
likely that later writes would be written before the cached ones. This is |
| 3263 |
beyond what a filesystem can take into account. Devices or controllers are |
| 3264 |
usually equipped with batteries or capacitors to write the cache contents even |
| 3265 |
after power is cut. (\fIBattery backed write cache\fP) |
| 3266 |
.sp |
| 3267 |
\fBData get silently changed on write (3)\fP |
| 3268 |
.sp |
| 3269 |
Such thing should not happen frequently, but still can happen spuriously due |
| 3270 |
the complex internal workings of devices or physical effects of the storage |
| 3271 |
media itself. |
| 3272 |
.INDENT 0.0 |
| 3273 |
.IP \(bu 2 |
| 3274 |
\fIProblem\fP: while the data are written atomically, the contents get changed |
| 3275 |
.IP \(bu 2 |
| 3276 |
\fIDetection\fP: checksum mismatch on read |
| 3277 |
.IP \(bu 2 |
| 3278 |
\fIRepair\fP: use another copy or rebuild from multiple blocks using some |
| 3279 |
encoding scheme |
| 3280 |
.UNINDENT |
| 3281 |
.sp |
| 3282 |
\fBData get silently written to another offset (3)\fP |
| 3283 |
.sp |
| 3284 |
This would be another serious problem as the filesystem has no information |
| 3285 |
when it happens. For that reason the measures have to be done ahead of time. |
| 3286 |
This problem is also commonly called \fIghost write\fP\&. |
| 3287 |
.sp |
| 3288 |
The metadata blocks have the checksum embedded in the blocks, so a correct |
| 3289 |
atomic write would not corrupt the checksum. It\(aqs likely that after reading |
| 3290 |
such block the data inside would not be consistent with the rest. To rule that |
| 3291 |
out there\(aqs embedded block number in the metadata block. It\(aqs the logical |
| 3292 |
block number because this is what the logical structure expects and verifies. |
| 3293 |
.sp |
| 3294 |
The following is based on information publicly available, user feedback, |
| 3295 |
community discussions or bug report analyses. It\(aqs not complete and further |
| 3296 |
research is encouraged when in doubt. |
| 3297 |
.SS Main memory |
| 3298 |
.sp |
| 3299 |
The data structures and raw data blocks are temporarily stored in computer |
| 3300 |
memory before they get written to the device. It is critical that memory is |
| 3301 |
reliable because even simple bit flips can have vast consequences and lead to |
| 3302 |
damaged structures, not only in the filesystem but in the whole operating |
| 3303 |
system. |
| 3304 |
.sp |
| 3305 |
Based on experience in the community, memory bit flips are more common than one |
| 3306 |
would think. When it happens, it\(aqs reported by the tree\-checker or by a checksum |
| 3307 |
mismatch after reading blocks. There are some very obvious instances of bit |
| 3308 |
flips that happen, e.g. in an ordered sequence of keys in metadata blocks. We can |
| 3309 |
easily infer from the other data what values get damaged and how. However, fixing |
| 3310 |
that is not straightforward and would require cross\-referencing data from the |
| 3311 |
entire filesystem to see the scope. |
| 3312 |
.sp |
| 3313 |
If available, ECC memory should lower the chances of bit flips, but this |
| 3314 |
type of memory is not available in all cases. A memory test should be performed |
| 3315 |
in case there\(aqs a visible bit flip pattern, though this may not detect a faulty |
| 3316 |
memory module because the actual load of the system could be the factor making |
| 3317 |
the problems appear. In recent years attacks on how the memory modules operate |
| 3318 |
have been demonstrated (\fIrowhammer\fP) achieving specific bits to be flipped. |
| 3319 |
While these were targeted, this shows that a series of reads or writes can |
| 3320 |
affect unrelated parts of memory. |
| 3321 |
.sp |
| 3322 |
Block group profiles with redundancy (like RAID1) will not protect against |
| 3323 |
memory errors as the blocks are first stored in memory before they are written |
| 3324 |
to the devices from the same source. |
| 3325 |
.sp |
| 3326 |
A filesystem mounted read\-only will not affect the underlying block device in |
| 3327 |
almost 100% (with highly unlikely exceptions). The exception is a tree\-log that |
| 3328 |
needs to be replayed during mount (and before the read\-only mount takes place), |
| 3329 |
working memory is needed for that and that can be affected by bit flips. |
| 3330 |
There\(aqs a theoretical case where bit flip changes the filesystem status from |
| 3331 |
read\-only to read\-write. |
| 3332 |
.sp |
| 3333 |
Further reading: |
| 3334 |
.INDENT 0.0 |
| 3335 |
.IP \(bu 2 |
| 3336 |
\%<https://\:en\:.wikipedia\:.org/\:wiki/\:Row_hammer> |
| 3337 |
.IP \(bu 2 |
| 3338 |
memory overclocking, XMP, potential risks |
| 3339 |
.UNINDENT |
| 3340 |
.sp |
| 3341 |
What to do: |
| 3342 |
.INDENT 0.0 |
| 3343 |
.IP \(bu 2 |
| 3344 |
run \fImemtest\fP, note that sometimes memory errors happen only when the system |
| 3345 |
is under heavy load that the default memtest cannot trigger |
| 3346 |
.IP \(bu 2 |
| 3347 |
memory errors may appear as filesystem going read\-only due to \(dqpre write\(dq |
| 3348 |
check, that verify meta data before they get written but fail some basic |
| 3349 |
consistency checks |
| 3350 |
.IP \(bu 2 |
| 3351 |
newly built systems should be tested before being put to production use, |
| 3352 |
ideally start a IO/CPU load that will be run on such system later; namely |
| 3353 |
systems that will utilize overclocking or special performance features |
| 3354 |
.UNINDENT |
| 3355 |
.SS Direct memory access (DMA) |
| 3356 |
.sp |
| 3357 |
Another class of errors is related to DMA (direct memory access) performed |
| 3358 |
by device drivers. While this could be considered a software error, the |
| 3359 |
data transfers that happen without CPU assistance may accidentally corrupt |
| 3360 |
other pages. Storage devices utilize DMA for performance reasons, the |
| 3361 |
filesystem structures and data pages are passed back and forth, making |
| 3362 |
errors possible in case page life time is not properly tracked. |
| 3363 |
.sp |
| 3364 |
There are lots of quirks (device\-specific workarounds) in Linux kernel |
| 3365 |
drivers (regarding not only DMA) that are added when found. The quirks |
| 3366 |
may avoid specific errors or disable some features to avoid worse problems. |
| 3367 |
.sp |
| 3368 |
What to do: |
| 3369 |
.INDENT 0.0 |
| 3370 |
.IP \(bu 2 |
| 3371 |
use up\-to\-date kernel (recent releases or maintained long term support versions) |
| 3372 |
.IP \(bu 2 |
| 3373 |
as this may be caused by faulty drivers, keep the systems up\-to\-date |
| 3374 |
.UNINDENT |
| 3375 |
.SS Rotational disks (HDD) |
| 3376 |
.sp |
| 3377 |
Rotational HDDs typically fail at the level of individual sectors or small clusters. |
| 3378 |
Read failures are caught on the levels below the filesystem and are returned to |
| 3379 |
the user as \fIEIO \- Input/output error\fP\&. Reading the blocks repeatedly may |
| 3380 |
return the data eventually, but this is better done by specialized tools and |
| 3381 |
filesystem takes the result of the lower layers. Rewriting the sectors may |
| 3382 |
trigger internal remapping but this inevitably leads to data loss. |
| 3383 |
.sp |
| 3384 |
Disk firmware is technically software but from the filesystem perspective is |
| 3385 |
part of the hardware. IO requests are processed, and caching or various |
| 3386 |
other optimizations are performed, which may lead to bugs under high load or |
| 3387 |
unexpected physical conditions or unsupported use cases. |
| 3388 |
.sp |
| 3389 |
Disks are connected by cables with two ends, both of which can cause problems |
| 3390 |
when not attached properly. Data transfers are protected by checksums and the |
| 3391 |
lower layers try hard to transfer the data correctly or not at all. The errors |
| 3392 |
from badly\-connecting cables may manifest as large amount of failed read or |
| 3393 |
write requests, or as short error bursts depending on physical conditions. |
| 3394 |
.sp |
| 3395 |
What to do: |
| 3396 |
.INDENT 0.0 |
| 3397 |
.IP \(bu 2 |
| 3398 |
check \fBsmartctl\fP for potential issues |
| 3399 |
.UNINDENT |
| 3400 |
.SS Solid state drives (SSD) |
| 3401 |
.sp |
| 3402 |
The mechanism of information storage is different from HDDs and this affects |
| 3403 |
the failure mode as well. The data are stored in cells grouped in large blocks |
| 3404 |
with limited number of resets and other write constraints. The firmware tries |
| 3405 |
to avoid unnecessary resets and performs optimizations to maximize the storage |
| 3406 |
media lifetime. The known techniques are deduplication (blocks with same |
| 3407 |
fingerprint/hash are mapped to same physical block), compression or internal |
| 3408 |
remapping and garbage collection of used memory cells. Due to the additional |
| 3409 |
processing there are measures to verify the data e.g. by ECC codes. |
| 3410 |
.sp |
| 3411 |
The observations of failing SSDs show that the whole electronic fails at once |
| 3412 |
or affects a lot of data (e.g. stored on one chip). Recovering such data |
| 3413 |
may need specialized equipment and reading data repeatedly does not help as |
| 3414 |
it\(aqs possible with HDDs. |
| 3415 |
.sp |
| 3416 |
There are several technologies of the memory cells with different |
| 3417 |
characteristics and price. The lifetime is directly affected by the type and |
| 3418 |
frequency of data written. Writing \(dqtoo much\(dq distinct data (e.g. encrypted) |
| 3419 |
may render the internal deduplication ineffective and lead to a lot of rewrites |
| 3420 |
and increased wear of the memory cells. |
| 3421 |
.sp |
| 3422 |
There are several technologies and manufacturers so it\(aqs hard to describe them |
| 3423 |
but there are some that exhibit similar behaviour: |
| 3424 |
.INDENT 0.0 |
| 3425 |
.IP \(bu 2 |
| 3426 |
expensive SSD will use more durable memory cells and is optimized for |
| 3427 |
reliability and high load |
| 3428 |
.IP \(bu 2 |
| 3429 |
cheap SSD is projected for a lower load (\(dqdesktop user\(dq) and is optimized for |
| 3430 |
cost, it may employ the optimizations and/or extended error reporting |
| 3431 |
partially or not at all |
| 3432 |
.UNINDENT |
| 3433 |
.sp |
| 3434 |
It\(aqs not possible to reliably determine the expected lifetime of an SSD due to |
| 3435 |
lack of information about how it works or due to lack of reliable stats provided |
| 3436 |
by the device. |
| 3437 |
.sp |
| 3438 |
Metadata writes tend to be the biggest component of lifetime writes to a SSD, |
| 3439 |
so there is some value in reducing them. Depending on the device class (high |
| 3440 |
end/low end) the features like DUP block group profiles may affect the |
| 3441 |
reliability in both ways: |
| 3442 |
.INDENT 0.0 |
| 3443 |
.IP \(bu 2 |
| 3444 |
\fIhigh end\fP are typically more reliable and using \fIsingle\fP for data and |
| 3445 |
metadata could be suitable to reduce device wear |
| 3446 |
.IP \(bu 2 |
| 3447 |
\fIlow end\fP could lack ability to identify errors so an additional redundancy |
| 3448 |
at the filesystem level (checksums, \fIDUP\fP) could help |
| 3449 |
.UNINDENT |
| 3450 |
.sp |
| 3451 |
Only users who consume 50 to 100% of the SSD\(aqs actual lifetime writes need to be |
| 3452 |
concerned by the write amplification of btrfs DUP metadata. Most users will be |
| 3453 |
far below 50% of the actual lifetime, or will write the drive to death and |
| 3454 |
discover how many writes 100% of the actual lifetime was. SSD firmware often |
| 3455 |
adds its own write multipliers that can be arbitrary and unpredictable and |
| 3456 |
dependent on application behavior, and these will typically have far greater |
| 3457 |
effect on SSD lifespan than DUP metadata. It\(aqs more or less impossible to |
| 3458 |
predict when a SSD will run out of lifetime writes to within a factor of two, so |
| 3459 |
it\(aqs hard to justify wear reduction as a benefit. |
| 3460 |
.sp |
| 3461 |
Further reading: |
| 3462 |
.INDENT 0.0 |
| 3463 |
.IP \(bu 2 |
| 3464 |
\%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-and-deduplication-end-spinning-disk-2012> |
| 3465 |
.IP \(bu 2 |
| 3466 |
\%<https://\:www\:.snia\:.org/\:educational-library/\:realities-solid-state-storage-2013-2013> |
| 3467 |
.IP \(bu 2 |
| 3468 |
\%<https://\:www\:.snia\:.org/\:educational-library/\:ssd-performance-primer-2013> |
| 3469 |
.IP \(bu 2 |
| 3470 |
\%<https://\:www\:.snia\:.org/\:educational-library/\:how-controllers-maximize-ssd-life-2013> |
| 3471 |
.UNINDENT |
| 3472 |
.sp |
| 3473 |
What to do: |
| 3474 |
.INDENT 0.0 |
| 3475 |
.IP \(bu 2 |
| 3476 |
run \fBsmartctl\fP or self\-tests to look for potential issues |
| 3477 |
.IP \(bu 2 |
| 3478 |
keep the firmware up\-to\-date |
| 3479 |
.UNINDENT |
| 3480 |
.SS NVM express, non\-volatile memory (NVMe) |
| 3481 |
.sp |
| 3482 |
NVMe is a type of persistent memory usually connected over a system bus (PCIe) |
| 3483 |
or similar interface and the speeds are an order of magnitude faster than SSD. |
| 3484 |
It is also a non\-rotating type of storage, and is not typically connected by a |
| 3485 |
cable. It\(aqs not a SCSI type device either but rather a complete specification |
| 3486 |
for logical device interface. |
| 3487 |
.sp |
| 3488 |
In a way the errors could be compared to a combination of SSD class and regular |
| 3489 |
memory. Errors may exhibit as random bit flips or IO failures. There are tools |
| 3490 |
to access the internal log (\fBnvme log\fP and \fBnvme\-cli\fP) for a more detailed |
| 3491 |
analysis. |
| 3492 |
.sp |
| 3493 |
There are separate error detection and correction steps performed e.g. on the |
| 3494 |
bus level and in most cases never making in to the filesystem level. Once this |
| 3495 |
happens it could mean there\(aqs some systematic error like overheating or bad |
| 3496 |
physical connection of the device. You may want to run self\-tests (using |
| 3497 |
\fBsmartctl\fP). |
| 3498 |
.INDENT 0.0 |
| 3499 |
.IP \(bu 2 |
| 3500 |
\%<https://\:en\:.wikipedia\:.org/\:wiki/\:NVM_Express> |
| 3501 |
.IP \(bu 2 |
| 3502 |
\%<https://\:www\:.smartmontools\:.org/\:wiki/\:NVMe_Support> |
| 3503 |
.UNINDENT |
| 3504 |
.SS Drive firmware |
| 3505 |
.sp |
| 3506 |
Firmware is technically still software but embedded into the hardware. As all |
| 3507 |
software has bugs, so does firmware. Storage devices can update the firmware |
| 3508 |
and fix known bugs. In some cases the it\(aqs possible to avoid certain bugs by |
| 3509 |
quirks (device\-specific workarounds) in Linux kernel. |
| 3510 |
.sp |
| 3511 |
A faulty firmware can cause wide range of corruptions from small and localized |
| 3512 |
to large affecting lots of data. Self\-repair capabilities may not be sufficient. |
| 3513 |
.sp |
| 3514 |
What to do: |
| 3515 |
.INDENT 0.0 |
| 3516 |
.IP \(bu 2 |
| 3517 |
check for firmware updates in case there are known problems, note that |
| 3518 |
updating firmware can be risky on itself |
| 3519 |
.IP \(bu 2 |
| 3520 |
use up\-to\-date kernel (recent releases or maintained long term support versions) |
| 3521 |
.UNINDENT |
| 3522 |
.SS SD flash cards |
| 3523 |
.sp |
| 3524 |
There are a lot of devices with low power consumption and thus using storage |
| 3525 |
media based on low power consumption too, typically flash memory stored on |
| 3526 |
a chip enclosed in a detachable card package. An improperly inserted card may be |
| 3527 |
damaged by electrical spikes when the device is turned on or off. The chips |
| 3528 |
storing data in turn may be damaged permanently. All types of flash memory |
| 3529 |
have a limited number of rewrites, so the data are internally translated by FTL |
| 3530 |
(flash translation layer). This is implemented in firmware (technically a |
| 3531 |
software) and prone to bugs that manifest as hardware errors. |
| 3532 |
.sp |
| 3533 |
Adding redundancy like using DUP profiles for both data and metadata can help |
| 3534 |
in some cases but a full backup might be the best option once problems appear |
| 3535 |
and replacing the card could be required as well. |
| 3536 |
.SS Hardware as the main source of filesystem corruptions |
| 3537 |
.sp |
| 3538 |
\fBIf you use unreliable hardware and don\(aqt know about that, don\(aqt blame the |
| 3539 |
filesystem when it tells you.\fP |
| 3540 |
.SH SEE ALSO |
| 3541 |
.sp |
| 3542 |
acl(5) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man5/\:acl\:.5\:.html>, |
| 3543 |
btrfs(8) \%<>, |
| 3544 |
chattr(1) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man1/\:chattr\:.1\:.html>, |
| 3545 |
fstrim(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:fstrim\:.8\:.html>, |
| 3546 |
ioctl(2) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man2/\:ioctl\:.2\:.html>, |
| 3547 |
btrfs\-ioctl(2) \%<>, |
| 3548 |
mkfs.btrfs(8) \%<>, |
| 3549 |
mount(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:mount\:.8\:.html>, |
| 3550 |
swapon(8) \%<https://\:man7\:.org/\:linux/\:man-pages/\:man8/\:swapon\:.8\:.html> |
| 3551 |
.\" End of generated man page. |