| 1 |
============= |
| 2 |
HugeTLB Pages |
| 3 |
============= |
| 4 |
|
| 5 |
Overview |
| 6 |
======== |
| 7 |
|
| 8 |
The intent of this file is to give a brief summary of hugetlbpage support in |
| 9 |
the Linux kernel. This support is built on top of multiple page size support |
| 10 |
that is provided by most modern architectures. For example, x86 CPUs normally |
| 11 |
support 4K and 2M (1G if architecturally supported) page sizes, ia64 |
| 12 |
architecture supports multiple page sizes 4K, 8K, 64K, 256K, 1M, 4M, 16M, |
| 13 |
256M and ppc64 supports 4K and 16M. A TLB is a cache of virtual-to-physical |
| 14 |
translations. Typically this is a very scarce resource on processor. |
| 15 |
Operating systems try to make best use of limited number of TLB resources. |
| 16 |
This optimization is more critical now as bigger and bigger physical memories |
| 17 |
(several GBs) are more readily available. |
| 18 |
|
| 19 |
Users can use the huge page support in Linux kernel by either using the mmap |
| 20 |
system call or standard SYSV shared memory system calls (shmget, shmat). |
| 21 |
|
| 22 |
First the Linux kernel needs to be built with the CONFIG_HUGETLBFS |
| 23 |
(present under "File systems") and CONFIG_HUGETLB_PAGE (selected |
| 24 |
automatically when CONFIG_HUGETLBFS is selected) configuration |
| 25 |
options. |
| 26 |
|
| 27 |
The ``/proc/meminfo`` file provides information about the total number of |
| 28 |
persistent hugetlb pages in the kernel's huge page pool. It also displays |
| 29 |
default huge page size and information about the number of free, reserved |
| 30 |
and surplus huge pages in the pool of huge pages of default size. |
| 31 |
The huge page size is needed for generating the proper alignment and |
| 32 |
size of the arguments to system calls that map huge page regions. |
| 33 |
|
| 34 |
The output of ``cat /proc/meminfo`` will include lines like:: |
| 35 |
|
| 36 |
HugePages_Total: uuu |
| 37 |
HugePages_Free: vvv |
| 38 |
HugePages_Rsvd: www |
| 39 |
HugePages_Surp: xxx |
| 40 |
Hugepagesize: yyy kB |
| 41 |
Hugetlb: zzz kB |
| 42 |
|
| 43 |
where: |
| 44 |
|
| 45 |
HugePages_Total |
| 46 |
is the size of the pool of huge pages. |
| 47 |
HugePages_Free |
| 48 |
is the number of huge pages in the pool that are not yet |
| 49 |
allocated. |
| 50 |
HugePages_Rsvd |
| 51 |
is short for "reserved," and is the number of huge pages for |
| 52 |
which a commitment to allocate from the pool has been made, |
| 53 |
but no allocation has yet been made. Reserved huge pages |
| 54 |
guarantee that an application will be able to allocate a |
| 55 |
huge page from the pool of huge pages at fault time. |
| 56 |
HugePages_Surp |
| 57 |
is short for "surplus," and is the number of huge pages in |
| 58 |
the pool above the value in ``/proc/sys/vm/nr_hugepages``. The |
| 59 |
maximum number of surplus huge pages is controlled by |
| 60 |
``/proc/sys/vm/nr_overcommit_hugepages``. |
| 61 |
Note: When the feature of freeing unused vmemmap pages associated |
| 62 |
with each hugetlb page is enabled, the number of surplus huge pages |
| 63 |
may be temporarily larger than the maximum number of surplus huge |
| 64 |
pages when the system is under memory pressure. |
| 65 |
Hugepagesize |
| 66 |
is the default hugepage size (in kB). |
| 67 |
Hugetlb |
| 68 |
is the total amount of memory (in kB), consumed by huge |
| 69 |
pages of all sizes. |
| 70 |
If huge pages of different sizes are in use, this number |
| 71 |
will exceed HugePages_Total \* Hugepagesize. To get more |
| 72 |
detailed information, please, refer to |
| 73 |
``/sys/kernel/mm/hugepages`` (described below). |
| 74 |
|
| 75 |
|
| 76 |
``/proc/filesystems`` should also show a filesystem of type "hugetlbfs" |
| 77 |
configured in the kernel. |
| 78 |
|
| 79 |
``/proc/sys/vm/nr_hugepages`` indicates the current number of "persistent" huge |
| 80 |
pages in the kernel's huge page pool. "Persistent" huge pages will be |
| 81 |
returned to the huge page pool when freed by a task. A user with root |
| 82 |
privileges can dynamically allocate more or free some persistent huge pages |
| 83 |
by increasing or decreasing the value of ``nr_hugepages``. |
| 84 |
|
| 85 |
Note: When the feature of freeing unused vmemmap pages associated with each |
| 86 |
hugetlb page is enabled, we can fail to free the huge pages triggered by |
| 87 |
the user when the system is under memory pressure. Please try again later. |
| 88 |
|
| 89 |
Pages that are used as huge pages are reserved inside the kernel and cannot |
| 90 |
be used for other purposes. Huge pages cannot be swapped out under |
| 91 |
memory pressure. |
| 92 |
|
| 93 |
Once a number of huge pages have been pre-allocated to the kernel huge page |
| 94 |
pool, a user with appropriate privilege can use either the mmap system call |
| 95 |
or shared memory system calls to use the huge pages. See the discussion of |
| 96 |
:ref:`Using Huge Pages <using_huge_pages>`, below. |
| 97 |
|
| 98 |
The administrator can allocate persistent huge pages on the kernel boot |
| 99 |
command line by specifying the "hugepages=N" parameter, where 'N' = the |
| 100 |
number of huge pages requested. This is the most reliable method of |
| 101 |
allocating huge pages as memory has not yet become fragmented. |
| 102 |
|
| 103 |
Some platforms support multiple huge page sizes. To allocate huge pages |
| 104 |
of a specific size, one must precede the huge pages boot command parameters |
| 105 |
with a huge page size selection parameter "hugepagesz=<size>". <size> must |
| 106 |
be specified in bytes with optional scale suffix [kKmMgG]. The default huge |
| 107 |
page size may be selected with the "default_hugepagesz=<size>" boot parameter. |
| 108 |
|
| 109 |
Hugetlb boot command line parameter semantics |
| 110 |
|
| 111 |
hugepagesz |
| 112 |
Specify a huge page size. Used in conjunction with hugepages |
| 113 |
parameter to preallocate a number of huge pages of the specified |
| 114 |
size. Hence, hugepagesz and hugepages are typically specified in |
| 115 |
pairs such as:: |
| 116 |
|
| 117 |
hugepagesz=2M hugepages=512 |
| 118 |
|
| 119 |
hugepagesz can only be specified once on the command line for a |
| 120 |
specific huge page size. Valid huge page sizes are architecture |
| 121 |
dependent. |
| 122 |
hugepages |
| 123 |
Specify the number of huge pages to preallocate. This typically |
| 124 |
follows a valid hugepagesz or default_hugepagesz parameter. However, |
| 125 |
if hugepages is the first or only hugetlb command line parameter it |
| 126 |
implicitly specifies the number of huge pages of default size to |
| 127 |
allocate. If the number of huge pages of default size is implicitly |
| 128 |
specified, it can not be overwritten by a hugepagesz,hugepages |
| 129 |
parameter pair for the default size. This parameter also has a |
| 130 |
node format. The node format specifies the number of huge pages |
| 131 |
to allocate on specific nodes. |
| 132 |
|
| 133 |
For example, on an architecture with 2M default huge page size:: |
| 134 |
|
| 135 |
hugepages=256 hugepagesz=2M hugepages=512 |
| 136 |
|
| 137 |
will result in 256 2M huge pages being allocated and a warning message |
| 138 |
indicating that the hugepages=512 parameter is ignored. If a hugepages |
| 139 |
parameter is preceded by an invalid hugepagesz parameter, it will |
| 140 |
be ignored. |
| 141 |
|
| 142 |
Node format example:: |
| 143 |
|
| 144 |
hugepagesz=2M hugepages=0:1,1:2 |
| 145 |
|
| 146 |
It will allocate 1 2M hugepage on node0 and 2 2M hugepages on node1. |
| 147 |
If the node number is invalid, the parameter will be ignored. |
| 148 |
hugepage_alloc_threads |
| 149 |
Specify the number of threads that should be used to allocate hugepages |
| 150 |
during boot. This parameter can be used to improve system bootup time |
| 151 |
when allocating a large amount of huge pages. |
| 152 |
|
| 153 |
The default value is 25% of the available hardware threads. |
| 154 |
Example to use 8 allocation threads:: |
| 155 |
|
| 156 |
hugepage_alloc_threads=8 |
| 157 |
|
| 158 |
Note that this parameter only applies to non-gigantic huge pages. |
| 159 |
default_hugepagesz |
| 160 |
Specify the default huge page size. This parameter can |
| 161 |
only be specified once on the command line. default_hugepagesz can |
| 162 |
optionally be followed by the hugepages parameter to preallocate a |
| 163 |
specific number of huge pages of default size. The number of default |
| 164 |
sized huge pages to preallocate can also be implicitly specified as |
| 165 |
mentioned in the hugepages section above. Therefore, on an |
| 166 |
architecture with 2M default huge page size:: |
| 167 |
|
| 168 |
hugepages=256 |
| 169 |
default_hugepagesz=2M hugepages=256 |
| 170 |
hugepages=256 default_hugepagesz=2M |
| 171 |
|
| 172 |
will all result in 256 2M huge pages being allocated. Valid default |
| 173 |
huge page size is architecture dependent. |
| 174 |
hugetlb_free_vmemmap |
| 175 |
When CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP is set, this enables HugeTLB |
| 176 |
Vmemmap Optimization (HVO). |
| 177 |
|
| 178 |
When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages`` |
| 179 |
indicates the current number of pre-allocated huge pages of the default size. |
| 180 |
Thus, one can use the following command to dynamically allocate/deallocate |
| 181 |
default sized persistent huge pages:: |
| 182 |
|
| 183 |
echo 20 > /proc/sys/vm/nr_hugepages |
| 184 |
|
| 185 |
This command will try to adjust the number of default sized huge pages in the |
| 186 |
huge page pool to 20, allocating or freeing huge pages, as required. |
| 187 |
|
| 188 |
On a NUMA platform, the kernel will attempt to distribute the huge page pool |
| 189 |
over all the set of allowed nodes specified by the NUMA memory policy of the |
| 190 |
task that modifies ``nr_hugepages``. The default for the allowed nodes--when the |
| 191 |
task has default memory policy--is all on-line nodes with memory. Allowed |
| 192 |
nodes with insufficient available, contiguous memory for a huge page will be |
| 193 |
silently skipped when allocating persistent huge pages. See the |
| 194 |
:ref:`discussion below <mem_policy_and_hp_alloc>` |
| 195 |
of the interaction of task memory policy, cpusets and per node attributes |
| 196 |
with the allocation and freeing of persistent huge pages. |
| 197 |
|
| 198 |
The success or failure of huge page allocation depends on the amount of |
| 199 |
physically contiguous memory that is present in system at the time of the |
| 200 |
allocation attempt. If the kernel is unable to allocate huge pages from |
| 201 |
some nodes in a NUMA system, it will attempt to make up the difference by |
| 202 |
allocating extra pages on other nodes with sufficient available contiguous |
| 203 |
memory, if any. |
| 204 |
|
| 205 |
System administrators may want to put this command in one of the local rc |
| 206 |
init files. This will enable the kernel to allocate huge pages early in |
| 207 |
the boot process when the possibility of getting physical contiguous pages |
| 208 |
is still very high. Administrators can verify the number of huge pages |
| 209 |
actually allocated by checking the sysctl or meminfo. To check the per node |
| 210 |
distribution of huge pages in a NUMA system, use:: |
| 211 |
|
| 212 |
cat /sys/devices/system/node/node*/meminfo | fgrep Huge |
| 213 |
|
| 214 |
``/proc/sys/vm/nr_overcommit_hugepages`` specifies how large the pool of |
| 215 |
huge pages can grow, if more huge pages than ``/proc/sys/vm/nr_hugepages`` are |
| 216 |
requested by applications. Writing any non-zero value into this file |
| 217 |
indicates that the hugetlb subsystem is allowed to try to obtain that |
| 218 |
number of "surplus" huge pages from the kernel's normal page pool, when the |
| 219 |
persistent huge page pool is exhausted. As these surplus huge pages become |
| 220 |
unused, they are freed back to the kernel's normal page pool. |
| 221 |
|
| 222 |
When increasing the huge page pool size via ``nr_hugepages``, any existing |
| 223 |
surplus pages will first be promoted to persistent huge pages. Then, additional |
| 224 |
huge pages will be allocated, if necessary and if possible, to fulfill |
| 225 |
the new persistent huge page pool size. |
| 226 |
|
| 227 |
The administrator may shrink the pool of persistent huge pages for |
| 228 |
the default huge page size by setting the ``nr_hugepages`` sysctl to a |
| 229 |
smaller value. The kernel will attempt to balance the freeing of huge pages |
| 230 |
across all nodes in the memory policy of the task modifying ``nr_hugepages``. |
| 231 |
Any free huge pages on the selected nodes will be freed back to the kernel's |
| 232 |
normal page pool. |
| 233 |
|
| 234 |
Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` such that |
| 235 |
it becomes less than the number of huge pages in use will convert the balance |
| 236 |
of the in-use huge pages to surplus huge pages. This will occur even if |
| 237 |
the number of surplus pages would exceed the overcommit value. As long as |
| 238 |
this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is |
| 239 |
increased sufficiently, or the surplus huge pages go out of use and are freed-- |
| 240 |
no more surplus huge pages will be allowed to be allocated. |
| 241 |
|
| 242 |
With support for multiple huge page pools at run-time available, much of |
| 243 |
the huge page userspace interface in ``/proc/sys/vm`` has been duplicated in |
| 244 |
sysfs. |
| 245 |
The ``/proc`` interfaces discussed above have been retained for backwards |
| 246 |
compatibility. The root huge page control directory in sysfs is:: |
| 247 |
|
| 248 |
/sys/kernel/mm/hugepages |
| 249 |
|
| 250 |
For each huge page size supported by the running kernel, a subdirectory |
| 251 |
will exist, of the form:: |
| 252 |
|
| 253 |
hugepages-${size}kB |
| 254 |
|
| 255 |
Inside each of these directories, the set of files contained in ``/proc`` |
| 256 |
will exist. In addition, two additional interfaces for demoting huge |
| 257 |
pages may exist:: |
| 258 |
|
| 259 |
demote |
| 260 |
demote_size |
| 261 |
nr_hugepages |
| 262 |
nr_hugepages_mempolicy |
| 263 |
nr_overcommit_hugepages |
| 264 |
free_hugepages |
| 265 |
resv_hugepages |
| 266 |
surplus_hugepages |
| 267 |
|
| 268 |
The demote interfaces provide the ability to split a huge page into |
| 269 |
smaller huge pages. For example, the x86 architecture supports both |
| 270 |
1GB and 2MB huge pages sizes. A 1GB huge page can be split into 512 |
| 271 |
2MB huge pages. Demote interfaces are not available for the smallest |
| 272 |
huge page size. The demote interfaces are: |
| 273 |
|
| 274 |
demote_size |
| 275 |
is the size of demoted pages. When a page is demoted a corresponding |
| 276 |
number of huge pages of demote_size will be created. By default, |
| 277 |
demote_size is set to the next smaller huge page size. If there are |
| 278 |
multiple smaller huge page sizes, demote_size can be set to any of |
| 279 |
these smaller sizes. Only huge page sizes less than the current huge |
| 280 |
pages size are allowed. |
| 281 |
|
| 282 |
demote |
| 283 |
is used to demote a number of huge pages. A user with root privileges |
| 284 |
can write to this file. It may not be possible to demote the |
| 285 |
requested number of huge pages. To determine how many pages were |
| 286 |
actually demoted, compare the value of nr_hugepages before and after |
| 287 |
writing to the demote interface. demote is a write only interface. |
| 288 |
|
| 289 |
The interfaces which are the same as in ``/proc`` (all except demote and |
| 290 |
demote_size) function as described above for the default huge page-sized case. |
| 291 |
|
| 292 |
.. _mem_policy_and_hp_alloc: |
| 293 |
|
| 294 |
Interaction of Task Memory Policy with Huge Page Allocation/Freeing |
| 295 |
=================================================================== |
| 296 |
|
| 297 |
Whether huge pages are allocated and freed via the ``/proc`` interface or |
| 298 |
the ``/sysfs`` interface using the ``nr_hugepages_mempolicy`` attribute, the |
| 299 |
NUMA nodes from which huge pages are allocated or freed are controlled by the |
| 300 |
NUMA memory policy of the task that modifies the ``nr_hugepages_mempolicy`` |
| 301 |
sysctl or attribute. When the ``nr_hugepages`` attribute is used, mempolicy |
| 302 |
is ignored. |
| 303 |
|
| 304 |
The recommended method to allocate or free huge pages to/from the kernel |
| 305 |
huge page pool, using the ``nr_hugepages`` example above, is:: |
| 306 |
|
| 307 |
numactl --interleave <node-list> echo 20 \ |
| 308 |
>/proc/sys/vm/nr_hugepages_mempolicy |
| 309 |
|
| 310 |
or, more succinctly:: |
| 311 |
|
| 312 |
numactl -m <node-list> echo 20 >/proc/sys/vm/nr_hugepages_mempolicy |
| 313 |
|
| 314 |
This will allocate or free ``abs(20 - nr_hugepages)`` to or from the nodes |
| 315 |
specified in <node-list>, depending on whether number of persistent huge pages |
| 316 |
is initially less than or greater than 20, respectively. No huge pages will be |
| 317 |
allocated nor freed on any node not included in the specified <node-list>. |
| 318 |
|
| 319 |
When adjusting the persistent hugepage count via ``nr_hugepages_mempolicy``, any |
| 320 |
memory policy mode--bind, preferred, local or interleave--may be used. The |
| 321 |
resulting effect on persistent huge page allocation is as follows: |
| 322 |
|
| 323 |
#. Regardless of mempolicy mode [see |
| 324 |
Documentation/admin-guide/mm/numa_memory_policy.rst], |
| 325 |
persistent huge pages will be distributed across the node or nodes |
| 326 |
specified in the mempolicy as if "interleave" had been specified. |
| 327 |
However, if a node in the policy does not contain sufficient contiguous |
| 328 |
memory for a huge page, the allocation will not "fallback" to the nearest |
| 329 |
neighbor node with sufficient contiguous memory. To do this would cause |
| 330 |
undesirable imbalance in the distribution of the huge page pool, or |
| 331 |
possibly, allocation of persistent huge pages on nodes not allowed by |
| 332 |
the task's memory policy. |
| 333 |
|
| 334 |
#. One or more nodes may be specified with the bind or interleave policy. |
| 335 |
If more than one node is specified with the preferred policy, only the |
| 336 |
lowest numeric id will be used. Local policy will select the node where |
| 337 |
the task is running at the time the nodes_allowed mask is constructed. |
| 338 |
For local policy to be deterministic, the task must be bound to a cpu or |
| 339 |
cpus in a single node. Otherwise, the task could be migrated to some |
| 340 |
other node at any time after launch and the resulting node will be |
| 341 |
indeterminate. Thus, local policy is not very useful for this purpose. |
| 342 |
Any of the other mempolicy modes may be used to specify a single node. |
| 343 |
|
| 344 |
#. The nodes allowed mask will be derived from any non-default task mempolicy, |
| 345 |
whether this policy was set explicitly by the task itself or one of its |
| 346 |
ancestors, such as numactl. This means that if the task is invoked from a |
| 347 |
shell with non-default policy, that policy will be used. One can specify a |
| 348 |
node list of "all" with numactl --interleave or --membind [-m] to achieve |
| 349 |
interleaving over all nodes in the system or cpuset. |
| 350 |
|
| 351 |
#. Any task mempolicy specified--e.g., using numactl--will be constrained by |
| 352 |
the resource limits of any cpuset in which the task runs. Thus, there will |
| 353 |
be no way for a task with non-default policy running in a cpuset with a |
| 354 |
subset of the system nodes to allocate huge pages outside the cpuset |
| 355 |
without first moving to a cpuset that contains all of the desired nodes. |
| 356 |
|
| 357 |
#. Boot-time huge page allocation attempts to distribute the requested number |
| 358 |
of huge pages over all on-lines nodes with memory. |
| 359 |
|
| 360 |
Per Node Hugepages Attributes |
| 361 |
============================= |
| 362 |
|
| 363 |
A subset of the contents of the root huge page control directory in sysfs, |
| 364 |
described above, will be replicated under each the system device of each |
| 365 |
NUMA node with memory in:: |
| 366 |
|
| 367 |
/sys/devices/system/node/node[0-9]*/hugepages/ |
| 368 |
|
| 369 |
Under this directory, the subdirectory for each supported huge page size |
| 370 |
contains the following attribute files:: |
| 371 |
|
| 372 |
nr_hugepages |
| 373 |
free_hugepages |
| 374 |
surplus_hugepages |
| 375 |
|
| 376 |
The free\_' and surplus\_' attribute files are read-only. They return the number |
| 377 |
of free and surplus [overcommitted] huge pages, respectively, on the parent |
| 378 |
node. |
| 379 |
|
| 380 |
The ``nr_hugepages`` attribute returns the total number of huge pages on the |
| 381 |
specified node. When this attribute is written, the number of persistent huge |
| 382 |
pages on the parent node will be adjusted to the specified value, if sufficient |
| 383 |
resources exist, regardless of the task's mempolicy or cpuset constraints. |
| 384 |
|
| 385 |
Note that the number of overcommit and reserve pages remain global quantities, |
| 386 |
as we don't know until fault time, when the faulting task's mempolicy is |
| 387 |
applied, from which node the huge page allocation will be attempted. |
| 388 |
|
| 389 |
The hugetlb may be migrated between the per-node hugepages pool in the following |
| 390 |
scenarios: memory offline, memory failure, longterm pinning, syscalls(mbind, |
| 391 |
migrate_pages and move_pages), alloc_contig_range() and alloc_contig_pages(). |
| 392 |
Now only memory offline, memory failure and syscalls allow fallbacking to allocate |
| 393 |
a new hugetlb on a different node if the current node is unable to allocate during |
| 394 |
hugetlb migration, that means these 3 cases can break the per-node hugepages pool. |
| 395 |
|
| 396 |
.. _using_huge_pages: |
| 397 |
|
| 398 |
Using Huge Pages |
| 399 |
================ |
| 400 |
|
| 401 |
If the user applications are going to request huge pages using mmap system |
| 402 |
call, then it is required that system administrator mount a file system of |
| 403 |
type hugetlbfs:: |
| 404 |
|
| 405 |
mount -t hugetlbfs \ |
| 406 |
-o uid=<value>,gid=<value>,mode=<value>,pagesize=<value>,size=<value>,\ |
| 407 |
min_size=<value>,nr_inodes=<value> none /mnt/huge |
| 408 |
|
| 409 |
This command mounts a (pseudo) filesystem of type hugetlbfs on the directory |
| 410 |
``/mnt/huge``. Any file created on ``/mnt/huge`` uses huge pages. |
| 411 |
|
| 412 |
The ``uid`` and ``gid`` options sets the owner and group of the root of the |
| 413 |
file system. By default the ``uid`` and ``gid`` of the current process |
| 414 |
are taken. |
| 415 |
|
| 416 |
The ``mode`` option sets the mode of root of file system to value & 01777. |
| 417 |
This value is given in octal. By default the value 0755 is picked. |
| 418 |
|
| 419 |
If the platform supports multiple huge page sizes, the ``pagesize`` option can |
| 420 |
be used to specify the huge page size and associated pool. ``pagesize`` |
| 421 |
is specified in bytes. If ``pagesize`` is not specified the platform's |
| 422 |
default huge page size and associated pool will be used. |
| 423 |
|
| 424 |
The ``size`` option sets the maximum value of memory (huge pages) allowed |
| 425 |
for that filesystem (``/mnt/huge``). The ``size`` option can be specified |
| 426 |
in bytes, or as a percentage of the specified huge page pool (``nr_hugepages``). |
| 427 |
The size is rounded down to HPAGE_SIZE boundary. |
| 428 |
|
| 429 |
The ``min_size`` option sets the minimum value of memory (huge pages) allowed |
| 430 |
for the filesystem. ``min_size`` can be specified in the same way as ``size``, |
| 431 |
either bytes or a percentage of the huge page pool. |
| 432 |
At mount time, the number of huge pages specified by ``min_size`` are reserved |
| 433 |
for use by the filesystem. |
| 434 |
If there are not enough free huge pages available, the mount will fail. |
| 435 |
As huge pages are allocated to the filesystem and freed, the reserve count |
| 436 |
is adjusted so that the sum of allocated and reserved huge pages is always |
| 437 |
at least ``min_size``. |
| 438 |
|
| 439 |
The option ``nr_inodes`` sets the maximum number of inodes that ``/mnt/huge`` |
| 440 |
can use. |
| 441 |
|
| 442 |
If the ``size``, ``min_size`` or ``nr_inodes`` option is not provided on |
| 443 |
command line then no limits are set. |
| 444 |
|
| 445 |
For ``pagesize``, ``size``, ``min_size`` and ``nr_inodes`` options, you can |
| 446 |
use [G|g]/[M|m]/[K|k] to represent giga/mega/kilo. |
| 447 |
For example, size=2K has the same meaning as size=2048. |
| 448 |
|
| 449 |
While read system calls are supported on files that reside on hugetlb |
| 450 |
file systems, write system calls are not. |
| 451 |
|
| 452 |
Regular chown, chgrp, and chmod commands (with right permissions) could be |
| 453 |
used to change the file attributes on hugetlbfs. |
| 454 |
|
| 455 |
Also, it is important to note that no such mount command is required if |
| 456 |
applications are going to use only shmat/shmget system calls or mmap with |
| 457 |
MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see |
| 458 |
:ref:`map_hugetlb <map_hugetlb>` below. |
| 459 |
|
| 460 |
Users who wish to use hugetlb memory via shared memory segment should be |
| 461 |
members of a supplementary group and system admin needs to configure that gid |
| 462 |
into ``/proc/sys/vm/hugetlb_shm_group``. It is possible for same or different |
| 463 |
applications to use any combination of mmaps and shm* calls, though the mount of |
| 464 |
filesystem will be required for using mmap calls without MAP_HUGETLB. |
| 465 |
|
| 466 |
Syscalls that operate on memory backed by hugetlb pages only have their lengths |
| 467 |
aligned to the native page size of the processor; they will normally fail with |
| 468 |
errno set to EINVAL or exclude hugetlb pages that extend beyond the length if |
| 469 |
not hugepage aligned. For example, munmap(2) will fail if memory is backed by |
| 470 |
a hugetlb page and the length is smaller than the hugepage size. |
| 471 |
|
| 472 |
|
| 473 |
Examples |
| 474 |
======== |
| 475 |
|
| 476 |
.. _map_hugetlb: |
| 477 |
|
| 478 |
``map_hugetlb`` |
| 479 |
see tools/testing/selftests/mm/map_hugetlb.c |
| 480 |
|
| 481 |
``hugepage-shm`` |
| 482 |
see tools/testing/selftests/mm/hugepage-shm.c |
| 483 |
|
| 484 |
``hugepage-mmap`` |
| 485 |
see tools/testing/selftests/mm/hugepage-mmap.c |
| 486 |
|
| 487 |
The `libhugetlbfs`_ library provides a wide range of userspace tools |
| 488 |
to help with huge page usability, environment setup, and control. |
| 489 |
|
| 490 |
.. _libhugetlbfs: https://github.com/libhugetlbfs/libhugetlbfs |