| 1 |
======================= |
| 2 |
Kernel Samepage Merging |
| 3 |
======================= |
| 4 |
|
| 5 |
Overview |
| 6 |
======== |
| 7 |
|
| 8 |
KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y, |
| 9 |
added to the Linux kernel in 2.6.32. See ``mm/ksm.c`` for its implementation, |
| 10 |
and http://lwn.net/Articles/306704/ and https://lwn.net/Articles/330589/ |
| 11 |
|
| 12 |
KSM was originally developed for use with KVM (where it was known as |
| 13 |
Kernel Shared Memory), to fit more virtual machines into physical memory, |
| 14 |
by sharing the data common between them. But it can be useful to any |
| 15 |
application which generates many instances of the same data. |
| 16 |
|
| 17 |
The KSM daemon ksmd periodically scans those areas of user memory |
| 18 |
which have been registered with it, looking for pages of identical |
| 19 |
content which can be replaced by a single write-protected page (which |
| 20 |
is automatically copied if a process later wants to update its |
| 21 |
content). The amount of pages that KSM daemon scans in a single pass |
| 22 |
and the time between the passes are configured using :ref:`sysfs |
| 23 |
interface <ksm_sysfs>` |
| 24 |
|
| 25 |
KSM only merges anonymous (private) pages, never pagecache (file) pages. |
| 26 |
KSM's merged pages were originally locked into kernel memory, but can now |
| 27 |
be swapped out just like other user pages (but sharing is broken when they |
| 28 |
are swapped back in: ksmd must rediscover their identity and merge again). |
| 29 |
|
| 30 |
Controlling KSM with madvise |
| 31 |
============================ |
| 32 |
|
| 33 |
KSM only operates on those areas of address space which an application |
| 34 |
has advised to be likely candidates for merging, by using the madvise(2) |
| 35 |
system call:: |
| 36 |
|
| 37 |
int madvise(addr, length, MADV_MERGEABLE) |
| 38 |
|
| 39 |
The app may call |
| 40 |
|
| 41 |
:: |
| 42 |
|
| 43 |
int madvise(addr, length, MADV_UNMERGEABLE) |
| 44 |
|
| 45 |
to cancel that advice and restore unshared pages: whereupon KSM |
| 46 |
unmerges whatever it merged in that range. Note: this unmerging call |
| 47 |
may suddenly require more memory than is available - possibly failing |
| 48 |
with EAGAIN, but more probably arousing the Out-Of-Memory killer. |
| 49 |
|
| 50 |
If KSM is not configured into the running kernel, madvise MADV_MERGEABLE |
| 51 |
and MADV_UNMERGEABLE simply fail with EINVAL. If the running kernel was |
| 52 |
built with CONFIG_KSM=y, those calls will normally succeed: even if the |
| 53 |
KSM daemon is not currently running, MADV_MERGEABLE still registers |
| 54 |
the range for whenever the KSM daemon is started; even if the range |
| 55 |
cannot contain any pages which KSM could actually merge; even if |
| 56 |
MADV_UNMERGEABLE is applied to a range which was never MADV_MERGEABLE. |
| 57 |
|
| 58 |
If a region of memory must be split into at least one new MADV_MERGEABLE |
| 59 |
or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process |
| 60 |
will exceed ``vm.max_map_count`` (see Documentation/admin-guide/sysctl/vm.rst). |
| 61 |
|
| 62 |
Like other madvise calls, they are intended for use on mapped areas of |
| 63 |
the user address space: they will report ENOMEM if the specified range |
| 64 |
includes unmapped gaps (though working on the intervening mapped areas), |
| 65 |
and might fail with EAGAIN if not enough memory for internal structures. |
| 66 |
|
| 67 |
Applications should be considerate in their use of MADV_MERGEABLE, |
| 68 |
restricting its use to areas likely to benefit. KSM's scans may use a lot |
| 69 |
of processing power: some installations will disable KSM for that reason. |
| 70 |
|
| 71 |
.. _ksm_sysfs: |
| 72 |
|
| 73 |
KSM daemon sysfs interface |
| 74 |
========================== |
| 75 |
|
| 76 |
The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``, |
| 77 |
readable by all but writable only by root: |
| 78 |
|
| 79 |
pages_to_scan |
| 80 |
how many pages to scan before ksmd goes to sleep |
| 81 |
e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``. |
| 82 |
|
| 83 |
The pages_to_scan value cannot be changed if ``advisor_mode`` has |
| 84 |
been set to scan-time. |
| 85 |
|
| 86 |
Default: 100 (chosen for demonstration purposes) |
| 87 |
|
| 88 |
sleep_millisecs |
| 89 |
how many milliseconds ksmd should sleep before next scan |
| 90 |
e.g. ``echo 20 > /sys/kernel/mm/ksm/sleep_millisecs`` |
| 91 |
|
| 92 |
Default: 20 (chosen for demonstration purposes) |
| 93 |
|
| 94 |
merge_across_nodes |
| 95 |
specifies if pages from different NUMA nodes can be merged. |
| 96 |
When set to 0, ksm merges only pages which physically reside |
| 97 |
in the memory area of same NUMA node. That brings lower |
| 98 |
latency to access of shared pages. Systems with more nodes, at |
| 99 |
significant NUMA distances, are likely to benefit from the |
| 100 |
lower latency of setting 0. Smaller systems, which need to |
| 101 |
minimize memory usage, are likely to benefit from the greater |
| 102 |
sharing of setting 1 (default). You may wish to compare how |
| 103 |
your system performs under each setting, before deciding on |
| 104 |
which to use. ``merge_across_nodes`` setting can be changed only |
| 105 |
when there are no ksm shared pages in the system: set run 2 to |
| 106 |
unmerge pages first, then to 1 after changing |
| 107 |
``merge_across_nodes``, to remerge according to the new setting. |
| 108 |
|
| 109 |
Default: 1 (merging across nodes as in earlier releases) |
| 110 |
|
| 111 |
run |
| 112 |
* set to 0 to stop ksmd from running but keep merged pages, |
| 113 |
* set to 1 to run ksmd e.g. ``echo 1 > /sys/kernel/mm/ksm/run``, |
| 114 |
* set to 2 to stop ksmd and unmerge all pages currently merged, but |
| 115 |
leave mergeable areas registered for next run. |
| 116 |
|
| 117 |
Default: 0 (must be changed to 1 to activate KSM, except if |
| 118 |
CONFIG_SYSFS is disabled) |
| 119 |
|
| 120 |
use_zero_pages |
| 121 |
specifies whether empty pages (i.e. allocated pages that only |
| 122 |
contain zeroes) should be treated specially. When set to 1, |
| 123 |
empty pages are merged with the kernel zero page(s) instead of |
| 124 |
with each other as it would happen normally. This can improve |
| 125 |
the performance on architectures with coloured zero pages, |
| 126 |
depending on the workload. Care should be taken when enabling |
| 127 |
this setting, as it can potentially degrade the performance of |
| 128 |
KSM for some workloads, for example if the checksums of pages |
| 129 |
candidate for merging match the checksum of an empty |
| 130 |
page. This setting can be changed at any time, it is only |
| 131 |
effective for pages merged after the change. |
| 132 |
|
| 133 |
Default: 0 (normal KSM behaviour as in earlier releases) |
| 134 |
|
| 135 |
max_page_sharing |
| 136 |
Maximum sharing allowed for each KSM page. This enforces a |
| 137 |
deduplication limit to avoid high latency for virtual memory |
| 138 |
operations that involve traversal of the virtual mappings that |
| 139 |
share the KSM page. The minimum value is 2 as a newly created |
| 140 |
KSM page will have at least two sharers. The higher this value |
| 141 |
the faster KSM will merge the memory and the higher the |
| 142 |
deduplication factor will be, but the slower the worst case |
| 143 |
virtual mappings traversal could be for any given KSM |
| 144 |
page. Slowing down this traversal means there will be higher |
| 145 |
latency for certain virtual memory operations happening during |
| 146 |
swapping, compaction, NUMA balancing and page migration, in |
| 147 |
turn decreasing responsiveness for the caller of those virtual |
| 148 |
memory operations. The scheduler latency of other tasks not |
| 149 |
involved with the VM operations doing the virtual mappings |
| 150 |
traversal is not affected by this parameter as these |
| 151 |
traversals are always schedule friendly themselves. |
| 152 |
|
| 153 |
stable_node_chains_prune_millisecs |
| 154 |
specifies how frequently KSM checks the metadata of the pages |
| 155 |
that hit the deduplication limit for stale information. |
| 156 |
Smaller milllisecs values will free up the KSM metadata with |
| 157 |
lower latency, but they will make ksmd use more CPU during the |
| 158 |
scan. It's a noop if not a single KSM page hit the |
| 159 |
``max_page_sharing`` yet. |
| 160 |
|
| 161 |
smart_scan |
| 162 |
Historically KSM checked every candidate page for each scan. It did |
| 163 |
not take into account historic information. When smart scan is |
| 164 |
enabled, pages that have previously not been de-duplicated get |
| 165 |
skipped. How often these pages are skipped depends on how often |
| 166 |
de-duplication has already been tried and failed. By default this |
| 167 |
optimization is enabled. The ``pages_skipped`` metric shows how |
| 168 |
effective the setting is. |
| 169 |
|
| 170 |
advisor_mode |
| 171 |
The ``advisor_mode`` selects the current advisor. Two modes are |
| 172 |
supported: none and scan-time. The default is none. By setting |
| 173 |
``advisor_mode`` to scan-time, the scan time advisor is enabled. |
| 174 |
The section about ``advisor`` explains in detail how the scan time |
| 175 |
advisor works. |
| 176 |
|
| 177 |
adivsor_max_cpu |
| 178 |
specifies the upper limit of the cpu percent usage of the ksmd |
| 179 |
background thread. The default is 70. |
| 180 |
|
| 181 |
advisor_target_scan_time |
| 182 |
specifies the target scan time in seconds to scan all the candidate |
| 183 |
pages. The default value is 200 seconds. |
| 184 |
|
| 185 |
advisor_min_pages_to_scan |
| 186 |
specifies the lower limit of the ``pages_to_scan`` parameter of the |
| 187 |
scan time advisor. The default is 500. |
| 188 |
|
| 189 |
adivsor_max_pages_to_scan |
| 190 |
specifies the upper limit of the ``pages_to_scan`` parameter of the |
| 191 |
scan time advisor. The default is 30000. |
| 192 |
|
| 193 |
The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``: |
| 194 |
|
| 195 |
general_profit |
| 196 |
how effective is KSM. The calculation is explained below. |
| 197 |
pages_scanned |
| 198 |
how many pages are being scanned for ksm |
| 199 |
pages_shared |
| 200 |
how many shared pages are being used |
| 201 |
pages_sharing |
| 202 |
how many more sites are sharing them i.e. how much saved |
| 203 |
pages_unshared |
| 204 |
how many pages unique but repeatedly checked for merging |
| 205 |
pages_volatile |
| 206 |
how many pages changing too fast to be placed in a tree |
| 207 |
pages_skipped |
| 208 |
how many pages did the "smart" page scanning algorithm skip |
| 209 |
full_scans |
| 210 |
how many times all mergeable areas have been scanned |
| 211 |
stable_node_chains |
| 212 |
the number of KSM pages that hit the ``max_page_sharing`` limit |
| 213 |
stable_node_dups |
| 214 |
number of duplicated KSM pages |
| 215 |
ksm_zero_pages |
| 216 |
how many zero pages that are still mapped into processes were mapped by |
| 217 |
KSM when deduplicating. |
| 218 |
|
| 219 |
When ``use_zero_pages`` is/was enabled, the sum of ``pages_sharing`` + |
| 220 |
``ksm_zero_pages`` represents the actual number of pages saved by KSM. |
| 221 |
if ``use_zero_pages`` has never been enabled, ``ksm_zero_pages`` is 0. |
| 222 |
|
| 223 |
A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good |
| 224 |
sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing`` |
| 225 |
indicates wasted effort. ``pages_volatile`` embraces several |
| 226 |
different kinds of activity, but a high proportion there would also |
| 227 |
indicate poor use of madvise MADV_MERGEABLE. |
| 228 |
|
| 229 |
The maximum possible ``pages_sharing/pages_shared`` ratio is limited by the |
| 230 |
``max_page_sharing`` tunable. To increase the ratio ``max_page_sharing`` must |
| 231 |
be increased accordingly. |
| 232 |
|
| 233 |
Monitoring KSM profit |
| 234 |
===================== |
| 235 |
|
| 236 |
KSM can save memory by merging identical pages, but also can consume |
| 237 |
additional memory, because it needs to generate a number of rmap_items to |
| 238 |
save each scanned page's brief rmap information. Some of these pages may |
| 239 |
be merged, but some may not be abled to be merged after being checked |
| 240 |
several times, which are unprofitable memory consumed. |
| 241 |
|
| 242 |
1) How to determine whether KSM save memory or consume memory in system-wide |
| 243 |
range? Here is a simple approximate calculation for reference:: |
| 244 |
|
| 245 |
general_profit =~ ksm_saved_pages * sizeof(page) - (all_rmap_items) * |
| 246 |
sizeof(rmap_item); |
| 247 |
|
| 248 |
where ksm_saved_pages equals to the sum of ``pages_sharing`` + |
| 249 |
``ksm_zero_pages`` of the system, and all_rmap_items can be easily |
| 250 |
obtained by summing ``pages_sharing``, ``pages_shared``, ``pages_unshared`` |
| 251 |
and ``pages_volatile``. |
| 252 |
|
| 253 |
2) The KSM profit inner a single process can be similarly obtained by the |
| 254 |
following approximate calculation:: |
| 255 |
|
| 256 |
process_profit =~ ksm_saved_pages * sizeof(page) - |
| 257 |
ksm_rmap_items * sizeof(rmap_item). |
| 258 |
|
| 259 |
where ksm_saved_pages equals to the sum of ``ksm_merging_pages`` and |
| 260 |
``ksm_zero_pages``, both of which are shown under the directory |
| 261 |
``/proc/<pid>/ksm_stat``, and ksm_rmap_items is also shown in |
| 262 |
``/proc/<pid>/ksm_stat``. The process profit is also shown in |
| 263 |
``/proc/<pid>/ksm_stat`` as ksm_process_profit. |
| 264 |
|
| 265 |
From the perspective of application, a high ratio of ``ksm_rmap_items`` to |
| 266 |
``ksm_merging_pages`` means a bad madvise-applied policy, so developers or |
| 267 |
administrators have to rethink how to change madvise policy. Giving an example |
| 268 |
for reference, a page's size is usually 4K, and the rmap_item's size is |
| 269 |
separately 32B on 32-bit CPU architecture and 64B on 64-bit CPU architecture. |
| 270 |
so if the ``ksm_rmap_items/ksm_merging_pages`` ratio exceeds 64 on 64-bit CPU |
| 271 |
or exceeds 128 on 32-bit CPU, then the app's madvise policy should be dropped, |
| 272 |
because the ksm profit is approximately zero or negative. |
| 273 |
|
| 274 |
Monitoring KSM events |
| 275 |
===================== |
| 276 |
|
| 277 |
There are some counters in /proc/vmstat that may be used to monitor KSM events. |
| 278 |
KSM might help save memory, it's a tradeoff by may suffering delay on KSM COW |
| 279 |
or on swapping in copy. Those events could help users evaluate whether or how |
| 280 |
to use KSM. For example, if cow_ksm increases too fast, user may decrease the |
| 281 |
range of madvise(, , MADV_MERGEABLE). |
| 282 |
|
| 283 |
cow_ksm |
| 284 |
is incremented every time a KSM page triggers copy on write (COW) |
| 285 |
when users try to write to a KSM page, we have to make a copy. |
| 286 |
|
| 287 |
ksm_swpin_copy |
| 288 |
is incremented every time a KSM page is copied when swapping in |
| 289 |
note that KSM page might be copied when swapping in because do_swap_page() |
| 290 |
cannot do all the locking needed to reconstitute a cross-anon_vma KSM page. |
| 291 |
|
| 292 |
Advisor |
| 293 |
======= |
| 294 |
|
| 295 |
The number of candidate pages for KSM is dynamic. It can be often observed |
| 296 |
that during the startup of an application more candidate pages need to be |
| 297 |
processed. Without an advisor the ``pages_to_scan`` parameter needs to be |
| 298 |
sized for the maximum number of candidate pages. The scan time advisor can |
| 299 |
changes the ``pages_to_scan`` parameter based on demand. |
| 300 |
|
| 301 |
The advisor can be enabled, so KSM can automatically adapt to changes in the |
| 302 |
number of candidate pages to scan. Two advisors are implemented: none and |
| 303 |
scan-time. With none, no advisor is enabled. The default is none. |
| 304 |
|
| 305 |
The scan time advisor changes the ``pages_to_scan`` parameter based on the |
| 306 |
observed scan times. The possible values for the ``pages_to_scan`` parameter is |
| 307 |
limited by the ``advisor_max_cpu`` parameter. In addition there is also the |
| 308 |
``advisor_target_scan_time`` parameter. This parameter sets the target time to |
| 309 |
scan all the KSM candidate pages. The parameter ``advisor_target_scan_time`` |
| 310 |
decides how aggressive the scan time advisor scans candidate pages. Lower |
| 311 |
values make the scan time advisor to scan more aggressively. This is the most |
| 312 |
important parameter for the configuration of the scan time advisor. |
| 313 |
|
| 314 |
The initial value and the maximum value can be changed with |
| 315 |
``advisor_min_pages_to_scan`` and ``advisor_max_pages_to_scan``. The default |
| 316 |
values are sufficient for most workloads and use cases. |
| 317 |
|
| 318 |
The ``pages_to_scan`` parameter is re-calculated after a scan has been completed. |
| 319 |
|
| 320 |
|
| 321 |
-- |
| 322 |
Izik Eidus, |
| 323 |
Hugh Dickins, 17 Nov 2009 |