| 1 |
.. SPDX-License-Identifier: GPL-2.0 |
| 2 |
|
| 3 |
============================= |
| 4 |
DAMON-based LRU-lists Sorting |
| 5 |
============================= |
| 6 |
|
| 7 |
DAMON-based LRU-lists Sorting (DAMON_LRU_SORT) is a static kernel module that |
| 8 |
aimed to be used for proactive and lightweight data access pattern based |
| 9 |
(de)prioritization of pages on their LRU-lists for making LRU-lists a more |
| 10 |
trusworthy data access pattern source. |
| 11 |
|
| 12 |
Where Proactive LRU-lists Sorting is Required? |
| 13 |
============================================== |
| 14 |
|
| 15 |
As page-granularity access checking overhead could be significant on huge |
| 16 |
systems, LRU lists are normally not proactively sorted but partially and |
| 17 |
reactively sorted for special events including specific user requests, system |
| 18 |
calls and memory pressure. As a result, LRU lists are sometimes not so |
| 19 |
perfectly prepared to be used as a trustworthy access pattern source for some |
| 20 |
situations including reclamation target pages selection under sudden memory |
| 21 |
pressure. |
| 22 |
|
| 23 |
Because DAMON can identify access patterns of best-effort accuracy while |
| 24 |
inducing only user-specified range of overhead, proactively running |
| 25 |
DAMON_LRU_SORT could be helpful for making LRU lists more trustworthy access |
| 26 |
pattern source with low and controlled overhead. |
| 27 |
|
| 28 |
How It Works? |
| 29 |
============= |
| 30 |
|
| 31 |
DAMON_LRU_SORT finds hot pages (pages of memory regions that showing access |
| 32 |
rates that higher than a user-specified threshold) and cold pages (pages of |
| 33 |
memory regions that showing no access for a time that longer than a |
| 34 |
user-specified threshold) using DAMON, and prioritizes hot pages while |
| 35 |
deprioritizing cold pages on their LRU-lists. To avoid it consuming too much |
| 36 |
CPU for the prioritizations, a CPU time usage limit can be configured. Under |
| 37 |
the limit, it prioritizes and deprioritizes more hot and cold pages first, |
| 38 |
respectively. System administrators can also configure under what situation |
| 39 |
this scheme should automatically activated and deactivated with three memory |
| 40 |
pressure watermarks. |
| 41 |
|
| 42 |
Its default parameters for hotness/coldness thresholds and CPU quota limit are |
| 43 |
conservatively chosen. That is, the module under its default parameters could |
| 44 |
be widely used without harm for common situations while providing a level of |
| 45 |
benefits for systems having clear hot/cold access patterns under memory |
| 46 |
pressure while consuming only a limited small portion of CPU time. |
| 47 |
|
| 48 |
Interface: Module Parameters |
| 49 |
============================ |
| 50 |
|
| 51 |
To use this feature, you should first ensure your system is running on a kernel |
| 52 |
that is built with ``CONFIG_DAMON_LRU_SORT=y``. |
| 53 |
|
| 54 |
To let sysadmins enable or disable it and tune for the given system, |
| 55 |
DAMON_LRU_SORT utilizes module parameters. That is, you can put |
| 56 |
``damon_lru_sort.<parameter>=<value>`` on the kernel boot command line or write |
| 57 |
proper values to ``/sys/module/damon_lru_sort/parameters/<parameter>`` files. |
| 58 |
|
| 59 |
Below are the description of each parameter. |
| 60 |
|
| 61 |
enabled |
| 62 |
------- |
| 63 |
|
| 64 |
Enable or disable DAMON_LRU_SORT. |
| 65 |
|
| 66 |
You can enable DAMON_LRU_SORT by setting the value of this parameter as ``Y``. |
| 67 |
Setting it as ``N`` disables DAMON_LRU_SORT. Note that DAMON_LRU_SORT could do |
| 68 |
no real monitoring and LRU-lists sorting due to the watermarks-based activation |
| 69 |
condition. Refer to below descriptions for the watermarks parameter for this. |
| 70 |
|
| 71 |
commit_inputs |
| 72 |
------------- |
| 73 |
|
| 74 |
Make DAMON_LRU_SORT reads the input parameters again, except ``enabled``. |
| 75 |
|
| 76 |
Input parameters that updated while DAMON_LRU_SORT is running are not applied |
| 77 |
by default. Once this parameter is set as ``Y``, DAMON_LRU_SORT reads values |
| 78 |
of parametrs except ``enabled`` again. Once the re-reading is done, this |
| 79 |
parameter is set as ``N``. If invalid parameters are found while the |
| 80 |
re-reading, DAMON_LRU_SORT will be disabled. |
| 81 |
|
| 82 |
Once ``Y`` is written to this parameter, the user must not write to any |
| 83 |
parameters until reading ``commit_inputs`` again returns ``N``. If users |
| 84 |
violate this rule, the kernel may exhibit undefined behavior. |
| 85 |
|
| 86 |
active_mem_bp |
| 87 |
------------- |
| 88 |
|
| 89 |
Desired active to [in]active memory ratio in bp (1/10,000). |
| 90 |
|
| 91 |
While keeping the caps that set by other quotas, DAMON_LRU_SORT automatically |
| 92 |
increases and decreases the effective level of the quota aiming the LRU |
| 93 |
[de]prioritizations of the hot and cold memory resulting in this active to |
| 94 |
[in]active memory ratio. Value zero means disabling this auto-tuning feature. |
| 95 |
|
| 96 |
Disabled by default. |
| 97 |
|
| 98 |
autotune_monitoring_intervals |
| 99 |
----------------------------- |
| 100 |
|
| 101 |
If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes DAMON's |
| 102 |
sampling and aggregation intervals. The auto-tuning aims to capture meaningful |
| 103 |
amount of access events in each DAMON-snapshot, while keeping the sampling |
| 104 |
interval 5 milliseconds in minimum, and 10 seconds in maximum. Setting this as |
| 105 |
``N`` disables the auto-tuning. |
| 106 |
|
| 107 |
Disabled by default. |
| 108 |
|
| 109 |
filter_young_pages |
| 110 |
------------------ |
| 111 |
|
| 112 |
Filter [non-]young pages accordingly for LRU [de]prioritizations. |
| 113 |
|
| 114 |
If this is set, check page level access (youngness) once again before each |
| 115 |
LRU [de]prioritization operation. LRU prioritization operation is skipped |
| 116 |
if the page has not accessed since the last check (not young). LRU |
| 117 |
deprioritization operation is skipped if the page has accessed since the |
| 118 |
last check (young). The feature is enabled or disabled if this parameter is |
| 119 |
set as ``Y`` or ``N``, respectively. |
| 120 |
|
| 121 |
Disabled by default. |
| 122 |
|
| 123 |
hot_thres_access_freq |
| 124 |
--------------------- |
| 125 |
|
| 126 |
Access frequency threshold for hot memory regions identification in permil. |
| 127 |
|
| 128 |
If a memory region is accessed in frequency of this or higher, DAMON_LRU_SORT |
| 129 |
identifies the region as hot, and mark it as accessed on the LRU list, so that |
| 130 |
it could not be reclaimed under memory pressure. 50% by default. |
| 131 |
|
| 132 |
cold_min_age |
| 133 |
------------ |
| 134 |
|
| 135 |
Time threshold for cold memory regions identification in microseconds. |
| 136 |
|
| 137 |
If a memory region is not accessed for this or longer time, DAMON_LRU_SORT |
| 138 |
identifies the region as cold, and mark it as unaccessed on the LRU list, so |
| 139 |
that it could be reclaimed first under memory pressure. 120 seconds by |
| 140 |
default. |
| 141 |
|
| 142 |
quota_ms |
| 143 |
-------- |
| 144 |
|
| 145 |
Limit of time for trying the LRU lists sorting in milliseconds. |
| 146 |
|
| 147 |
DAMON_LRU_SORT tries to use only up to this time within a time window |
| 148 |
(quota_reset_interval_ms) for trying LRU lists sorting. This can be used |
| 149 |
for limiting CPU consumption of DAMON_LRU_SORT. If the value is zero, the |
| 150 |
limit is disabled. |
| 151 |
|
| 152 |
10 ms by default. |
| 153 |
|
| 154 |
quota_reset_interval_ms |
| 155 |
----------------------- |
| 156 |
|
| 157 |
The time quota charge reset interval in milliseconds. |
| 158 |
|
| 159 |
The charge reset interval for the quota of time (quota_ms). That is, |
| 160 |
DAMON_LRU_SORT does not try LRU-lists sorting for more than quota_ms |
| 161 |
milliseconds or quota_sz bytes within quota_reset_interval_ms milliseconds. |
| 162 |
|
| 163 |
1 second by default. |
| 164 |
|
| 165 |
wmarks_interval |
| 166 |
--------------- |
| 167 |
|
| 168 |
The watermarks check time interval in microseconds. |
| 169 |
|
| 170 |
Minimal time to wait before checking the watermarks, when DAMON_LRU_SORT is |
| 171 |
enabled but inactive due to its watermarks rule. 5 seconds by default. |
| 172 |
|
| 173 |
wmarks_high |
| 174 |
----------- |
| 175 |
|
| 176 |
Free memory rate (per thousand) for the high watermark. |
| 177 |
|
| 178 |
If free memory of the system in bytes per thousand bytes is higher than this, |
| 179 |
DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the |
| 180 |
watermarks. 200 (20%) by default. |
| 181 |
|
| 182 |
wmarks_mid |
| 183 |
---------- |
| 184 |
|
| 185 |
Free memory rate (per thousand) for the middle watermark. |
| 186 |
|
| 187 |
If free memory of the system in bytes per thousand bytes is between this and |
| 188 |
the low watermark, DAMON_LRU_SORT becomes active, so starts the monitoring and |
| 189 |
the LRU-lists sorting. 150 (15%) by default. |
| 190 |
|
| 191 |
wmarks_low |
| 192 |
---------- |
| 193 |
|
| 194 |
Free memory rate (per thousand) for the low watermark. |
| 195 |
|
| 196 |
If free memory of the system in bytes per thousand bytes is lower than this, |
| 197 |
DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the |
| 198 |
watermarks. 50 (5%) by default. |
| 199 |
|
| 200 |
sample_interval |
| 201 |
--------------- |
| 202 |
|
| 203 |
Sampling interval for the monitoring in microseconds. |
| 204 |
|
| 205 |
The sampling interval of DAMON for the cold memory monitoring. Please refer to |
| 206 |
the DAMON documentation (:doc:`usage`) for more detail. 5ms by default. |
| 207 |
|
| 208 |
aggr_interval |
| 209 |
------------- |
| 210 |
|
| 211 |
Aggregation interval for the monitoring in microseconds. |
| 212 |
|
| 213 |
The aggregation interval of DAMON for the cold memory monitoring. Please |
| 214 |
refer to the DAMON documentation (:doc:`usage`) for more detail. 100ms by |
| 215 |
default. |
| 216 |
|
| 217 |
min_nr_regions |
| 218 |
-------------- |
| 219 |
|
| 220 |
Minimum number of monitoring regions. |
| 221 |
|
| 222 |
The minimal number of monitoring regions of DAMON for the cold memory |
| 223 |
monitoring. This can be used to set lower-bound of the monitoring quality. |
| 224 |
But, setting this too high could result in increased monitoring overhead. |
| 225 |
Please refer to the DAMON documentation (:doc:`usage`) for more detail. 10 by |
| 226 |
default. |
| 227 |
|
| 228 |
max_nr_regions |
| 229 |
-------------- |
| 230 |
|
| 231 |
Maximum number of monitoring regions. |
| 232 |
|
| 233 |
The maximum number of monitoring regions of DAMON for the cold memory |
| 234 |
monitoring. This can be used to set upper-bound of the monitoring overhead. |
| 235 |
However, setting this too low could result in bad monitoring quality. Please |
| 236 |
refer to the DAMON documentation (:doc:`usage`) for more detail. 1000 by |
| 237 |
defaults. |
| 238 |
|
| 239 |
monitor_region_start |
| 240 |
-------------------- |
| 241 |
|
| 242 |
Start of target memory region in physical address. |
| 243 |
|
| 244 |
The start physical address of memory region that DAMON_LRU_SORT will do work |
| 245 |
against. By default, biggest System RAM is used as the region. |
| 246 |
|
| 247 |
monitor_region_end |
| 248 |
------------------ |
| 249 |
|
| 250 |
End of target memory region in physical address. |
| 251 |
|
| 252 |
The end physical address of memory region that DAMON_LRU_SORT will do work |
| 253 |
against. By default, biggest System RAM is used as the region. |
| 254 |
|
| 255 |
addr_unit |
| 256 |
--------- |
| 257 |
|
| 258 |
A scale factor for memory addresses and bytes. |
| 259 |
|
| 260 |
This parameter is for setting and getting the :ref:`address unit |
| 261 |
<damon_design_addr_unit>` parameter of the DAMON instance for DAMON_RECLAIM. |
| 262 |
|
| 263 |
``monitor_region_start`` and ``monitor_region_end`` should be provided in this |
| 264 |
unit. For example, let's suppose ``addr_unit``, ``monitor_region_start`` and |
| 265 |
``monitor_region_end`` are set as ``1024``, ``0`` and ``10``, respectively. |
| 266 |
Then DAMON_LRU_SORT will work for 10 KiB length of physical address range that |
| 267 |
starts from address zero (``[0 * 1024, 10 * 1024)`` in bytes). |
| 268 |
|
| 269 |
Stat parameters having ``bytes_`` prefix are also in this unit. For example, |
| 270 |
let's suppose values of ``addr_unit``, ``bytes_lru_sort_tried_hot_regions`` and |
| 271 |
``bytes_lru_sorted_hot_regions`` are ``1024``, ``42``, and ``32``, |
| 272 |
respectively. Then it means DAMON_LRU_SORT tried to LRU-sort 42 KiB of hot |
| 273 |
memory and successfully LRU-sorted 32 KiB of the memory in total. |
| 274 |
|
| 275 |
If unsure, use only the default value (``1``) and forget about this. |
| 276 |
|
| 277 |
kdamond_pid |
| 278 |
----------- |
| 279 |
|
| 280 |
PID of the DAMON thread. |
| 281 |
|
| 282 |
If DAMON_LRU_SORT is enabled, this becomes the PID of the worker thread. Else, |
| 283 |
-1. |
| 284 |
|
| 285 |
nr_lru_sort_tried_hot_regions |
| 286 |
----------------------------- |
| 287 |
|
| 288 |
Number of hot memory regions that tried to be LRU-sorted. |
| 289 |
|
| 290 |
bytes_lru_sort_tried_hot_regions |
| 291 |
-------------------------------- |
| 292 |
|
| 293 |
Total bytes of hot memory regions that tried to be LRU-sorted. |
| 294 |
|
| 295 |
nr_lru_sorted_hot_regions |
| 296 |
------------------------- |
| 297 |
|
| 298 |
Number of hot memory regions that successfully be LRU-sorted. |
| 299 |
|
| 300 |
bytes_lru_sorted_hot_regions |
| 301 |
---------------------------- |
| 302 |
|
| 303 |
Total bytes of hot memory regions that successfully be LRU-sorted. |
| 304 |
|
| 305 |
nr_hot_quota_exceeds |
| 306 |
-------------------- |
| 307 |
|
| 308 |
Number of times that the time quota limit for hot regions have exceeded. |
| 309 |
|
| 310 |
nr_lru_sort_tried_cold_regions |
| 311 |
------------------------------ |
| 312 |
|
| 313 |
Number of cold memory regions that tried to be LRU-sorted. |
| 314 |
|
| 315 |
bytes_lru_sort_tried_cold_regions |
| 316 |
--------------------------------- |
| 317 |
|
| 318 |
Total bytes of cold memory regions that tried to be LRU-sorted. |
| 319 |
|
| 320 |
nr_lru_sorted_cold_regions |
| 321 |
-------------------------- |
| 322 |
|
| 323 |
Number of cold memory regions that successfully be LRU-sorted. |
| 324 |
|
| 325 |
bytes_lru_sorted_cold_regions |
| 326 |
----------------------------- |
| 327 |
|
| 328 |
Total bytes of cold memory regions that successfully be LRU-sorted. |
| 329 |
|
| 330 |
nr_cold_quota_exceeds |
| 331 |
--------------------- |
| 332 |
|
| 333 |
Number of times that the time quota limit for cold regions have exceeded. |
| 334 |
|
| 335 |
Example |
| 336 |
======= |
| 337 |
|
| 338 |
Below runtime example commands make DAMON_LRU_SORT to find memory regions |
| 339 |
having >=50% access frequency and LRU-prioritize while LRU-deprioritizing |
| 340 |
memory regions that not accessed for 120 seconds. The prioritization and |
| 341 |
deprioritization is limited to be done using only up to 1% CPU time to avoid |
| 342 |
DAMON_LRU_SORT consuming too much CPU time for the (de)prioritization. It also |
| 343 |
asks DAMON_LRU_SORT to do nothing if the system's free memory rate is more than |
| 344 |
50%, but start the real works if it becomes lower than 40%. If DAMON_RECLAIM |
| 345 |
doesn't make progress and therefore the free memory rate becomes lower than |
| 346 |
20%, it asks DAMON_LRU_SORT to do nothing again, so that we can fall back to |
| 347 |
the LRU-list based page granularity reclamation. :: |
| 348 |
|
| 349 |
# cd /sys/module/damon_lru_sort/parameters |
| 350 |
# echo 500 > hot_thres_access_freq |
| 351 |
# echo 120000000 > cold_min_age |
| 352 |
# echo 10 > quota_ms |
| 353 |
# echo 1000 > quota_reset_interval_ms |
| 354 |
# echo 500 > wmarks_high |
| 355 |
# echo 400 > wmarks_mid |
| 356 |
# echo 200 > wmarks_low |
| 357 |
# echo Y > enabled |