Debian Codegraph @

Documentation/admin-guide/mm/damon/usage.rst

raw
1 .. SPDX-License-Identifier: GPL-2.0
2
3 ===============
4 Detailed Usages
5 ===============
6
7 DAMON provides below interfaces for different users.
8
9 - *Special-purpose DAMON modules.*
10 :ref:`This <damon_modules_special_purpose>` is for people who are building,
11 distributing, and/or administrating the kernel with special-purpose DAMON
12 usages. Using this, users can use DAMON's major features for the given
13 purposes in build, boot, or runtime in simple ways.
14 - *DAMON user space tool.*
15 `This <https://github.com/damonitor/damo>`_ is for privileged people such as
16 system administrators who want a just-working human-friendly interface.
17 Using this, users can use the DAMON’s major features in a human-friendly way.
18 It may not be highly tuned for special cases, though. For more detail,
19 please refer to its `usage document
20 <https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
21 - *sysfs interface.*
22 :ref:`This <sysfs_interface>` is for privileged user space programmers who
23 want more optimized use of DAMON. Using this, users can use DAMON’s major
24 features by reading from and writing to special sysfs files. Therefore,
25 you can write and use your personalized DAMON sysfs wrapper programs that
26 reads/writes the sysfs files instead of you. The `DAMON user space tool
27 <https://github.com/damonitor/damo>`_ is one example of such programs.
28 - *Kernel Space Programming Interface.*
29 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this,
30 users can utilize every feature of DAMON most flexibly and efficiently by
31 writing kernel space DAMON application programs for you. You can even extend
32 DAMON for various address spaces. For detail, please refer to the interface
33 :doc:`document </mm/damon/api>`.
34
35 .. _sysfs_interface:
36
37 sysfs Interface
38 ===============
39
40 DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It
41 creates multiple directories and files under its sysfs directory,
42 ``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading
43 from the files under the directory.
44
45 For a short example, users can monitor the virtual address space of a given
46 workload as below. ::
47
48 # cd /sys/kernel/mm/damon/admin/
49 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
50 # echo vaddr > kdamonds/0/contexts/0/operations
51 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
52 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
53 # echo on > kdamonds/0/state
54
55 Files Hierarchy
56 ---------------
57
58 The files hierarchy of DAMON sysfs interface is shown below. In the below
59 figure, parents-children relations are represented with indentations, each
60 directory is having ``/`` suffix, and files in each directory are separated by
61 comma (",").
62
63 .. parsed-literal::
64
65 :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
66 │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
67 │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
68 │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
69 │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit
70 │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
71 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
72 │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us
73 │ │ │ │ │ │ nr_regions/min,max
74 │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets
75 │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target
76 │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions
77 │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end
78 │ │ │ │ │ │ │ │ ...
79 │ │ │ │ │ │ ...
80 │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes
81 │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us
82 │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/
83 │ │ │ │ │ │ │ │ sz/min,max
84 │ │ │ │ │ │ │ │ nr_accesses/min,max
85 │ │ │ │ │ │ │ │ age/min,max
86 │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,effective_bytes
87 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
88 │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals
89 │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path
90 │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
91 │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
92 │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,target_idx,min,max
93 │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
94 │ │ │ │ │ │ │ │ 0/id,weight
95 │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots
96 │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
97 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
98 │ │ │ │ │ │ │ │ ...
99 │ │ │ │ │ │ ...
100 │ │ │ │ ...
101 │ │ ...
102
103 .. _sysfs_root:
104
105 Root
106 ----
107
108 The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
109 has one directory named ``admin``. The directory contains the files for
110 privileged user space programs' control of DAMON. User space tools or daemons
111 having the root permission could use this directory.
112
113 .. _sysfs_kdamonds:
114
115 kdamonds/
116 ---------
117
118 Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
119 controlling the kdamonds (refer to
120 :ref:`design <damon_design_execution_model_and_data_structures>` for more
121 details) exists. In the beginning, this directory has only one file,
122 ``nr_kdamonds``. Writing a number (``N``) to the file creates the number of
123 child directories named ``0`` to ``N-1``. Each directory represents each
124 kdamond.
125
126 .. _sysfs_kdamond:
127
128 kdamonds/<N>/
129 -------------
130
131 In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
132 and one directory (``contexts``) exist.
133
134 Reading ``state`` returns ``on`` if the kdamond is currently running, or
135 ``off`` if it is not running.
136
137 Users can write below commands for the kdamond to the ``state`` file.
138
139 - ``on``: Start running.
140 - ``off``: Stop running.
141 - ``commit``: Read the user inputs in the sysfs files except ``state`` file
142 again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be
143 ignored if no target region is specified.
144 - ``update_tuned_intervals``: Update the contents of ``sample_us`` and
145 ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling
146 interval`` and ``aggregation interval`` for the files. Please refer to
147 :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>`
148 for more details.
149 - ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes'
150 :ref:`quota goals <sysfs_schemes_quota_goals>`.
151 - ``update_schemes_stats``: Update the contents of stats files for each
152 DAMON-based operation scheme of the kdamond. For details of the stats,
153 please refer to :ref:`stats section <sysfs_schemes_stats>`.
154 - ``update_schemes_tried_regions``: Update the DAMON-based operation scheme
155 action tried regions directory for each DAMON-based operation scheme of the
156 kdamond. For details of the DAMON-based operation scheme action tried
157 regions directory, please refer to
158 :ref:`tried_regions section <sysfs_schemes_tried_regions>`.
159 - ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes``
160 files.
161 - ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
162 action tried regions directory for each DAMON-based operation scheme of the
163 kdamond.
164 - ``update_schemes_effective_quotas``: Update the contents of
165 ``effective_bytes`` files for each DAMON-based operation scheme of the
166 kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`.
167
168 If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
169
170 Users can ask the kernel to periodically update files showing auto-tuned
171 parameters and DAMOS stats instead of manually writing
172 ``update_tuned_intervals`` like keywords to ``state`` file. For this, users
173 should write the desired update time interval in milliseconds to ``refresh_ms``
174 file. If the interval is zero, the periodic update is disabled. Reading the
175 file shows currently set time interval.
176
177 ``contexts`` directory contains files for controlling the monitoring contexts
178 that this kdamond will execute.
179
180 .. _sysfs_contexts:
181
182 kdamonds/<N>/contexts/
183 ----------------------
184
185 In the beginning, this directory has only one file, ``nr_contexts``. Writing a
186 number (``N``) to the file creates the number of child directories named as
187 ``0`` to ``N-1``. Each directory represents each monitoring context (refer to
188 :ref:`design <damon_design_execution_model_and_data_structures>` for more
189 details). At the moment, only one context per kdamond is supported, so only
190 ``0`` or ``1`` can be written to the file.
191
192 .. _sysfs_context:
193
194 contexts/<N>/
195 -------------
196
197 In each context directory, three files (``avail_operations``, ``operations``
198 and ``addr_unit``) and three directories (``monitoring_attrs``, ``targets``,
199 and ``schemes``) exist.
200
201 DAMON supports multiple types of :ref:`monitoring operations
202 <damon_design_configurable_operations_set>`, including those for virtual address
203 space and the physical address space. You can get the list of available
204 monitoring operations set on the currently running kernel by reading
205 ``avail_operations`` file. Based on the kernel configuration, the file will
206 list different available operation sets. Please refer to the :ref:`design
207 <damon_operations_set>` for the list of all available operation sets and their
208 brief explanations.
209
210 You can set and get what type of monitoring operations DAMON will use for the
211 context by writing one of the keywords listed in ``avail_operations`` file and
212 reading from the ``operations`` file.
213
214 ``addr_unit`` file is for setting and getting the :ref:`address unit
215 <damon_design_addr_unit>` parameter of the operations set.
216
217 .. _sysfs_monitoring_attrs:
218
219 contexts/<N>/monitoring_attrs/
220 ------------------------------
221
222 Files for specifying attributes of the monitoring including required quality
223 and efficiency of the monitoring are in ``monitoring_attrs`` directory.
224 Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
225 directory.
226
227 Under ``intervals`` directory, three files for DAMON's sampling interval
228 (``sample_us``), aggregation interval (``aggr_us``), and update interval
229 (``update_us``) exist. You can set and get the values in micro-seconds by
230 writing to and reading from the files.
231
232 Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
233 of DAMON's monitoring regions (``min`` and ``max``, respectively), which
234 controls the monitoring overhead, exist. You can set and get the values by
235 writing to and rading from the files.
236
237 For more details about the intervals and monitoring regions range, please refer
238 to the Design document (:doc:`/mm/damon/design`).
239
240 .. _damon_usage_sysfs_monitoring_intervals_goal:
241
242 contexts/<N>/monitoring_attrs/intervals/intervals_goal/
243 -------------------------------------------------------
244
245 Under the ``intervals`` directory, one directory for automated tuning of
246 ``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists.
247 Under the directory, four files for the auto-tuning control, namely
248 ``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist.
249 Please refer to the :ref:`design document of the feature
250 <damon_design_monitoring_intervals_autotuning>` for the internal of the tuning
251 mechanism. Reading and writing the four files under ``intervals_goal``
252 directory shows and updates the tuning parameters that described in the
253 :ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same
254 names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The
255 tuning-applied current values of the two intervals can be read from the
256 ``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to
257 the ``state`` file.
258
259 .. _sysfs_targets:
260
261 contexts/<N>/targets/
262 ---------------------
263
264 In the beginning, this directory has only one file, ``nr_targets``. Writing a
265 number (``N``) to the file creates the number of child directories named ``0``
266 to ``N-1``. Each directory represents each monitoring target.
267
268 .. _sysfs_target:
269
270 targets/<N>/
271 ------------
272
273 In each target directory, two files (``pid_target`` and ``obsolete_target``)
274 and one directory (``regions``) exist.
275
276 If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
277 be a process. You can specify the process to DAMON by writing the pid of the
278 process to the ``pid_target`` file.
279
280 Users can selectively remove targets in the middle of the targets array by
281 writing non-zero value to ``obsolete_target`` file and committing it (writing
282 ``commit`` to ``state`` file). DAMON will remove the matching targets from its
283 internal targets array. Users are responsible to construct target directories
284 again, so that those correctly represent the changed internal targets array.
285
286
287 .. _sysfs_regions:
288
289 targets/<N>/regions
290 -------------------
291
292 In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are
293 required to set the monitoring target address ranges. In case of ``vaddr``
294 operations set, it is not mandatory, but users can optionally set the initial
295 monitoring region to specific address ranges. Please refer to the :ref:`design
296 <damon_design_vaddr_target_regions_construction>` for more details.
297
298 For such cases, users can explicitly set the initial monitoring target regions
299 as they want, by writing proper values to the files under this directory.
300
301 In the beginning, this directory has only one file, ``nr_regions``. Writing a
302 number (``N``) to the file creates the number of child directories named ``0``
303 to ``N-1``. Each directory represents each initial monitoring target region.
304
305 If ``nr_regions`` is zero when committing new DAMON parameters online (writing
306 ``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit
307 logic ignores the target regions. In other words, the current monitoring
308 results for the target are preserved.
309
310 .. _sysfs_region:
311
312 regions/<N>/
313 ------------
314
315 In each region directory, you will find two files (``start`` and ``end``). You
316 can set and get the start and end addresses of the initial monitoring target
317 region by writing to and reading from the files, respectively.
318
319 Each region should not overlap with others. ``end`` of directory ``N`` should
320 be equal or smaller than ``start`` of directory ``N+1``.
321
322 .. _sysfs_schemes:
323
324 contexts/<N>/schemes/
325 ---------------------
326
327 The directory for DAMON-based Operation Schemes (:ref:`DAMOS
328 <damon_design_damos>`). Users can get and set the schemes by reading from and
329 writing to files under this directory.
330
331 In the beginning, this directory has only one file, ``nr_schemes``. Writing a
332 number (``N``) to the file creates the number of child directories named ``0``
333 to ``N-1``. Each directory represents each DAMON-based operation scheme.
334
335 .. _sysfs_scheme:
336
337 schemes/<N>/
338 ------------
339
340 In each scheme directory, eight directories (``access_pattern``, ``quotas``,
341 ``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
342 ``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
343 and ``apply_interval``) exist.
344
345 The ``action`` file is for setting and getting the scheme's :ref:`action
346 <damon_design_damos_action>`. The keywords that can be written to and read
347 from the file and their meaning are same to those of the list on
348 :ref:`design doc <damon_design_damos_action>`.
349
350 The ``target_nid`` file is for setting the migration target node, which is
351 only meaningful when the ``action`` is either ``migrate_hot`` or
352 ``migrate_cold``.
353
354 The ``apply_interval_us`` file is for setting and getting the scheme's
355 :ref:`apply_interval <damon_design_damos>` in microseconds.
356
357 .. _sysfs_access_pattern:
358
359 schemes/<N>/access_pattern/
360 ---------------------------
361
362 The directory for the target access :ref:`pattern
363 <damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
364
365 Under the ``access_pattern`` directory, three directories (``sz``,
366 ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
367 exist. You can set and get the access pattern for the given scheme by writing
368 to and reading from the ``min`` and ``max`` files under ``sz``,
369 ``nr_accesses``, and ``age`` directories, respectively. Note that the ``min``
370 and the ``max`` form a closed interval.
371
372 .. _sysfs_quotas:
373
374 schemes/<N>/quotas/
375 -------------------
376
377 The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
378 DAMON-based operation scheme.
379
380 Under ``quotas`` directory, four files (``ms``, ``bytes``,
381 ``reset_interval_ms``, ``effective_bytes``) and two directories (``weights`` and
382 ``goals``) exist.
383
384 You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
385 ``reset interval`` in milliseconds by writing the values to the three files,
386 respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds
387 for applying the ``action`` to memory regions of the ``access_pattern``, and to
388 apply the action to only up to ``bytes`` bytes of memory regions within the
389 ``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the
390 quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is
391 set.
392
393 The time quota is internally transformed to a size quota. Between the
394 transformed size quota and user-specified size quota, smaller one is applied.
395 Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the
396 effective size quota is further adjusted. Reading ``effective_bytes`` returns
397 the current effective size quota. The file is not updated in real time, so
398 users should ask DAMON sysfs interface to update the content of the file for
399 the stats by writing a special keyword, ``update_schemes_effective_quotas`` to
400 the relevant ``kdamonds/<N>/state`` file.
401
402 Under ``weights`` directory, three files (``sz_permil``,
403 ``nr_accesses_permil``, and ``age_permil``) exist.
404 You can set the :ref:`prioritization weights
405 <damon_design_damos_quotas_prioritization>` for size, access frequency, and age
406 in per-thousand unit by writing the values to the three files under the
407 ``weights`` directory.
408
409 .. _sysfs_schemes_quota_goals:
410
411 schemes/<N>/quotas/goals/
412 -------------------------
413
414 The directory for the :ref:`automatic quota tuning goals
415 <damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation
416 scheme.
417
418 In the beginning, this directory has only one file, ``nr_goals``. Writing a
419 number (``N``) to the file creates the number of child directories named ``0``
420 to ``N-1``. Each directory represents each goal and current achievement.
421 Among the multiple feedback, the best one is used.
422
423 Each goal directory contains five files, namely ``target_metric``,
424 ``target_value``, ``current_value`` ``nid`` and ``path``. Users can set and
425 get the five parameters for the quota auto-tuning goals that specified on the
426 :ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and
427 reading from each of the files. Note that users should further write
428 ``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond
429 directory <sysfs_kdamond>` to pass the feedback to DAMON.
430
431 .. _sysfs_watermarks:
432
433 schemes/<N>/watermarks/
434 -----------------------
435
436 The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
437 given DAMON-based operation scheme.
438
439 Under the watermarks directory, five files (``metric``, ``interval_us``,
440 ``high``, ``mid``, and ``low``) for setting the metric, the time interval
441 between check of the metric, and the three watermarks exist. You can set and
442 get the five values by writing to the files, respectively.
443
444 Keywords and meanings of those that can be written to the ``metric`` file are
445 as below.
446
447 - none: Ignore the watermarks
448 - free_mem_rate: System's free memory rate (per thousand)
449
450 The ``interval`` should written in microseconds unit.
451
452 .. _sysfs_filters:
453
454 schemes/<N>/{core\_,ops\_,}filters/
455 -----------------------------------
456
457 Directories for :ref:`filters <damon_design_damos_filters>` of the given
458 DAMON-based operation scheme.
459
460 ``core_filters`` and ``ops_filters`` directories are for the filters handled by
461 the DAMON core layer and operations set layer, respectively. ``filters``
462 directory can be used for installing filters regardless of their handled
463 layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be
464 installed before those of ``filters``. All three directories have same files.
465
466 Use of ``filters`` directory can make expecting evaluation orders of given
467 filters with the files under directory bit confusing. Users are hence
468 recommended to use ``core_filters`` and ``ops_filters`` directories. The
469 ``filters`` directory could be deprecated in future.
470
471 In the beginning, the directory has only one file, ``nr_filters``. Writing a
472 number (``N``) to the file creates the number of child directories named ``0``
473 to ``N-1``. Each directory represents each filter. The filters are evaluated
474 in the numeric order.
475
476 Each filter directory contains nine files, namely ``type``, ``matching``,
477 ``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max``
478 and ``target_idx``. To ``type`` file, you can write the type of the filter.
479 Refer to :ref:`the design doc <damon_design_damos_filters>` for available type
480 names, their meaning and on what layer those are handled.
481
482 For ``memcg`` type, you can specify the memory cgroup of the interest by
483 writing the path of the memory cgroup from the cgroups mount point to
484 ``memcg_path`` file. For ``addr`` type, you can specify the start and end
485 address of the range (open-ended interval) to ``addr_start`` and ``addr_end``
486 files, respectively. For ``hugepage_size`` type, you can specify the minimum
487 and maximum size of the range (closed interval) to ``min`` and ``max`` files,
488 respectively. For ``target`` type, you can specify the index of the target
489 between the list of the DAMON context's monitoring targets list to
490 ``target_idx`` file.
491
492 You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter
493 is for memory that matches the ``type``. You can write ``Y`` or ``N`` to
494 ``allow`` file to specify if applying the action to the memory that satisfies
495 the ``type`` and ``matching`` should be allowed or not.
496
497 For example, below restricts a DAMOS action to be applied to only non-anonymous
498 pages of all memory cgroups except ``/having_care_already``.::
499
500 # cd ops_filters/0/
501 # echo 2 > nr_filters
502 # # disallow anonymous pages
503 echo anon > 0/type
504 echo Y > 0/matching
505 echo N > 0/allow
506 # # further filter out all cgroups except one at '/having_care_already'
507 echo memcg > 1/type
508 echo /having_care_already > 1/memcg_path
509 echo Y > 1/matching
510 echo N > 1/allow
511
512 Refer to the :ref:`DAMOS filters design documentation
513 <damon_design_damos_filters>` for more details including how multiple filters
514 of different ``allow`` works, when each of the filters are supported, and
515 differences on stats.
516
517 .. _damon_sysfs_dests:
518
519 schemes/<N>/dests/
520 ------------------
521
522 Directory for specifying the destinations of given DAMON-based operation
523 scheme's action. This directory is ignored if the action of the given scheme
524 is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}``
525 actions are supporting multiple destinations.
526
527 In the beginning, the directory has only one file, ``nr_dests``. Writing a
528 number (``N``) to the file creates the number of child directories named ``0``
529 to ``N-1``. Each directory represents each action destination.
530
531 Each destination directory contains two files, namely ``id`` and ``weight``.
532 Users can write and read the identifier of the destination to ``id`` file.
533 For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node
534 id should be written to ``id`` file. Users can write and read the weight of
535 the destination among the given destinations to the ``weight`` file. The
536 weight can be an arbitrary integer. When DAMOS apply the action to each entity
537 of the memory region, it will select the destination of the action based on the
538 relative weights of the destinations.
539
540 .. _sysfs_schemes_stats:
541
542 schemes/<N>/stats/
543 ------------------
544
545 DAMON counts statistics for each scheme. This statistics can be used for
546 online analysis or tuning of the schemes. Refer to :ref:`design doc
547 <damon_design_damos_stat>` for more details about the stats.
548
549 The statistics can be retrieved by reading the files under ``stats`` directory
550 (``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``,
551 ``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and
552 ``max_nr_snapshots``), respectively.
553
554 The files are not updated in real time by default. Users should ask DAMON
555 sysfs interface to periodically update those using ``refresh_ms``, or do a one
556 time update by writing a special keyword, ``update_schemes_stats`` to the
557 relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory
558 <sysfs_kdamond>` for more details.
559
560 .. _sysfs_schemes_tried_regions:
561
562 schemes/<N>/tried_regions/
563 --------------------------
564
565 This directory initially has one file, ``total_bytes``.
566
567 When a special keyword, ``update_schemes_tried_regions``, is written to the
568 relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
569 that reading it returns the total size of the scheme tried regions, and creates
570 directories named integer starting from ``0`` under this directory. Each
571 directory contains files exposing detailed information about each of the memory
572 region that the corresponding scheme's ``action`` has tried to be applied under
573 this directory, during next :ref:`apply interval <damon_design_damos>` of the
574 corresponding scheme. The information includes address range, ``nr_accesses``,
575 and ``age`` of the region.
576
577 Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
578 file will only update the ``total_bytes`` file, and will not create the
579 subdirectories.
580
581 The directories will be removed when another special keyword,
582 ``clear_schemes_tried_regions``, is written to the relevant
583 ``kdamonds/<N>/state`` file.
584
585 The expected usage of this directory is investigations of schemes' behaviors,
586 and query-like efficient data access monitoring results retrievals. For the
587 latter use case, in particular, users can set the ``action`` as ``stat`` and
588 set the ``access pattern`` as their interested pattern that they want to query.
589
590 .. _sysfs_schemes_tried_region:
591
592 tried_regions/<N>/
593 ------------------
594
595 In each region directory, you will find five files (``start``, ``end``,
596 ``nr_accesses``, ``age``, and ``sz_filter_passed``). Reading the files will
597 show the properties of the region that corresponding DAMON-based operation
598 scheme ``action`` has tried to be applied.
599
600 Example
601 ~~~~~~~
602
603 Below commands applies a scheme saying "If a memory region of size in [4KiB,
604 8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
605 interval in [10, 20], page out the region. For the paging out, use only up to
606 10ms per second, and also don't page out more than 1GiB per second. Under the
607 limitation, page out memory regions having longer age first. Also, check the
608 free memory rate of the system every 5 seconds, start the monitoring and paging
609 out when the free memory rate becomes lower than 50%, but stop it if the free
610 memory rate becomes larger than 60%, or lower than 30%". ::
611
612 # cd <sysfs>/kernel/mm/damon/admin
613 # # populate directories
614 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
615 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
616 # cd kdamonds/0/contexts/0/schemes/0
617 # # set the basic access pattern and the action
618 # echo 4096 > access_pattern/sz/min
619 # echo 8192 > access_pattern/sz/max
620 # echo 0 > access_pattern/nr_accesses/min
621 # echo 5 > access_pattern/nr_accesses/max
622 # echo 10 > access_pattern/age/min
623 # echo 20 > access_pattern/age/max
624 # echo pageout > action
625 # # set quotas
626 # echo 10 > quotas/ms
627 # echo $((1024*1024*1024)) > quotas/bytes
628 # echo 1000 > quotas/reset_interval_ms
629 # # set watermark
630 # echo free_mem_rate > watermarks/metric
631 # echo 5000000 > watermarks/interval_us
632 # echo 600 > watermarks/high
633 # echo 500 > watermarks/mid
634 # echo 300 > watermarks/low
635
636 Please note that it's highly recommended to use user space tools like `damo
637 <https://github.com/damonitor/damo>`_ rather than manually reading and writing
638 the files as above. Above is only for an example.
639
640 .. _tracepoint:
641
642 Tracepoints for Monitoring Results
643 ==================================
644
645 Users can get the monitoring results via the :ref:`tried_regions
646 <sysfs_schemes_tried_regions>`. The interface is useful for getting a
647 snapshot, but it could be inefficient for fully recording all the monitoring
648 results. For the purpose, two trace points, namely ``damon:damon_aggregated``
649 and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated``
650 provides the whole monitoring results, while ``damon:damos_before_apply``
651 provides the monitoring results for regions that each DAMON-based Operation
652 Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence,
653 ``damon:damos_before_apply`` is more useful for recording internal behavior of
654 DAMOS, or DAMOS target access
655 :ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient
656 monitoring results recording.
657
658 While the monitoring is turned on, you could record the tracepoint events and
659 show results using tracepoint supporting tools like ``perf``. For example::
660
661 # echo on > kdamonds/0/state
662 # perf record -e damon:damon_aggregated &
663 # sleep 5
664 # kill 9 $(pidof perf)
665 # echo off > kdamonds/0/state
666 # perf script
667 kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864
668 [...]
669
670 Each line of the perf script output represents each monitoring region. The
671 first five fields are as usual other tracepoint outputs. The sixth field
672 (``target_id=X``) shows the ide of the monitoring target of the region. The
673 seventh field (``nr_regions=X``) shows the total number of monitoring regions
674 for the target. The eighth field (``X-Y:``) shows the start (``X``) and end
675 (``Y``) addresses of the region in bytes. The ninth field (``X``) shows the
676 ``nr_accesses`` of the region (refer to
677 :ref:`design <damon_design_region_based_sampling>` for more details of the
678 counter). Finally the tenth field (``X``) shows the ``age`` of the region
679 (refer to :ref:`design <damon_design_age_tracking>` for more details of the
680 counter).
681
682 If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would
683 be somewhat like below::
684
685 kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136
686 [...]
687
688 Each line of the output represents each monitoring region that each DAMON-based
689 Operation Scheme was about to be applied at the traced time. The first five
690 fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``)
691 of the scheme in the list of the contexts of the context's kdamond, the index
692 of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in
693 addition to the output of ``damon_aggregated`` tracepoint.