Debian Codegraph @

inst/doc/parallelly-20-limit-workers.html

raw
1 <!DOCTYPE html>
2 <html lang="en">
3 <head>
4 <meta charset="UTF-8">
5 <title>Parallel Workers with CPU and Memory Limited</title>
6 <style>
7 body {
8 font-family: sans-serif;
9 line-height: 1.6;
10 padding-left: 3ex;
11 padding-right: 3ex;
12 background-color: white;
13 color: black;
14 }
15
16 a {
17 color: #4183C4;
18 text-decoration: none;
19 }
20
21 h1, h2, h3 {
22 margin: 2ex 0 1ex;
23 padding: 0;
24 font-weight: bold;
25 -webkit-font-smoothing: antialiased;
26 cursor: text;
27 position: relative;
28 }
29
30 h2 {
31 border-bottom: 1px solid #cccccc;
32 }
33
34 code {
35 margin: 0 2px;
36 padding: 0 5px;
37 white-space: nowrap;
38 border: 1px solid #eaeaea;
39 background-color: #f8f8f8;
40 border-radius: 3px;
41 }
42
43 pre code {
44 margin: 0;
45 padding: 0;
46 white-space: pre;
47 border: none;
48 background: transparent;
49 }
50
51 pre {
52 background-color: #f8f8f8;
53 border: 1px solid #cccccc;
54 line-height: 2.5x;
55 overflow: auto;
56 padding: 0.6ex 1ex;
57 border-radius: 3px;
58 }
59
60 pre code {
61 background-color: transparent;
62 border: none;
63 }
64 </style>
65 </head>
66 <body>
67 <h1>Parallel Workers with CPU and Memory Limited</h1>
68 <!--
69 %\VignetteIndexEntry{Parallel Workers with CPU and Memory Limited}
70 %\VignetteAuthor{Henrik Bengtsson}
71 %\VignetteKeyword{R}
72 %\VignetteKeyword{package}
73 %\VignetteKeyword{vignette}
74 %\VignetteEngine{parallelly::selfonly}
75 -->
76 <h1>Introduction</h1>
77 <p>This vignette gives examples of how to restrict CPU and memory usage of
78 parallel workers. This can be useful for optimizing the performance of
79 the parallel workers, but also lower the risk that they overuse the
80 CPU and memory on the machines they are running on.</p>
81 <h1>Examples</h1>
82 <h2>Example: Linux parallel workers with a lower process priority (“nice”)</h2>
83 <p>On Unix, we can run any process with a lower CPU priority using the
84 <code>nice</code> command. This can be used when we want to lower the risk of
85 negatively affecting other users and processes that run on the same
86 machine due to our R workers overusing the CPUs by mistake. To achieve
87 this, we can prepend <code>nice</code> to the <code>Rscript</code> call via the <code>rscript</code>
88 argument. This works both on local and remote Linux machines,
89 e.g.</p>
90 <pre><code class="language-r">library(parallelly)
91 cl &lt;- makeClusterPSOCK(2, rscript = c(&quot;nice&quot;, &quot;*&quot;))
92 </code></pre>
93 <pre><code class="language-r">library(parallelly)
94 workers &lt;- rep(&quot;n1.remote.org&quot;, 2)
95 cl &lt;- makeClusterPSOCK(2, rscript = c(&quot;nice&quot;, &quot;*&quot;))
96 </code></pre>
97 <p>The special <code>*</code> value expands to the proper <code>Rscript</code> on the machine
98 where the parallel workers are launched.</p>
99 <h2>Example: Linux parallel workers CPU and memory limited by CGroups</h2>
100 <p>This example launches two parallel workers each limited to 100% CPU
101 quota and 50 MiB of memory using Linux CGroups management. The 100%
102 CPU quota limit constrains each worker to use at most one CPU worth of
103 processing preventing them from overusing the machine, e.g. through
104 unintended nested parallelization. For more details, see <code>man systemd.resource-control</code>.</p>
105 <pre><code class="language-r">library(parallelly)
106 cl &lt;- makeClusterPSOCK(
107 2L,
108 rscript = c(
109 &quot;systemd-run&quot;, &quot;--user&quot;, &quot;--scope&quot;,
110 &quot;-p&quot;, &quot;CPUQuota=100%&quot;,
111 &quot;-p&quot;, &quot;MemoryMax=50M&quot;, &quot;-p&quot;, &quot;MemorySwapMax=50M&quot;,
112 &quot;*&quot;
113 )
114 )
115 </code></pre>
116 <p>Note, depending on your CGroups configuration, a non-privileged user
117 may or may not be able to set the CPU quota. If not, the <code>-p CPUQuota=100%</code> will be silently ignored.</p>
118 <p>The 50 MiB memory limit is strict - if a worker uses more than this,
119 the operating system will terminate the worker instantly. To
120 illustrate what happens, we first start by generating 1 million
121 numeric values each consuming 8 bytes, which in total consumes ~8 MB,
122 and then calculate the mean, the memory consumption is within 50-MiB
123 memory limit that each parallel worker has available;</p>
124 <pre><code class="language-r">library(parallel)
125 mu &lt;- clusterEvalQ(cl, { x &lt;- rnorm(n = 1e6); mean(x) })
126 mu &lt;- unlist(mu)
127 print(mu)
128 #&gt; [1] 0.0008072657 -0.0019693992
129 </code></pre>
130 <p>However, if we generate 10 times more values, the memory consumption
131 will grow to at least 80 MB, which is over the 50-MiB memory limit,
132 and we will get an error:</p>
133 <pre><code class="language-r">mu &lt;- clusterEvalQ(cl, { x &lt;- rnorm(n = 10e6); mean(x) })
134 #&gt; Error in unserialize(node$con) : error reading from connection
135 </code></pre>
136 <p>This is because the operating system terminated the two background R
137 processes, because they overused the memory. This is why the main R
138 process no longer can communicate with the parallel workers. We can
139 see that both workers are down, by calling:</p>
140 <pre><code class="language-r">isNodeAlive(cl)
141 #&gt; [1] FALSE FALSE
142 </code></pre>
143 <p>We can use <code>cloneNode()</code> to relaunch workers that are no longer
144 alive, e.g.</p>
145 <pre><code class="language-r">is_down &lt;- !isNodeAlive(cl)
146 cl[is_down] &lt;- cloneNode(cl[is_down])
147 isNodeAlive(cl)
148 #&gt; [1] TRUE TRUE
149 </code></pre>
150 <h2>Example: MS Windows parallel workers with specific CPU affinities</h2>
151 <p>This example works only on MS Windows machines. It launches four
152 local workers, where two are running on CPU Group #0 and two on CPU
153 Group #1.</p>
154 <pre><code class="language-r">library(parallelly)
155 rscript &lt;- I(c(
156 Sys.getenv(&quot;COMSPEC&quot;), &quot;/c&quot;,
157 &quot;start&quot;, &quot;/B&quot;,
158 &quot;/NODE&quot;, cpu_group=NA_integer_,
159 &quot;/AFFINITY&quot;, &quot;0xFFFFFFFFFFFFFFFE&quot;,
160 &quot;*&quot;)
161 )
162
163 rscript[&quot;cpu_group&quot;] &lt;- 0
164 cl_0 &lt;- makeClusterPSOCK(2, rscript = rscript)
165
166 rscript[&quot;cpu_group&quot;] &lt;- 1
167 cl_1 &lt;- makeClusterPSOCK(2, rscript = rscript)
168
169 cl &lt;- c(cl_0, cl_1)
170 </code></pre>
171 <p>The special <code>*</code> value expands to the proper <code>Rscript</code> on the machine
172 where the parallel workers are launched.</p>
173 <!-- See also: https://lovickconsulting.com/2021/11/18/running-r-clusters-on-an-amd-threadripper-3990x-in-windows-10-2/ -->
174 </body>
175 </html>