Debian Codegraph @

inst/doc/parallelly-20-limit-workers.md

raw
1 <!--
2 %\VignetteIndexEntry{Parallel Workers with CPU and Memory Limited}
3 %\VignetteAuthor{Henrik Bengtsson}
4 %\VignetteKeyword{R}
5 %\VignetteKeyword{package}
6 %\VignetteKeyword{vignette}
7 %\VignetteEngine{parallelly::selfonly}
8 -->
9
10
11 # Introduction
12
13 This vignette gives examples of how to restrict CPU and memory usage of
14 parallel workers. This can be useful for optimizing the performance of
15 the parallel workers, but also lower the risk that they overuse the
16 CPU and memory on the machines they are running on.
17
18
19 # Examples
20
21 ## Example: Linux parallel workers with a lower process priority ("nice")
22
23 On Unix, we can run any process with a lower CPU priority using the
24 `nice` command. This can be used when we want to lower the risk of
25 negatively affecting other users and processes that run on the same
26 machine due to our R workers overusing the CPUs by mistake. To achieve
27 this, we can prepend `nice` to the `Rscript` call via the `rscript`
28 argument. This works both on local and remote Linux machines,
29 e.g.
30
31 ```r
32 library(parallelly)
33 cl <- makeClusterPSOCK(2, rscript = c("nice", "*"))
34 ```
35
36 ```r
37 library(parallelly)
38 workers <- rep("n1.remote.org", 2)
39 cl <- makeClusterPSOCK(2, rscript = c("nice", "*"))
40 ```
41
42 The special `*` value expands to the proper `Rscript` on the machine
43 where the parallel workers are launched.
44
45
46
47 ## Example: Linux parallel workers CPU and memory limited by CGroups
48
49 This example launches two parallel workers each limited to 100% CPU
50 quota and 50 MiB of memory using Linux CGroups management. The 100%
51 CPU quota limit constrains each worker to use at most one CPU worth of
52 processing preventing them from overusing the machine, e.g. through
53 unintended nested parallelization. For more details, see `man
54 systemd.resource-control`.
55
56 ```r
57 library(parallelly)
58 cl <- makeClusterPSOCK(
59 2L,
60 rscript = c(
61 "systemd-run", "--user", "--scope",
62 "-p", "CPUQuota=100%",
63 "-p", "MemoryMax=50M", "-p", "MemorySwapMax=50M",
64 "*"
65 )
66 )
67 ```
68
69 Note, depending on your CGroups configuration, a non-privileged user
70 may or may not be able to set the CPU quota. If not, the `-p
71 CPUQuota=100%` will be silently ignored.
72
73 The 50 MiB memory limit is strict - if a worker uses more than this,
74 the operating system will terminate the worker instantly. To
75 illustrate what happens, we first start by generating 1 million
76 numeric values each consuming 8 bytes, which in total consumes ~8 MB,
77 and then calculate the mean, the memory consumption is within 50-MiB
78 memory limit that each parallel worker has available;
79
80 ```r
81 library(parallel)
82 mu <- clusterEvalQ(cl, { x <- rnorm(n = 1e6); mean(x) })
83 mu <- unlist(mu)
84 print(mu)
85 #> [1] 0.0008072657 -0.0019693992
86 ```
87
88 However, if we generate 10 times more values, the memory consumption
89 will grow to at least 80 MB, which is over the 50-MiB memory limit,
90 and we will get an error:
91
92 ```r
93 mu <- clusterEvalQ(cl, { x <- rnorm(n = 10e6); mean(x) })
94 #> Error in unserialize(node$con) : error reading from connection
95 ```
96
97 This is because the operating system terminated the two background R
98 processes, because they overused the memory. This is why the main R
99 process no longer can communicate with the parallel workers. We can
100 see that both workers are down, by calling:
101
102 ```r
103 isNodeAlive(cl)
104 #> [1] FALSE FALSE
105 ```
106
107 We can use `cloneNode()` to relaunch workers that are no longer
108 alive, e.g.
109
110 ```r
111 is_down <- !isNodeAlive(cl)
112 cl[is_down] <- cloneNode(cl[is_down])
113 isNodeAlive(cl)
114 #> [1] TRUE TRUE
115 ```
116
117
118 ## Example: MS Windows parallel workers with specific CPU affinities
119
120 This example works only on MS Windows machines. It launches four
121 local workers, where two are running on CPU Group #0 and two on CPU
122 Group #1.
123
124 ```r
125 library(parallelly)
126 rscript <- I(c(
127 Sys.getenv("COMSPEC"), "/c",
128 "start", "/B",
129 "/NODE", cpu_group=NA_integer_,
130 "/AFFINITY", "0xFFFFFFFFFFFFFFFE",
131 "*")
132 )
133
134 rscript["cpu_group"] <- 0
135 cl_0 <- makeClusterPSOCK(2, rscript = rscript)
136
137 rscript["cpu_group"] <- 1
138 cl_1 <- makeClusterPSOCK(2, rscript = rscript)
139
140 cl <- c(cl_0, cl_1)
141 ```
142
143 The special `*` value expands to the proper `Rscript` on the machine
144 where the parallel workers are launched.
145
146 <!-- See also: https://lovickconsulting.com/2021/11/18/running-r-clusters-on-an-amd-threadripper-3990x-in-windows-10-2/ -->