Debian Codegraph @

inst/doc/parallelly-21-container-workers.html

raw
1 <!DOCTYPE html>
2 <html lang="en">
3 <head>
4 <meta charset="UTF-8">
5 <title>Parallel Workers Running in Linux Containers</title>
6 <style>
7 body {
8 font-family: sans-serif;
9 line-height: 1.6;
10 padding-left: 3ex;
11 padding-right: 3ex;
12 background-color: white;
13 color: black;
14 }
15
16 a {
17 color: #4183C4;
18 text-decoration: none;
19 }
20
21 h1, h2, h3 {
22 margin: 2ex 0 1ex;
23 padding: 0;
24 font-weight: bold;
25 -webkit-font-smoothing: antialiased;
26 cursor: text;
27 position: relative;
28 }
29
30 h2 {
31 border-bottom: 1px solid #cccccc;
32 }
33
34 code {
35 margin: 0 2px;
36 padding: 0 5px;
37 white-space: nowrap;
38 border: 1px solid #eaeaea;
39 background-color: #f8f8f8;
40 border-radius: 3px;
41 }
42
43 pre code {
44 margin: 0;
45 padding: 0;
46 white-space: pre;
47 border: none;
48 background: transparent;
49 }
50
51 pre {
52 background-color: #f8f8f8;
53 border: 1px solid #cccccc;
54 line-height: 2.5x;
55 overflow: auto;
56 padding: 0.6ex 1ex;
57 border-radius: 3px;
58 }
59
60 pre code {
61 background-color: transparent;
62 border: none;
63 }
64 </style>
65 </head>
66 <body>
67 <h1>Parallel Workers Running in Linux Containers</h1>
68 <!--
69 %\VignetteIndexEntry{Parallel Workers Running in Linux Containers}
70 %\VignetteAuthor{Henrik Bengtsson}
71 %\VignetteKeyword{R}
72 %\VignetteKeyword{package}
73 %\VignetteKeyword{vignette}
74 %\VignetteKeyword{Docker}
75 %\VignetteKeyword{Apptainer}
76 %\VignetteEngine{parallelly::selfonly}
77 -->
78 <h1>Introduction</h1>
79 <p>This vignette shows how to set up parallel workers running in Linux
80 containers, e.g. Docker (<a href="https://www.docker.com/">https://www.docker.com/</a>), Apptainer
81 (<a href="https://apptainer.org/">https://apptainer.org/</a>), and udocker
82 (<a href="https://indigo-dc.github.io/udocker/">https://indigo-dc.github.io/udocker/</a>).</p>
83 <h1>Examples</h1>
84 <h2>Example: Two parallel workers running in Docker</h2>
85 <p>This example sets up two parallel workers running Docker image
86 ‘rocker/r-parallel’ (<a href="https://hub.docker.com/r/rocker/r-parallel">https://hub.docker.com/r/rocker/r-parallel</a>).</p>
87 <pre><code class="language-r">library(parallelly)
88 cl &lt;- makeClusterPSOCK(
89 rep(&quot;localhost&quot;, times = 2L),
90 ## Launch Rscript inside Linux container via Docker
91 rscript = c(
92 &quot;docker&quot;, &quot;run&quot;, &quot;--net=host&quot;, &quot;rocker/r-parallel&quot;,
93 &quot;Rscript&quot;
94 ),
95 ## IMPORTANT: Because Docker runs inside a virtual machine (VM) on macOS
96 ## and MS Windows (not Linux), when the R worker tries to connect back to
97 ## the default 'localhost' it will fail, because the main R session is
98 ## not running in the VM, but outside on the host. To reach the host on
99 ## macOS and MS Windows, make sure to use master = &quot;host.docker.internal&quot;
100 master = if (.Platform$OS.type == &quot;unix&quot;) NULL else &quot;host.docker.internal&quot;,
101 )
102 print(cl)
103 #&gt; Socket cluster with 2 nodes on host 'localhost' (R version 4.3.3
104 #&gt; (2024-02-29), platform x86_64-pc-linux-gnu)
105 </code></pre>
106 <h2>Example: Two parallel workers running in Apptainer</h2>
107 <p>This example shows how to set up two parallel workers running Docker
108 image ‘rocker/r-parallel’
109 (<a href="https://hub.docker.com/r/rocker/r-parallel">https://hub.docker.com/r/rocker/r-parallel</a>) via Apptainer
110 (<a href="https://apptainer.org/">https://apptainer.org/</a>).</p>
111 <pre><code class="language-r">library(parallelly)
112 cl &lt;- makeClusterPSOCK(
113 rep(&quot;localhost&quot;, times = 2L),
114 ## Launch Rscript inside Linux container via Apptainer
115 rscript = c(
116 &quot;apptainer&quot;, &quot;exec&quot;, &quot;docker://rocker/r-parallel&quot;,
117 &quot;Rscript&quot;
118 )
119 )
120 print(cl)
121 #&gt; Socket cluster with 2 nodes on host 'localhost' (R version 3.6.1
122 #&gt; (2019-07-05), platform x86_64-pc-linux-gnu)
123 </code></pre>
124 <h2>Example: Two parallel workers running in udocker</h2>
125 <p>This example shows how to set up two parallel workers running Docker
126 image ‘rocker/r-parallel’
127 (<a href="https://hub.docker.com/r/rocker/r-parallel">https://hub.docker.com/r/rocker/r-parallel</a>) via udocker
128 (<a href="https://indigo-dc.github.io/udocker/">https://indigo-dc.github.io/udocker/</a>).</p>
129 <pre><code class="language-r">library(parallelly)
130 cl &lt;- makeClusterPSOCK(
131 rep(&quot;localhost&quot;, times = 2L),
132 ## Launch Rscript inside Linux container via Docker
133 rscript = c(
134 &quot;udocker&quot;, &quot;--quiet&quot;, &quot;run&quot;, &quot;rocker/r-parallel&quot;,
135 &quot;Rscript&quot;
136 )
137 )
138 print(cl)
139 #&gt; Socket cluster with 2 nodes on host 'localhost' (R version 3.6.1
140 #&gt; (2019-07-05), platform x86_64-pc-linux-gnu)
141 </code></pre>
142 </body>
143 </html>