Debian Codegraph @

inst/doc/parallelly-21-container-workers.md

raw
1 <!--
2 %\VignetteIndexEntry{Parallel Workers Running in Linux Containers}
3 %\VignetteAuthor{Henrik Bengtsson}
4 %\VignetteKeyword{R}
5 %\VignetteKeyword{package}
6 %\VignetteKeyword{vignette}
7 %\VignetteKeyword{Docker}
8 %\VignetteKeyword{Apptainer}
9 %\VignetteEngine{parallelly::selfonly}
10 -->
11
12
13 # Introduction
14
15 This vignette shows how to set up parallel workers running in Linux
16 containers, e.g. Docker (<https://www.docker.com/>), Apptainer
17 (<https://apptainer.org/>), and udocker
18 (<https://indigo-dc.github.io/udocker/>).
19
20
21
22 # Examples
23
24 ## Example: Two parallel workers running in Docker
25
26 This example sets up two parallel workers running Docker image
27 'rocker/r-parallel' (<https://hub.docker.com/r/rocker/r-parallel>).
28
29 ```r
30 library(parallelly)
31 cl <- makeClusterPSOCK(
32 rep("localhost", times = 2L),
33 ## Launch Rscript inside Linux container via Docker
34 rscript = c(
35 "docker", "run", "--net=host", "rocker/r-parallel",
36 "Rscript"
37 ),
38 ## IMPORTANT: Because Docker runs inside a virtual machine (VM) on macOS
39 ## and MS Windows (not Linux), when the R worker tries to connect back to
40 ## the default 'localhost' it will fail, because the main R session is
41 ## not running in the VM, but outside on the host. To reach the host on
42 ## macOS and MS Windows, make sure to use master = "host.docker.internal"
43 master = if (.Platform$OS.type == "unix") NULL else "host.docker.internal",
44 )
45 print(cl)
46 #> Socket cluster with 2 nodes on host 'localhost' (R version 4.3.3
47 #> (2024-02-29), platform x86_64-pc-linux-gnu)
48 ```
49
50
51 ## Example: Two parallel workers running in Apptainer
52
53 This example shows how to set up two parallel workers running Docker
54 image 'rocker/r-parallel'
55 (<https://hub.docker.com/r/rocker/r-parallel>) via Apptainer
56 (<https://apptainer.org/>).
57
58 ```r
59 library(parallelly)
60 cl <- makeClusterPSOCK(
61 rep("localhost", times = 2L),
62 ## Launch Rscript inside Linux container via Apptainer
63 rscript = c(
64 "apptainer", "exec", "docker://rocker/r-parallel",
65 "Rscript"
66 )
67 )
68 print(cl)
69 #> Socket cluster with 2 nodes on host 'localhost' (R version 3.6.1
70 #> (2019-07-05), platform x86_64-pc-linux-gnu)
71 ```
72
73
74 ## Example: Two parallel workers running in udocker
75
76 This example shows how to set up two parallel workers running Docker
77 image 'rocker/r-parallel'
78 (<https://hub.docker.com/r/rocker/r-parallel>) via udocker
79 (<https://indigo-dc.github.io/udocker/>).
80
81 ```r
82 library(parallelly)
83 cl <- makeClusterPSOCK(
84 rep("localhost", times = 2L),
85 ## Launch Rscript inside Linux container via Docker
86 rscript = c(
87 "udocker", "--quiet", "run", "rocker/r-parallel",
88 "Rscript"
89 )
90 )
91 print(cl)
92 #> Socket cluster with 2 nodes on host 'localhost' (R version 3.6.1
93 #> (2019-07-05), platform x86_64-pc-linux-gnu)
94 ```