Debian Codegraph @

vignettes/parallelly-15-cloud-workers.md

raw
1 <!--
2 %\VignetteIndexEntry{Parallel Workers in the Cloud}
3 %\VignetteAuthor{Henrik Bengtsson}
4 %\VignetteKeyword{R}
5 %\VignetteKeyword{package}
6 %\VignetteKeyword{vignette}
7 %\VignetteEngine{parallelly::selfonly}
8 -->
9
10
11 # Introduction
12
13 This vignette illustrates how to launch parallel workers on cloud
14 services such as Amazon AWS (<https://aws.amazon.com/>) and Google
15 Compute Engine (<https://cloud.google.com/products/compute>).
16
17
18 # Examples
19
20 ## Example: Remote worker running on GCE
21
22 This example launches a parallel worker on Google Compute Engine (GCE)
23 running a container based VM (with a #cloud-config specification).
24
25 ```r
26 library(parallelly)
27
28 public_ip <- "1.2.3.4"
29 user <- "johnny"
30 ssh_private_key_file <- "~/.ssh/google_compute_engine"
31 cl <- makeClusterPSOCK(
32 ## Public IP number of GCE instance
33 public_ip,
34 ## User name (== SSH key label (sic!))
35 user = user,
36 ## Use private SSH key registered with GCE
37 rshopts = c(
38 "-o", "StrictHostKeyChecking=no",
39 "-o", "IdentitiesOnly=yes",
40 "-i", ssh_private_key_file
41 ),
42 ## Launch Rscript inside Docker container
43 rscript = c(
44 "docker", "run", "--net=host", "rocker/r-parallel",
45 "Rscript"
46 )
47 )
48 ```
49
50
51 ## Example: Remote worker running on AWS
52
53 This example, which is a bit dated, launches a parallel worker on
54 Amazon AWS EC2 running one of the Amazon Machine Images (AMI) provided
55 by Posit (<https://www.louisaslett.com/RStudio_AMI/>).
56
57 ```r
58 library(parallelly)
59
60 public_ip <- "1.2.3.4"
61 ssh_private_key_file <- "~/.ssh/my-private-aws-key.pem"
62
63 cl <- makeClusterPSOCK(
64 ## Public IP number of EC2 instance
65 public_ip,
66 ## User name (always 'ubuntu')
67 user = "ubuntu",
68 ## Use private SSH key registered with AWS
69 rshopts = c(
70 "-o", "StrictHostKeyChecking=no",
71 "-o", "IdentitiesOnly=yes",
72 "-i", ssh_private_key_file
73 ),
74 ## Set up .libPaths() for the 'ubuntu' user
75 ## and then install the future package
76 rscript_startup = quote(local({
77 p <- Sys.getenv("R_LIBS_USER")
78 dir.create(p, recursive = TRUE, showWarnings = FALSE)
79 .libPaths(p)
80 install.packages("future")
81 }))
82 )
83 ```