Debian Codegraph @

inst/doc/parallelly-15-cloud-workers.html

raw
1 <!DOCTYPE html>
2 <html lang="en">
3 <head>
4 <meta charset="UTF-8">
5 <title>Parallel Workers in the Cloud</title>
6 <style>
7 body {
8 font-family: sans-serif;
9 line-height: 1.6;
10 padding-left: 3ex;
11 padding-right: 3ex;
12 background-color: white;
13 color: black;
14 }
15
16 a {
17 color: #4183C4;
18 text-decoration: none;
19 }
20
21 h1, h2, h3 {
22 margin: 2ex 0 1ex;
23 padding: 0;
24 font-weight: bold;
25 -webkit-font-smoothing: antialiased;
26 cursor: text;
27 position: relative;
28 }
29
30 h2 {
31 border-bottom: 1px solid #cccccc;
32 }
33
34 code {
35 margin: 0 2px;
36 padding: 0 5px;
37 white-space: nowrap;
38 border: 1px solid #eaeaea;
39 background-color: #f8f8f8;
40 border-radius: 3px;
41 }
42
43 pre code {
44 margin: 0;
45 padding: 0;
46 white-space: pre;
47 border: none;
48 background: transparent;
49 }
50
51 pre {
52 background-color: #f8f8f8;
53 border: 1px solid #cccccc;
54 line-height: 2.5x;
55 overflow: auto;
56 padding: 0.6ex 1ex;
57 border-radius: 3px;
58 }
59
60 pre code {
61 background-color: transparent;
62 border: none;
63 }
64 </style>
65 </head>
66 <body>
67 <h1>Parallel Workers in the Cloud</h1>
68 <!--
69 %\VignetteIndexEntry{Parallel Workers in the Cloud}
70 %\VignetteAuthor{Henrik Bengtsson}
71 %\VignetteKeyword{R}
72 %\VignetteKeyword{package}
73 %\VignetteKeyword{vignette}
74 %\VignetteEngine{parallelly::selfonly}
75 -->
76 <h1>Introduction</h1>
77 <p>This vignette illustrates how to launch parallel workers on cloud
78 services such as Amazon AWS (<a href="https://aws.amazon.com/">https://aws.amazon.com/</a>) and Google
79 Compute Engine (<a href="https://cloud.google.com/products/compute">https://cloud.google.com/products/compute</a>).</p>
80 <h1>Examples</h1>
81 <h2>Example: Remote worker running on GCE</h2>
82 <p>This example launches a parallel worker on Google Compute Engine (GCE)
83 running a container based VM (with a #cloud-config specification).</p>
84 <pre><code class="language-r">library(parallelly)
85
86 public_ip &lt;- &quot;1.2.3.4&quot;
87 user &lt;- &quot;johnny&quot;
88 ssh_private_key_file &lt;- &quot;~/.ssh/google_compute_engine&quot;
89 cl &lt;- makeClusterPSOCK(
90 ## Public IP number of GCE instance
91 public_ip,
92 ## User name (== SSH key label (sic!))
93 user = user,
94 ## Use private SSH key registered with GCE
95 rshopts = c(
96 &quot;-o&quot;, &quot;StrictHostKeyChecking=no&quot;,
97 &quot;-o&quot;, &quot;IdentitiesOnly=yes&quot;,
98 &quot;-i&quot;, ssh_private_key_file
99 ),
100 ## Launch Rscript inside Docker container
101 rscript = c(
102 &quot;docker&quot;, &quot;run&quot;, &quot;--net=host&quot;, &quot;rocker/r-parallel&quot;,
103 &quot;Rscript&quot;
104 )
105 )
106 </code></pre>
107 <h2>Example: Remote worker running on AWS</h2>
108 <p>This example, which is a bit dated, launches a parallel worker on
109 Amazon AWS EC2 running one of the Amazon Machine Images (AMI) provided
110 by Posit (<a href="https://www.louisaslett.com/RStudio_AMI/">https://www.louisaslett.com/RStudio_AMI/</a>).</p>
111 <pre><code class="language-r">library(parallelly)
112
113 public_ip &lt;- &quot;1.2.3.4&quot;
114 ssh_private_key_file &lt;- &quot;~/.ssh/my-private-aws-key.pem&quot;
115
116 cl &lt;- makeClusterPSOCK(
117 ## Public IP number of EC2 instance
118 public_ip,
119 ## User name (always 'ubuntu')
120 user = &quot;ubuntu&quot;,
121 ## Use private SSH key registered with AWS
122 rshopts = c(
123 &quot;-o&quot;, &quot;StrictHostKeyChecking=no&quot;,
124 &quot;-o&quot;, &quot;IdentitiesOnly=yes&quot;,
125 &quot;-i&quot;, ssh_private_key_file
126 ),
127 ## Set up .libPaths() for the 'ubuntu' user
128 ## and then install the future package
129 rscript_startup = quote(local({
130 p &lt;- Sys.getenv(&quot;R_LIBS_USER&quot;)
131 dir.create(p, recursive = TRUE, showWarnings = FALSE)
132 .libPaths(p)
133 install.packages(&quot;future&quot;)
134 }))
135 )
136 </code></pre>
137 </body>
138 </html>