-
Notifications
You must be signed in to change notification settings - Fork 2
/
index.html
514 lines (509 loc) · 19.4 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>CRAN Task View: Computational Environments and Reproducibility</title>
<link rel="stylesheet" type="text/css" href="https://cran.r-project.org/R.css" />
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta name="citation_title" content="CRAN Task View: Computational Environments and Reproducibility" />
<meta name="citation_author" content="Daniel Nüst" />
<meta name="citation_publication_date" content="2019-01-11" />
<meta name="DC.title" content="CRAN Task View: Computational Environments and Reproducibility" />
<meta name="DC.creator" content="Daniel Nüst" />
<meta name="DC.issued" content="2019-01-11" />
</head>
<body>
<h2>CRAN Task View: Computational Environments and Reproducibility</h2>
<table summary="ComputationalEnvironments task view information">
<tr><td valign="top"><b>Maintainer:</b></td><td>Daniel Nüst</td></tr>
<tr><td valign="top"><b>Contact:</b></td><td>daniel.nuest at uni-muenster.de</td></tr>
<tr><td valign="top"><b>Version:</b></td><td>2019-01-11</td></tr>
</table>
<div>
<p>
This Task View contains information about controlling and documenting computational environments in R. The base version of R does not provide features to manage different version of R or collections of packages easily, so a number of approaches and packages exist to simplify computational environments for the sake of development, testing, bug-fixing, and reproducibility. The
<a href="https://cran.r-project.org/view=ReproducibleResearch">
ReproducibleResearch Task View
</a>
provides further discussion of packages around scientific reproducibility.
</p>
<p>
If you have any comments or suggestions for additions or improvements for this Task View, go to GitHub and
<a href="https://github.com/o2r-project/ctv-computational-environments/issues">
submit an issue
</a>, or make some changes and
<a href="https://github.com/o2r-project/ctv-computational-environments/pulls">
submit a pull request
</a>. If you can’t contribute on GitHub,
<a href="mailto:daniel.nuest@uni-muenster.de">
send Daniel an email
</a>. If you have an issue with one of the packages discussed below, please contact the maintainer of that package.
</p>
<p>
<strong>
Contributors:
</strong>
<span class="citation" data-cites="nuest">
[@nuest]
</span>
(https://github.com/nuest/),
<span class="citation" data-cites="jdblischak">
[@jdblischak]
</span>
(https://github.com/jdblischak/)
</p>
<h2 id="virtual-machines-and-containers">
Virtual Machines and Containers
</h2>
<p>
<strong>
Virtual machines
</strong>
(VMs) are a straightforward way to encapsulate your runtime environment around the actual data and code.
</p>
<blockquote>
<p>
<em>
In computing, a virtual machine (VM) is an emulation of a computer system. Virtual machines are based on computer architectures and provide functionality of a physical computer.
</em>
<a href="https://en.wikipedia.org/wiki/Virtual_machine">
https://en.wikipedia.org/wiki/Virtual_machine
</a>
</p>
</blockquote>
<p>
Their advantage for many users is the provided visual user interface.
<a href="https://www.virtualbox.org/">
VirtualBox
</a>
is a Free and Open Source (FOSS) virtualization product you can install on most operating systems (OS), then known as the “host” OS, and supports a number of
<a href="https://www.virtualbox.org/wiki/Guest_OSes">
“guest” operating systems
</a>, many of which can install and run R just like on a non-virtual OS. VMs must be booted like regular OS and have a virtual disk, which you can archive or share with collaborators (though having to handle the large file size). VMs can share directories, network, and other devices with their host.
</p>
<p>
<a href="https://en.wikipedia.org/wiki/Linux_containers">
<strong>
Containers
</strong>
</a>
are an effective way to apply virtual environments at the system level. Compared to
<a href="#virtual-machines">
virtual machines
</a>
their most important advantage for controlled computational environments are performance and transparency. Containers share the hosts core libraries and can “boot” within milliseconds and have negligible computational overhead. They can be created with the help of scripts or “recipes”, which are simple text files. These recipes can be included in code repositories and easily shared online.
</p>
<p>
The most widespread container solution is
<strong>
<a href="https://docker.com">
Docker
</a>
</strong>. It is available for recent and common operating systems.
<em>
Dockerfiles
</em>
are the recipes that can be built to
<em>
Docker images
</em>, which can be run and become
<em>
Docker containers
</em>. Using Docker requires some proficiency with a
<a href="https://en.wikipedia.org/wiki/Command-line_interface">
command line interface
</a>
(CLI). The
<a href="http://hub.docker.com/">
Docker Hub
</a>
is an image repository with a large number of pre-built images for different use cases.
</p>
<h3 id="rocker">
Rocker
</h3>
<p>
The
<a href="http://rocker-project.org/">
<strong>
Rocker
</strong>
project
</a>
provides a number of Docker
<a href="https://www.rocker-project.org/images/">
images for R
</a>, including the
<a href="https://docs.docker.com/docker-hub/official_repos/">
official
</a>
<a href="https://hub.docker.com/_/r-base/">
<tt>r-base</tt>
</a>
image. All Rocker images are available on the Docker Hub. Rocker images are a stable and widespread tool for running R in local and cloud environments and have established useful best practices around containers with R.
</p>
<p>
<strong>
Bioconductor
</strong>
provides a
<a href="http://bioconductor.org/help/docker/">
collection of images
</a>
based on
<tt>rocker/rstudio</tt>.
</p>
<p>
<em>
Since images can extend existing ones, using a suitable Rocker image as a base for your own computations is a very good approach to control your computational environment.
</em>
The simplest way is to run the
<tt>rocker/rstudio</tt>
container and work with the
<a href="https://www.rstudio.com/products/rstudio/">
RStudio IDE
</a>
in your web browser. Alternatively, you can develop your analysis on your computer and “package” it in a container only when preparing for a software release of scientific publication.
</p>
<h3 id="other-r-distributions-and-operating-systems">
Other R distributions and operating systems
</h3>
<p>
Docker images for other than the “regular” R distribution and the
<a href="https://www.debian.org">
Debian
</a>
-based Rocker images are available on Docker Hub, though none at the level of maturity and features of Rocker.
</p>
<ul>
<li>
<strong>
<a href="https://mran.microsoft.com/open">
MRO
</a>
</strong>
images are available as an independent contribution (i.e. not by MRO team) on Docker Hub,
<a href="https://hub.docker.com/r/nuest/mro/">
<tt>nuest/mro</tt>
</a>, and as CentOS-based Dockerfiles on GitHub,
<a href="https://github.com/jlisic/R-docker-centos">
<tt>jlisic/R-docker-centos</tt>
</a>.
</li>
<li>
<a href="http://www.renjin.org/about.html">
Renjin
</a>
images are available as an independent contribution on Docker Hub as
<a href="https://hub.docker.com/r/nuest/renjin/">
<tt>nuest/renjin</tt>
</a>
</li>
<li>
<a href="http://www.pqr-project.org/">
pqR
</a>
images are available as an independent contribution on Docker Hub as
<a href="https://hub.docker.com/r/nuest/pqr/">
<tt>nuest/pqr</tt>
</a>
</li>
</ul>
<h3 id="tools-for-working-with-containers">
Tools for working with containers
</h3>
<p>
<strong>
Docker
</strong>
</p>
<ul>
<li>
<a href="https://github.com/wch/harbor/">
<tt>harbor</tt>
</a>
(not on CRAN) provides all Docker commands with R functions. It may be used to control Docker containers that run either locally or remotely.
</li>
<li>
<a href="https://cran.r-project.org/web/packages/docker/index.html">docker</a>
is an alternative to the plain R
<tt>harbor</tt>
and provides Docker CLI commands using the
<a href="https://docker-py.readthedocs.io/en/stable/">
Docker SDK for Python
</a>
via the package
<a href="https://rstudio.github.io/reticulate/">
<tt>reticulate</tt>
</a>
and consequently runs on various operating systems including Windows. The package is best suited for apt Docker users, i.e. if you know the Docker commands and life cycle. Source code is
<a href="https://github.com/bhaskarvk/docker">
on GitHub
</a>.
</li>
<li>
<a href="https://github.com/cboettig/dockermachine">
<tt>dockermachine</tt>
</a>
(not on CRAN) provides a convenient R interface to the
<a href="https://docs.docker.com/machine/overview/">
<tt>docker-machine</tt>
</a>
command, so you can provision easily local or remote/cloud instances of containers.
</li>
<li>
<a href="https://cran.r-project.org/web/packages/analogsea/index.html">analogsea</a>
is a general purpose client for the Digital Ocean v2 API. In addition, the package includes functions to install various R tools including base R, RStudio server, and more. There’s an improving interface to interact with docker on your remote droplets via this package. (
<a href="https://github.com/sckott/analogsea">
GitHub
</a>)
</li>
<li>
<a href="https://github.com/cole-brokamp/rize">
<tt>rize</tt>
</a>
(not on CRAN) dockerises
<a href="https://shiny.rstudio.com/">
Shiny
</a>
applications.
</li>
<li>
<a href="https://github.com/o2r-project/containerit">
<tt>containerit</tt>
</a>
(not on CRAN) automatically creates Dockerfiles for arbitrary R sessions, script files, or workspace directories.
</li>
<li>
<a href="https://github.com/traitecoevo/dockertest">
<tt>dockertest</tt>
</a>
(not con CRAN) is a proof of concept for using the isolated environments of Docker containers to run tests.
</li>
<li>
<a href="https://cran.r-project.org/web/packages/liftr/index.html">liftr</a>
partially automates rendering R Markdown documents with Docker by adding YAML-metadata (
<a href="https://github.com/road2stat/dockflow/blob/master/config/sequencing.yml">
example
</a>), see
<a href="http://liftr.me/">
http://liftr.me/
</a>.
</li>
<li>
<a href="https://cran.r-project.org/web/packages/googleComputeEngineR/index.html">googleComputeEngineR</a>
(
<a href="https://cloudyr.github.io/googleComputeEngineR/">
website
</a>) provides an R interface to the Google Cloud Compute Engine API, for example for creating an RStudio VM, also using Docker to configure the environment.
</li>
<li>
<a href="https://cran.r-project.org/package=batchtools">
<strong>
<tt>batchtools</tt>
</strong>
</a>
(
<a href="https://github.com/mllg/batchtools">
repository
</a>,
<a href="http://dx.doi.org/10.21105/joss.00135">
JOSS paper
</a>) provides a parallel implementation of
<a href="https://en.wikipedia.org/wiki/Map_(parallel_pattern)">
Map
</a>
for
<a href="https://en.wikipedia.org/wiki/Supercomputer">
HPC
</a>
for different
<a href="https://en.wikipedia.org/wiki/Job_scheduler">
schedulers
</a>, including
<a href="https://docs.docker.com/engine/swarm/">
Docker Swarm
</a>.
</li>
</ul>
<h2 id="deployment">
Deployment
</h2>
<p>
Another alternative to share a well-defined computational environment is setting up R on a server.
</p>
<p>
<strong>
Interactive development environments
</strong>
</p>
<ul>
<li>
<a href="https://www.rstudio.com/products/rstudio/#Server">
RStudio Server
</a>
</li>
<li>
<a href="http://rcloud.social">
RCloud
</a>
(
<a href="https://hub.docker.com/r/rcl0ud/rcloud/">
Docker images
</a>
and
<a href="https://github.com/att/rcloud/tree/master/docker">
Dockerfile
</a>)
</li>
</ul>
<p>
<strong>
Apps and APIS
</strong>
</p>
<ul>
<li>
<a href="https://www.shinyproxy.io/">
ShinyProxy
</a>
by
<a href="https://www.openanalytics.eu/">
Open Analytics
</a>
</li>
<li>
<a href="https://cran.r-project.org/web/packages/plumber/index.html">plumber</a>
(
<a href="https://www.rplumber.io/">
website
</a>
allows creating web services as HTTP APIs in pure R.
</li>
<li>
<a href="http://rapache.net">
rApache
</a>
supports web application development using the
<a href="https://httpd.apache.org/">
Apache web server
</a>.
</li>
</ul>
<h2 id="package-management">
Package management
</h2>
<ul>
<li>
<a href="https://cran.r-project.org/web/packages/checkpoint/index.html">checkpoint</a>
(
<a href="https://github.com/RevolutionAnalytics/checkpoint">
GitHub
</a>)
</li>
<li>
<a href="https://cran.r-project.org/web/packages/rbundler/index.html">rbundler</a>
</li>
<li>
<a href="https://cran.r-project.org/web/packages/RSuite/index.html">RSuite</a>
(
<a href="https://github.com/WLOGSolutions/RSuite">
GitHub
</a>)
</li>
<li>
<a href="https://cran.r-project.org/web/packages/packrat/index.html">packrat</a>
(
<a href="https://github.com/rstudio/packrat">
GitHub
</a>)
</li>
<li>
pkgsnap (
<a href="https://github.com/MangoTheCat/pkgsnap">
GitHub
</a>)
</li>
<li>
<a href="https://cran.r-project.org/web/packages/switchr/index.html">switchr</a>
(
<a href="https://github.com/gmbecker/switchr">
GitHub
</a>)
</li>
<li>
<a href="https://cran.r-project.org/web/packages/switchrGist/index.html">switchrGist</a>
(
<a href="https://github.com/gmbecker/switchrGist">
GitHub
</a>)
</li>
</ul>
<h2 id="structure-templates-and-workflows">
Structure, templates and workflows
</h2>
<p>
A good project structure is essential to be sure about the actually used computational environment, which includes locally defines functions and data and not just used packages or the R version.
</p>
<ul>
<li>
<a href="https://cran.r-project.org/web/packages/here/index.html">here</a>
constructs paths to a project’s files.
</li>
<li>
<a href="https://github.com/benmarwick/researchcompendium">
Ben Marwick’s compendium
</a>
</li>
<li>
<a href="https://github.com/benmarwick/rrtools">
rrtools
</a>
</li>
</ul>
<h2 id="tracking-and-provenance">
Tracking and provenance
</h2>
<p>
A computational environment evolves as an analysis is developed. These packages help observing these changes, in addition to always recommended
<a href="https://en.wikipedia.org/wiki/Version%20control%20systems">
code versioning systems
</a>.
</p>
<ul>
<li>
<a href="https://github.com/ekernf01/freezr">
<tt>freezer</tt>
</a>
(not on CRAN) helps data analysis by capturing analyses executions including used code, results, and metadata.
</li>
<li>
<a href="https://github.com/NCEAS/recordr">
<tt>recordr</tt>
</a>
(not on CRAN) provides an automated way to capture data provenance of “runs” for R scripts and console commands.
</li>
</ul>
</div>
<h3>CRAN packages:</h3>
<ul>
<li><a href="https://cran.r-project.org/web/packages/analogsea/index.html">analogsea</a></li>
<li><a href="https://cran.r-project.org/web/packages/checkpoint/index.html">checkpoint</a></li>
<li><a href="https://cran.r-project.org/web/packages/docker/index.html">docker</a> (core)</li>
<li><a href="https://cran.r-project.org/web/packages/googleComputeEngineR/index.html">googleComputeEngineR</a></li>
<li><a href="https://cran.r-project.org/web/packages/here/index.html">here</a></li>
<li><a href="https://cran.r-project.org/web/packages/liftr/index.html">liftr</a></li>
<li><a href="https://cran.r-project.org/web/packages/packrat/index.html">packrat</a></li>
<li><a href="https://cran.r-project.org/web/packages/plumber/index.html">plumber</a></li>
<li><a href="https://cran.r-project.org/web/packages/rbundler/index.html">rbundler</a></li>
<li><a href="https://cran.r-project.org/web/packages/RSuite/index.html">RSuite</a></li>
<li><a href="https://cran.r-project.org/web/packages/switchr/index.html">switchr</a></li>
<li><a href="https://cran.r-project.org/web/packages/switchrGist/index.html">switchrGist</a></li>
</ul>
<h3>Related links:</h3>
<ul>
<li><a href="https://cran.r-project.org/view=ReproducibleResearch">
ReproducibleResearch TaskView
</a></li>
</ul>
</body>
</html>