index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>nutriverse</title>
    <link>/</link>
      <atom:link href="/index.xml" rel="self" type="application/rss+xml" />
    <description>nutriverse</description>
    <generator>Hugo -- gohugo.io</generator><language>en-gb</language><lastBuildDate>Thu, 25 Jun 2020 00:00:00 +0000</lastBuildDate>
    <item>
      <title>Build a model</title>
      <link>/start/models/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/start/models/</guid>
      <description>&lt;h2 id=&#34;intro&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;How do you create a statistical model using tidymodels? In this article, we will walk you through the steps. We start with data for modeling, learn how to specify and train models with different engines using the 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip package&lt;/a&gt;, and understand why these functions are designed this way.&lt;/p&gt;
&lt;p&gt;To use code in this article,  you will need to install the following packages: readr, rstanarm, and tidymodels.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for the parsnip package, along with the rest of tidymodels&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Helper packages&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(readr)       &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for importing data&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;data&#34;&gt;The Sea Urchins Data&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s use the data from 
&lt;a href=&#34;https://link.springer.com/article/10.1007/BF00349318&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Constable (1993)&lt;/a&gt; to explore how three different feeding regimes affect the size of sea urchins over time. The initial size of the sea urchins at the beginning of the experiment probably affects how big they grow as they are fed.&lt;/p&gt;
&lt;p&gt;To start, let&amp;rsquo;s read our urchins data into R, which we&amp;rsquo;ll do by providing 
&lt;a href=&#34;https://readr.tidyverse.org/reference/read_delim.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;readr::read_csv()&lt;/code&gt;&lt;/a&gt; with a url where our CSV data is located (&amp;ldquo;&lt;a href=&#34;https://tidymodels.org/start/models/urchins.csv&#34;&gt;https://tidymodels.org/start/models/urchins.csv&lt;/a&gt;&amp;rdquo;):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;urchins &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Data were assembled for a tutorial &lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# at https://www.flutterbys.com.au/stats/tut/tut7.5a.html&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;read_csv&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;https://tidymodels.org/start/models/urchins.csv&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Change the names to be a little more verbose&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;setNames&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;food_regime&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;initial_volume&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;width&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Factors are very helpful for modeling, so we convert one column&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(food_regime &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;factor&lt;/span&gt;(food_regime, levels &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Initial&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Low&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;High&amp;#34;&lt;/span&gt;)))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Parsed with column specification:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; cols(&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   TREAT = col_character(),&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   IV = col_double(),&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   SUTW = col_double()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; )&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let&amp;rsquo;s take a quick look at the data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;urchins
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 72 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    food_regime initial_volume width&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;fct&amp;gt;                &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 Initial                3.5 0.01 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 Initial                5   0.02 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 Initial                8   0.061&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 Initial               10   0.051&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 Initial               13   0.041&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 Initial               13   0.061&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 Initial               15   0.041&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 Initial               15   0.071&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 Initial               16   0.092&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 Initial               17   0.051&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 62 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The urchins data is a 
&lt;a href=&#34;https://tibble.tidyverse.org/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tibble&lt;/a&gt;. If you are new to tibbles, the best place to start is the 
&lt;a href=&#34;https://r4ds.had.co.nz/tibbles.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tibbles chapter&lt;/a&gt; in &lt;em&gt;R for Data Science&lt;/em&gt;. For each of the 72 urchins, we know their:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;experimental feeding regime group (&lt;code&gt;food_regime&lt;/code&gt;: either &lt;code&gt;Initial&lt;/code&gt;, &lt;code&gt;Low&lt;/code&gt;, or &lt;code&gt;High&lt;/code&gt;),&lt;/li&gt;
&lt;li&gt;size in milliliters at the start of the experiment (&lt;code&gt;initial_volume&lt;/code&gt;), and&lt;/li&gt;
&lt;li&gt;suture width at the end of the experiment (&lt;code&gt;width&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;As a first step in modeling, it&amp;rsquo;s always a good idea to plot the data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(urchins,
       &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; initial_volume, 
           y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; width, 
           group &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; food_regime, 
           col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; food_regime)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_smooth&lt;/span&gt;(method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; lm, se &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;scale_color_viridis_d&lt;/span&gt;(option &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;plasma&amp;#34;&lt;/span&gt;, end &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.7&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; `geom_smooth()` using formula &amp;#39;y ~ x&amp;#39;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/urchin-plot-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We can see that urchins that were larger in volume at the start of the experiment tended to have wider sutures at the end, but the slopes of the lines look different so this effect may depend on the feeding regime condition.&lt;/p&gt;
&lt;h2 id=&#34;build-model&#34;&gt;Build and fit a model&lt;/h2&gt;
&lt;p&gt;A standard two-way analysis of variance (
&lt;a href=&#34;https://www.itl.nist.gov/div898/handbook/prc/section4/prc43.htm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ANOVA&lt;/a&gt;) model makes sense for this dataset because we have both a continuous predictor and a categorical predictor. Since the slopes appear to be different for at least two of the feeding regimes, let&amp;rsquo;s build a model that allows for two-way interactions. Specifying an R formula with our variables in this way:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;width &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; initial_volume &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; food_regime
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;allows our regression model depending on initial volume to have separate slopes and intercepts for each food regime.&lt;/p&gt;
&lt;p&gt;For this kind of model, ordinary least squares is a good initial approach. With tidymodels, we start by specifying the &lt;em&gt;functional form&lt;/em&gt; of the model that we want using the 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip package&lt;/a&gt;. Since there is a numeric outcome and the model should be linear with slopes and intercepts, the model type is 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/linear_reg.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&amp;ldquo;linear regression&amp;rdquo;&lt;/a&gt;. We can declare this with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;linear_reg&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Linear Regression Model Specification (regression)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;That is pretty underwhelming since, on its own, it doesn&amp;rsquo;t really do much. However, now that the type of model has been specified, a method for &lt;em&gt;fitting&lt;/em&gt; or training the model can be stated using the &lt;strong&gt;engine&lt;/strong&gt;. The engine value is often a mash-up of the software that can be used to fit or train the model as well as the estimation method. For example, to use ordinary least squares, we can set the engine to be &lt;code&gt;lm&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;linear_reg&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;lm&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Linear Regression Model Specification (regression)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Computational engine: lm&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/linear_reg.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;documentation page for &lt;code&gt;linear_reg()&lt;/code&gt;&lt;/a&gt; lists the possible engines. We&amp;rsquo;ll save this model object as &lt;code&gt;lm_mod&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;lm_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;linear_reg&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;lm&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;From here, the model can be estimated or trained using the 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/fit.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;fit()&lt;/code&gt;&lt;/a&gt; function:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;lm_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  lm_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(width &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; initial_volume &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; food_regime, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; urchins)
lm_fit
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  3ms &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; stats::lm(formula = formula, data = data)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Coefficients:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                    (Intercept)                  initial_volume  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                      0.0331216                       0.0015546  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                 food_regimeLow                 food_regimeHigh  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                      0.0197824                       0.0214111  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  initial_volume:food_regimeLow  initial_volume:food_regimeHigh  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                     -0.0012594                       0.0005254&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Perhaps our analysis requires a description of the model parameter estimates and their statistical properties. Although the &lt;code&gt;summary()&lt;/code&gt; function for &lt;code&gt;lm&lt;/code&gt; objects can provide that, it gives the results back in an unwieldy format. Many models have a &lt;code&gt;tidy()&lt;/code&gt; method that provides the summary results in a more predictable and useful format (e.g. a data frame with standard column names):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;tidy&lt;/span&gt;(lm_fit)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   term                            estimate std.error statistic  p.value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;                              &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 (Intercept)                     0.0331    0.00962      3.44  0.00100 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 initial_volume                  0.00155   0.000398     3.91  0.000222&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 food_regimeLow                  0.0198    0.0130       1.52  0.133   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 food_regimeHigh                 0.0214    0.0145       1.47  0.145   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 initial_volume:food_regimeLow  -0.00126   0.000510    -2.47  0.0162  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6 initial_volume:food_regimeHigh  0.000525  0.000702     0.748 0.457&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;predict-model&#34;&gt;Use a model to predict&lt;/h2&gt;
&lt;p&gt;This fitted object &lt;code&gt;lm_fit&lt;/code&gt; has the &lt;code&gt;lm&lt;/code&gt; model output built-in, which you can access with &lt;code&gt;lm_fit$fit&lt;/code&gt;, but there are some benefits to using the fitted parsnip model object when it comes to predicting.&lt;/p&gt;
&lt;p&gt;Suppose that, for a publication, it would be particularly interesting to make a plot of the mean body size for urchins that started the experiment with an initial volume of 20ml. To create such a graph, we start with some new example data that we will make predictions for, to show in our graph:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;new_points &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;expand.grid&lt;/span&gt;(initial_volume &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;20&lt;/span&gt;, 
                          food_regime &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Initial&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Low&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;High&amp;#34;&lt;/span&gt;))
new_points
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   initial_volume food_regime&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1             20     Initial&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2             20         Low&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3             20        High&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To get our predicted results, we can use the &lt;code&gt;predict()&lt;/code&gt; function to find the mean values at 20ml.&lt;/p&gt;
&lt;p&gt;It is also important to communicate the variability, so we also need to find the predicted confidence intervals. If we had used &lt;code&gt;lm()&lt;/code&gt; to fit the model directly, a few minutes of reading the 
&lt;a href=&#34;https://stat.ethz.ch/R-manual/R-devel/library/stats/html/predict.lm.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;documentation page&lt;/a&gt; for &lt;code&gt;predict.lm()&lt;/code&gt; would explain how to do this. However, if we decide to use a different model to estimate urchin size (&lt;em&gt;spoiler:&lt;/em&gt; we will!), it is likely that a completely different syntax would be required.&lt;/p&gt;
&lt;p&gt;Instead, with tidymodels, the types of predicted values are standardized so that we can use the same syntax to get these values.&lt;/p&gt;
&lt;p&gt;First, let&amp;rsquo;s generate the mean body width values:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;mean_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(lm_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; new_points)
mean_pred
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    .pred&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 0.0642&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 0.0588&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 0.0961&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;When making predictions, the tidymodels convention is to always produce a tibble of results with standardized column names. This makes it easy to combine the original data and the predictions in a usable format:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;conf_int_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(lm_fit, 
                         new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; new_points, 
                         type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;conf_int&amp;#34;&lt;/span&gt;)
conf_int_pred
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .pred_lower .pred_upper&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;         &amp;lt;dbl&amp;gt;       &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1      0.0555      0.0729&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2      0.0499      0.0678&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3      0.0870      0.105&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Now combine: &lt;/span&gt;
plot_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  new_points &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(mean_pred) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(conf_int_pred)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# and plot:&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(plot_data, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; food_regime)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_errorbar&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(ymin &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred_lower, 
                    ymax &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred_upper),
                width &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;labs&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;urchin size&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/lm-all-pred-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;h2 id=&#34;new-engine&#34;&gt;Model with a different engine&lt;/h2&gt;
&lt;p&gt;Every one on your team is happy with that plot &lt;em&gt;except&lt;/em&gt; that one person who just read their first book on 
&lt;a href=&#34;https://bayesian.org/what-is-bayesian-analysis/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Bayesian analysis&lt;/a&gt;. They are interested in knowing if the results would be different if the model were estimated using a Bayesian approach. In such an analysis, a 
&lt;a href=&#34;https://towardsdatascience.com/introduction-to-bayesian-linear-regression-e66e60791ea7&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;prior distribution&lt;/em&gt;&lt;/a&gt; needs to be declared for each model parameter that represents the possible values of the parameters (before being exposed to the observed data). After some discussion, the group agrees that the priors should be bell-shaped but, since no one has any idea what the range of values should be, to take a conservative approach and make the priors &lt;em&gt;wide&lt;/em&gt; using a Cauchy distribution (which is the same as a t-distribution with a single degree of freedom).&lt;/p&gt;
&lt;p&gt;The 
&lt;a href=&#34;https://mc-stan.org/rstanarm/articles/priors.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;documentation&lt;/a&gt; on the rstanarm package shows us that the &lt;code&gt;stan_glm()&lt;/code&gt; function can be used to estimate this model, and that the function arguments that need to be specified are called &lt;code&gt;prior&lt;/code&gt; and &lt;code&gt;prior_intercept&lt;/code&gt;. It turns out that &lt;code&gt;linear_reg()&lt;/code&gt; has a 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/linear_reg.html#details&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;stan&lt;/code&gt; engine&lt;/a&gt;. Since these prior distribution arguments are specific to the Stan software, they are passed as arguments to 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/set_engine.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;parsnip::set_engine()&lt;/code&gt;&lt;/a&gt;. After that, the same exact &lt;code&gt;fit()&lt;/code&gt; call is used:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# set the prior distribution&lt;/span&gt;
prior_dist &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; rstanarm&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;student_t&lt;/span&gt;(df &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;123&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# make the parsnip model&lt;/span&gt;
bayes_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;   
  &lt;span style=&#34;color:#00f&#34;&gt;linear_reg&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;stan&amp;#34;&lt;/span&gt;, 
             prior_intercept &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; prior_dist, 
             prior &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; prior_dist) 

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# train the model&lt;/span&gt;
bayes_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  bayes_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(width &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; initial_volume &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; food_regime, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; urchins)

&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(bayes_fit, digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  1.5s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; stan_glm&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  family:       gaussian [identity]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  formula:      width ~ initial_volume * food_regime&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  observations: 72&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  predictors:   6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ------&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                                Median   MAD_SD  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; (Intercept)                     0.03452  0.00883&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; initial_volume                  0.00150  0.00037&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; food_regimeLow                  0.01805  0.01221&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; food_regimeHigh                 0.01934  0.01367&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; initial_volume:food_regimeLow  -0.00119  0.00047&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; initial_volume:food_regimeHigh  0.00061  0.00065&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Auxiliary parameter(s):&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       Median  MAD_SD &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; sigma 0.02121 0.00186&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ------&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; * For help interpreting the printed output see ?print.stanreg&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; * For info on the priors used see ?prior_summary.stanreg&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This kind of Bayesian analysis (like many models) involves randomly generated numbers in its fitting procedure. We can use &lt;code&gt;set.seed()&lt;/code&gt; to ensure that the same (pseudo-)random numbers are generated each time we run this code. The number &lt;code&gt;123&lt;/code&gt; isn&amp;rsquo;t special or related to our data; it is just a &amp;ldquo;seed&amp;rdquo; used to choose random numbers.&lt;/p&gt;
&lt;p&gt;To update the parameter table, the &lt;code&gt;tidy()&lt;/code&gt; method is once again used:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;tidy&lt;/span&gt;(bayes_fit, intervals &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   term                            estimate std.error     lower     upper&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;                              &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 (Intercept)                     0.0345    0.00883   0.0200    0.0490  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 initial_volume                  0.00150   0.000369  0.000895  0.00212 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 food_regimeLow                  0.0181    0.0122   -0.00181   0.0380  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 food_regimeHigh                 0.0193    0.0137   -0.00317   0.0420  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 initial_volume:food_regimeLow  -0.00119   0.000472 -0.00199  -0.000413&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6 initial_volume:food_regimeHigh  0.000610  0.000651 -0.000490  0.00170&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;A goal of the tidymodels packages is that the &lt;strong&gt;interfaces to common tasks are standardized&lt;/strong&gt; (as seen in the &lt;code&gt;tidy()&lt;/code&gt; results above). The same is true for getting predictions; we can use the same code even though the underlying packages use very different syntax:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;bayes_plot_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  new_points &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(bayes_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; new_points)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(bayes_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; new_points, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;conf_int&amp;#34;&lt;/span&gt;))

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(bayes_plot_data, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; food_regime)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_errorbar&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(ymin &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred_lower, ymax &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred_upper), width &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;labs&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;urchin size&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ggtitle&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Bayesian model with t(1) prior distribution&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/stan-pred-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;This isn&amp;rsquo;t very different from the non-Bayesian results (except in interpretation).&lt;/p&gt;
&lt;div class=&#34;note&#34;&gt;The &lt;a href=&#34;https://parsnip.tidymodels.org/&#34;&gt;parsnip&lt;/a&gt; package can work with many model types, engines, and arguments. Check out &lt;a href=&#34;/find/parsnip/&#34;&gt;tidymodels.org/find/parsnip&lt;/a&gt; to see what is available.&lt;/div&gt;
&lt;h2 id=&#34;why&#34;&gt;Why does it work that way?&lt;/h2&gt;
&lt;p&gt;The extra step of defining the model using a function like &lt;code&gt;linear_reg()&lt;/code&gt; might seem superfluous since a call to &lt;code&gt;lm()&lt;/code&gt; is much more succinct. However, the problem with standard modeling functions is that they don&amp;rsquo;t separate what you want to do from the execution. For example, the process of executing a formula has to happen repeatedly across model calls even when the formula does not change; we can&amp;rsquo;t recycle those computations.&lt;/p&gt;
&lt;p&gt;Also, using the tidymodels framework, we can do some interesting things by incrementally creating a model (instead of using single function call). 
&lt;a href=&#34;/start/tuning/&#34;&gt;Model tuning&lt;/a&gt; with tidymodels uses the specification of the model to declare what parts of the model should be tuned. That would be very difficult to do if &lt;code&gt;linear_reg()&lt;/code&gt; immediately fit the model.&lt;/p&gt;
&lt;p&gt;If you are familiar with the tidyverse, you may have noticed that our modeling code uses the magrittr pipe (&lt;code&gt;%&amp;gt;%&lt;/code&gt;). With dplyr and other tidyverse packages, the pipe works well because all of the functions take the &lt;em&gt;data&lt;/em&gt; as the first argument. For example:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;urchins &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;group_by&lt;/span&gt;(food_regime) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;summarize&lt;/span&gt;(med_vol &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;median&lt;/span&gt;(initial_volume))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   food_regime med_vol&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt;         &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 Initial        20.5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 Low            19.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 High           15&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;whereas the modeling code uses the pipe to pass around the &lt;em&gt;model object&lt;/em&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;bayes_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(width &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; initial_volume &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; food_regime, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; urchins)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This may seem jarring if you have used dplyr a lot, but it is extremely similar to how ggplot2 operates:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(urchins,
       &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(initial_volume, width)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;      &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# returns a ggplot object &lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_jitter&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;                         &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# same&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_smooth&lt;/span&gt;(method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; lm, se &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# same                    &lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;labs&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Volume&amp;#34;&lt;/span&gt;, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Width&amp;#34;&lt;/span&gt;)         &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# etc&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;session-info&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 4.0.0 (2020-04-24)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin17.0          
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/New_York            
#&amp;gt;  date     2020-05-19                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.6   2020-04-20 [1] CRAN (R 4.0.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 4.0.0)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 4.0.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 4.0.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 4.0.0)
#&amp;gt;  parsnip    * 0.1.1   2020-05-06 [1] CRAN (R 4.0.0)
#&amp;gt;  purrr      * 0.3.4   2020-04-17 [1] CRAN (R 4.0.0)
#&amp;gt;  readr      * 1.3.1   2018-12-21 [1] CRAN (R 4.0.0)
#&amp;gt;  recipes    * 0.1.12  2020-05-01 [1] CRAN (R 4.0.0)
#&amp;gt;  rlang        0.4.6   2020-05-02 [1] CRAN (R 4.0.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 4.0.0)
#&amp;gt;  rstanarm   * 2.19.3  2020-02-11 [1] CRAN (R 4.0.0)
#&amp;gt;  tibble     * 3.0.1   2020-04-20 [1] CRAN (R 4.0.0)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 4.0.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 4.0.0)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 4.0.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 4.0.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Regression models two ways</title>
      <link>/learn/models/parsnip-ranger-glmnet/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/models/parsnip-ranger-glmnet/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: AmesHousing, glmnet, randomForest, ranger, and tidymodels.&lt;/p&gt;
&lt;p&gt;We can create regression models with the tidymodels package 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip&lt;/a&gt; to predict continuous or numeric quantities. Here, let&amp;rsquo;s first fit a random forest model, which does &lt;em&gt;not&lt;/em&gt; require all numeric input (see discussion 
&lt;a href=&#34;https://bookdown.org/max/FES/categorical-trees.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;) and discuss how to use &lt;code&gt;fit()&lt;/code&gt; and &lt;code&gt;fit_xy()&lt;/code&gt;, as well as &lt;em&gt;data descriptors&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Second, let&amp;rsquo;s fit a regularized linear regression model to demonstrate how to move between different types of models using parsnip.&lt;/p&gt;
&lt;h2 id=&#34;the-ames-housing-data&#34;&gt;The Ames housing data&lt;/h2&gt;
&lt;p&gt;We&amp;rsquo;ll use the Ames housing data set to demonstrate how to create regression models using parsnip. First, set up the data set and create a simple training/test set split:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(AmesHousing)
ames &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;make_ames&lt;/span&gt;()

&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;4595&lt;/span&gt;)
data_split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(ames, strata &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Sale_Price&amp;#34;&lt;/span&gt;, p &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.75&lt;/span&gt;)

ames_train &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(data_split)
ames_test  &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(data_split)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The use of the test set here is &lt;em&gt;only for illustration&lt;/em&gt;; normally in a data analysis these data would be saved to the very end after many models have been evaluated.&lt;/p&gt;
&lt;h2 id=&#34;random-forest&#34;&gt;Random forest&lt;/h2&gt;
&lt;p&gt;We&amp;rsquo;ll start by fitting a random forest model to a small set of parameters. Let&amp;rsquo;s create a model with the predictors &lt;code&gt;Longitude&lt;/code&gt;, &lt;code&gt;Latitude&lt;/code&gt;, &lt;code&gt;Lot_Area&lt;/code&gt;, &lt;code&gt;Neighborhood&lt;/code&gt;, and &lt;code&gt;Year_Sold&lt;/code&gt;. A simple random forest model can be specified via:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_defaults &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;rand_forest&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;regression&amp;#34;&lt;/span&gt;)
rf_defaults
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Random Forest Model Specification (regression)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The model will be fit with the ranger package by default. Since we didn&amp;rsquo;t add any extra arguments to &lt;code&gt;fit&lt;/code&gt;, &lt;em&gt;many&lt;/em&gt; of the arguments will be set to their defaults from the function  &lt;code&gt;ranger::ranger()&lt;/code&gt;. The help pages for the model function describe the default parameters and you can also use the &lt;code&gt;translate()&lt;/code&gt; function to check out such details.&lt;/p&gt;
&lt;p&gt;The parsnip package provides two different interfaces to fit a model:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the formula interface (&lt;code&gt;fit()&lt;/code&gt;), and&lt;/li&gt;
&lt;li&gt;the non-formula interface (&lt;code&gt;fit_xy()&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Let&amp;rsquo;s start with the non-formula interface:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;preds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Longitude&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Latitude&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Lot_Area&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Neighborhood&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Year_Sold&amp;#34;&lt;/span&gt;)

rf_xy_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  rf_defaults &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ranger&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit_xy&lt;/span&gt;(
    x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train[, preds],
    y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(ames_train&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;Sale_Price)
  )

rf_xy_fit
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  952ms &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Ranger result&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  ranger::ranger(formula = formula, data = data, num.threads = 1,      verbose = FALSE, seed = sample.int(10^5, 1)) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Type:                             Regression &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of trees:                  500 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Sample size:                      2199 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of independent variables:  5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Mtry:                             2 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Target node size:                 5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variable importance mode:         none &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Splitrule:                        variance &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; OOB prediction error (MSE):       0.00844 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; R squared (OOB):                  0.736&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The non-formula interface doesn&amp;rsquo;t do anything to the predictors before passing them to the underlying model function. This particular model does &lt;em&gt;not&lt;/em&gt; require indicator variables (sometimes called &amp;ldquo;dummy variables&amp;rdquo;) to be created prior to fitting the model. Note that the output shows &amp;ldquo;Number of independent variables:  5&amp;rdquo;.&lt;/p&gt;
&lt;p&gt;For regression models, we can use the basic &lt;code&gt;predict()&lt;/code&gt; method, which returns a tibble with a column named &lt;code&gt;.pred&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;test_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  ames_test &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(Sale_Price) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(Sale_Price &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Sale_Price)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(rf_xy_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_test[, preds])
  )
test_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;slice&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Sale_Price .pred&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1       5.33  5.22&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2       5.02  5.21&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3       5.27  5.25&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4       5.60  5.51&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5       5.28  5.24&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# summarize performance&lt;/span&gt;
test_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;metrics&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Sale_Price, estimate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred) 
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 rmse    standard      0.0914&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 rsq     standard      0.717 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 mae     standard      0.0662&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;If the model required indicator variables, we would have to create them manually prior to using &lt;code&gt;fit()&lt;/code&gt; (perhaps using the recipes package).&lt;/li&gt;
&lt;li&gt;We had to manually log the outcome prior to modeling.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Now, for illustration, let&amp;rsquo;s use the formula method using some new parameter values:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;rand_forest&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;regression&amp;#34;&lt;/span&gt;, mtry &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, trees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1000&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ranger&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Sale_Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Longitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Latitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Lot_Area &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Neighborhood &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Year_Sold,
    data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train
  )
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  2.6s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Ranger result&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  ranger::ranger(formula = formula, data = data, mtry = ~3, num.trees = ~1000,      num.threads = 1, verbose = FALSE, seed = sample.int(10^5,          1)) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Type:                             Regression &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of trees:                  1000 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Sample size:                      2199 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of independent variables:  5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Mtry:                             3 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Target node size:                 5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variable importance mode:         none &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Splitrule:                        variance &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; OOB prediction error (MSE):       0.00848 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; R squared (OOB):                  0.735&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Suppose that we would like to use the randomForest package instead of ranger. To do so, the only part of the syntax that needs to change is the &lt;code&gt;set_engine()&lt;/code&gt; argument:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;rand_forest&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;regression&amp;#34;&lt;/span&gt;, mtry &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, trees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1000&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;randomForest&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Sale_Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Longitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Latitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Lot_Area &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Neighborhood &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Year_Sold,
    data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train
  )
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  2.1s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  randomForest(x = as.data.frame(x), y = y, ntree = ~1000, mtry = ~3) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                Type of random forest: regression&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                      Number of trees: 1000&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; No. of variables tried at each split: 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;           Mean of squared residuals: 0.013&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;                     % Var explained: 59.4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Look at the formula code that was printed out; one function uses the argument name &lt;code&gt;ntree&lt;/code&gt; and the other uses &lt;code&gt;num.trees&lt;/code&gt;. The parsnip models don&amp;rsquo;t require you to know the specific names of the main arguments.&lt;/p&gt;
&lt;p&gt;Now suppose that we want to modify the value of &lt;code&gt;mtry&lt;/code&gt; based on the number of predictors in the data. Usually, a good default value is &lt;code&gt;floor(sqrt(num_predictors))&lt;/code&gt; but a pure bagging model requires an &lt;code&gt;mtry&lt;/code&gt; value equal to the total number of parameters. There may be cases where you may not know how many predictors are going to be present when the model will be fit (perhaps due to the generation of indicator variables or a variable filter) so this might be difficult to know exactly ahead of time when you write your code.&lt;/p&gt;
&lt;p&gt;When the model it being fit by parsnip, 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/reference/descriptors.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;data descriptors&lt;/em&gt;&lt;/a&gt; are made available. These attempt to let you know what you will have available when the model is fit. When a model object is created (say using &lt;code&gt;rand_forest()&lt;/code&gt;), the values of the arguments that you give it are &lt;em&gt;immediately evaluated&lt;/em&gt; unless you delay them. To delay the evaluation of any argument, you can used &lt;code&gt;rlang::expr()&lt;/code&gt; to make an expression.&lt;/p&gt;
&lt;p&gt;Two relevant data descriptors for our example model are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;.preds()&lt;/code&gt;: the number of predictor &lt;em&gt;variables&lt;/em&gt; in the data set that are associated with the predictors &lt;strong&gt;prior to dummy variable creation&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;.cols()&lt;/code&gt;: the number of predictor &lt;em&gt;columns&lt;/em&gt; after dummy variables (or other encodings) are created.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Since ranger won&amp;rsquo;t create indicator values, &lt;code&gt;.preds()&lt;/code&gt; would be appropriate for &lt;code&gt;mtry&lt;/code&gt; for a bagging model.&lt;/p&gt;
&lt;p&gt;For example, let&amp;rsquo;s use an expression with the &lt;code&gt;.preds()&lt;/code&gt; descriptor to fit a bagging model:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;rand_forest&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;regression&amp;#34;&lt;/span&gt;, mtry &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;.preds&lt;/span&gt;(), trees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1000&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ranger&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Sale_Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Longitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Latitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Lot_Area &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Neighborhood &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Year_Sold,
    data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train
  )
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  3.6s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Ranger result&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  ranger::ranger(formula = formula, data = data, mtry = ~.preds(),      num.trees = ~1000, num.threads = 1, verbose = FALSE, seed = sample.int(10^5,          1)) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Type:                             Regression &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of trees:                  1000 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Sample size:                      2199 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of independent variables:  5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Mtry:                             5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Target node size:                 5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variable importance mode:         none &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Splitrule:                        variance &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; OOB prediction error (MSE):       0.00869 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; R squared (OOB):                  0.728&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;regularized-regression&#34;&gt;Regularized regression&lt;/h2&gt;
&lt;p&gt;A linear model might work for this data set as well. We can use the &lt;code&gt;linear_reg()&lt;/code&gt; parsnip model. There are two engines that can perform regularization/penalization, the glmnet and sparklyr packages. Let&amp;rsquo;s use the former here. The glmnet package only implements a non-formula method, but parsnip will allow either one to be used.&lt;/p&gt;
&lt;p&gt;When regularization is used, the predictors should first be centered and scaled before being passed to the model. The formula method won&amp;rsquo;t do that automatically so we will need to do this ourselves. We&amp;rsquo;ll use the 
&lt;a href=&#34;https://tidymodels.github.io/recipes/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipes&lt;/a&gt; package for these steps.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;norm_recipe &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(
    Sale_Price &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Longitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Latitude &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Lot_Area &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Neighborhood &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; Year_Sold, 
    data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train
  ) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_other&lt;/span&gt;(Neighborhood) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_nominal&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_center&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_scale&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_log&lt;/span&gt;(Sale_Price, base &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# estimate the means and standard deviations&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;(training &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_train, retain &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Now let&amp;#39;s fit the model using the processed version of the data&lt;/span&gt;

glmn_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;linear_reg&lt;/span&gt;(penalty &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.001&lt;/span&gt;, mixture &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;glmnet&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(Sale_Price &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;(norm_recipe))
glmn_fit
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  13ms &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:  glmnet::glmnet(x = as.matrix(x), y = y, family = &amp;#34;gaussian&amp;#34;,      alpha = ~0.5) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    Df  %Dev Lambda&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1   0 0.000 0.1370&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2   1 0.019 0.1250&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3   1 0.036 0.1140&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4   1 0.050 0.1040&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5   2 0.068 0.0946&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6   4 0.093 0.0862&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 7   5 0.125 0.0785&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 8   5 0.153 0.0716&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 9   7 0.184 0.0652&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10  7 0.214 0.0594&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 11  7 0.240 0.0541&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 12  8 0.262 0.0493&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 13  8 0.286 0.0449&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 14  8 0.306 0.0409&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 15  8 0.323 0.0373&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 16  8 0.338 0.0340&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 17  8 0.350 0.0310&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 18  8 0.361 0.0282&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 19  9 0.370 0.0257&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 20  9 0.379 0.0234&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 21  9 0.386 0.0213&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 22  9 0.392 0.0195&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 23  9 0.397 0.0177&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 24  9 0.401 0.0161&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 25  9 0.405 0.0147&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 26  9 0.408 0.0134&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 27 10 0.410 0.0122&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 28 11 0.413 0.0111&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 29 11 0.415 0.0101&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 30 11 0.417 0.0092&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 31 12 0.418 0.0084&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 32 12 0.420 0.0077&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 33 12 0.421 0.0070&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 34 12 0.422 0.0064&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 35 12 0.423 0.0058&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 36 12 0.423 0.0053&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 37 12 0.424 0.0048&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 38 12 0.425 0.0044&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 39 12 0.425 0.0040&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 40 12 0.425 0.0036&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 41 12 0.426 0.0033&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 42 12 0.426 0.0030&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 43 12 0.426 0.0028&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 44 12 0.426 0.0025&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 45 12 0.426 0.0023&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 46 12 0.426 0.0021&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 47 12 0.427 0.0019&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 48 12 0.427 0.0017&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 49 12 0.427 0.0016&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 50 12 0.427 0.0014&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 51 12 0.427 0.0013&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 52 12 0.427 0.0012&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 53 12 0.427 0.0011&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 54 12 0.427 0.0010&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 55 12 0.427 0.0009&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 56 12 0.427 0.0008&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 57 12 0.427 0.0008&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 58 12 0.427 0.0007&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 59 12 0.427 0.0006&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 60 12 0.427 0.0006&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 61 12 0.427 0.0005&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 62 12 0.427 0.0005&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 63 12 0.427 0.0004&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 64 12 0.427 0.0004&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 65 12 0.427 0.0004&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If &lt;code&gt;penalty&lt;/code&gt; were not specified, all of the &lt;code&gt;lambda&lt;/code&gt; values would be computed.&lt;/p&gt;
&lt;p&gt;To get the predictions for this specific value of &lt;code&gt;lambda&lt;/code&gt; (aka &lt;code&gt;penalty&lt;/code&gt;):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# First, get the processed version of the test set predictors:&lt;/span&gt;
test_normalized &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bake&lt;/span&gt;(norm_recipe, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; ames_test, &lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())

test_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  test_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;rename&lt;/span&gt;(`random forest` &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(glmn_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; test_normalized) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
      &lt;span style=&#34;color:#00f&#34;&gt;rename&lt;/span&gt;(glmnet &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred)
  )
test_results
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 731 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    Sale_Price `random forest` glmnet&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;         &amp;lt;dbl&amp;gt;           &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1       5.33            5.22   5.27&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2       5.02            5.21   5.17&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3       5.27            5.25   5.23&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4       5.60            5.51   5.25&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5       5.28            5.24   5.25&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6       5.17            5.19   5.19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7       5.02            4.97   5.19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8       5.46            5.50   5.49&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9       5.44            5.46   5.48&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10       5.33            5.50   5.47&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 721 more rows&lt;/span&gt;

test_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;metrics&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Sale_Price, estimate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; glmnet) 
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 rmse    standard      0.132 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 rsq     standard      0.410 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 mae     standard      0.0956&lt;/span&gt;

test_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;gather&lt;/span&gt;(model, prediction, &lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;Sale_Price) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; prediction, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Sale_Price)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_abline&lt;/span&gt;(col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;green&amp;#34;&lt;/span&gt;, lty &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.4&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;facet_wrap&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;model) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;coord_fixed&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/glmn-pred-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;This final plot compares the performance of the random forest and regularized regression models.&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package      * version date       lib source        
#&amp;gt;  AmesHousing  * 0.0.3   2017-12-17 [1] CRAN (R 3.6.0)
#&amp;gt;  broom        * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials        * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr        * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2      * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  glmnet       * 3.0-2   2019-12-11 [1] CRAN (R 3.6.0)
#&amp;gt;  infer        * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip      * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr        * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  randomForest * 4.6-14  2018-03-25 [1] CRAN (R 3.6.0)
#&amp;gt;  ranger       * 0.12.1  2020-01-10 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes      * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang          0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample      * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble       * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels   * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune         * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows    * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick    * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Classification models using a neural network</title>
      <link>/learn/models/parsnip-nnet/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/models/parsnip-nnet/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: keras and tidymodels. You will also need the python keras library installed (see &lt;code&gt;?keras::install_keras()&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;We can create classification models with the tidymodels package 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip&lt;/a&gt; to predict categorical quantities or class labels. Here, let&amp;rsquo;s fit a single classification model using a neural network and evaluate using a validation set. While the 
&lt;a href=&#34;https://tidymodels.github.io/tune/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tune&lt;/a&gt; package has functionality to also do this, the parsnip package is the center of attention in this article so that we can better understand its usage.&lt;/p&gt;
&lt;h2 id=&#34;fitting-a-neural-network&#34;&gt;Fitting a neural network&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s fit a model to a small, two predictor classification data set. The data are in the modeldata package (part of tidymodels) and have been split into training, validation, and test data sets. In this analysis, the test set is left untouched; this article tries to emulate a good data usage methodology where the test set would only be evaluated once at the end after a variety of models have been considered.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(bivariate)
&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(bivariate_train)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1009&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(bivariate_val)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 300&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;A plot of the data shows two right-skewed predictors:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(bivariate_train, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; A, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; B, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Class)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.2&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/biv-plot-1.svg&#34; width=&#34;576&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s use a single hidden layer neural network to predict the outcome. To do this, we transform the predictor columns to be more symmetric (via the &lt;code&gt;step_BoxCox()&lt;/code&gt; function) and on a common scale (using &lt;code&gt;step_normalize()&lt;/code&gt;). We can use 
&lt;a href=&#34;https://tidymodels.github.io/recipes/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipes&lt;/a&gt; to do so:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;biv_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(Class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bivariate_train) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_BoxCox&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())&lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;(training &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bivariate_train, retain &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# We will juice() to get the processed training set back&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# For validation:&lt;/span&gt;
val_normalized &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bake&lt;/span&gt;(biv_rec, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bivariate_val, &lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# For testing when we arrive at a final model: &lt;/span&gt;
test_normalized &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bake&lt;/span&gt;(biv_rec, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bivariate_test, &lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We can use the keras package to fit a model with 5 hidden units and a 10% dropout rate, to regularize the model:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;57974&lt;/span&gt;)
nnet_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mlp&lt;/span&gt;(epochs &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;, hidden_units &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, dropout &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.1&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_mode&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;classification&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Also set engine-specific `verbose` argument to prevent logging the results: &lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;keras&amp;#34;&lt;/span&gt;, verbose &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(Class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;(biv_rec))

nnet_fit
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  8.7s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Model: &amp;#34;sequential&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ________________________________________________________________________________&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Layer (type)                        Output Shape                    Param #     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ================================================================================&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dense (Dense)                       (None, 5)                       15          &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ________________________________________________________________________________&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dense_1 (Dense)                     (None, 5)                       30          &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ________________________________________________________________________________&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dropout (Dropout)                   (None, 5)                       0           &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ________________________________________________________________________________&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dense_2 (Dense)                     (None, 2)                       12          &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ================================================================================&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Total params: 57&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Trainable params: 57&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Non-trainable params: 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ________________________________________________________________________________&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;model-performance&#34;&gt;Model performance&lt;/h2&gt;
&lt;p&gt;In parsnip, the &lt;code&gt;predict()&lt;/code&gt; function can be used to characterize performance on the validation set. Since parsnip always produces tibble outputs, these can just be column bound to the original data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;val_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  bivariate_val &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(nnet_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; val_normalized),
    &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(nnet_fit, new_data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; val_normalized, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;prob&amp;#34;&lt;/span&gt;)
  )
val_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;slice&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5 x 6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       A     B Class .pred_class .pred_One .pred_Two&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;fct&amp;gt; &amp;lt;fct&amp;gt;           &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 1061.  74.5 One   Two             0.473    0.527 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 1241.  83.4 One   Two             0.484    0.516 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3  939.  71.9 One   One             0.636    0.364 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4  813.  77.1 One   One             0.925    0.0746&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 1706.  92.8 Two   Two             0.355    0.645&lt;/span&gt;

val_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;roc_auc&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Class, .pred_One)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 roc_auc binary         0.815&lt;/span&gt;

val_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;accuracy&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Class, .pred_class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric  .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 accuracy binary         0.737&lt;/span&gt;

val_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;conf_mat&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Class, .pred_class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;           Truth&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Prediction One Two&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        One 150  27&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        Two  52  71&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let&amp;rsquo;s also create a grid to get a visual sense of the class boundary for the validation set.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;a_rng &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;range&lt;/span&gt;(bivariate_train&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;A)
b_rng &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;range&lt;/span&gt;(bivariate_train&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;B)
x_grid &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;expand.grid&lt;/span&gt;(A &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(a_rng[1], a_rng[2], length.out &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;),
              B &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(b_rng[1], b_rng[2], length.out &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;))
x_grid_trans &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bake&lt;/span&gt;(biv_rec, x_grid)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Make predictions using the transformed predictors but &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# attach them to the predictors in the original units: &lt;/span&gt;
x_grid &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  x_grid &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(nnet_fit, x_grid_trans, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;prob&amp;#34;&lt;/span&gt;))

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(x_grid, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; A, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; B)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_contour&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(z &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred_One), breaks &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.5&lt;/span&gt;, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;black&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bivariate_val, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Class), alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.3&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/biv-boundary-1.svg&#34; width=&#34;576&#34; /&gt;&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  keras        2.2.5.0 2019-10-08 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Nested resampling</title>
      <link>/learn/work/nested-resampling/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/work/nested-resampling/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: furrr, kernlab, mlbench, scales, and tidymodels.&lt;/p&gt;
&lt;p&gt;In this article, we discuss an alternative method for evaluating and tuning models, called 
&lt;a href=&#34;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C7&amp;amp;q=%22nested&amp;#43;resampling%22&amp;#43;inner&amp;#43;outer&amp;amp;btnG=&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;nested resampling&lt;/a&gt;. While it is more computationally taxing and challenging to implement than other resampling methods, it has the potential to produce better estimates of model performance.&lt;/p&gt;
&lt;h2 id=&#34;resampling-models&#34;&gt;Resampling models&lt;/h2&gt;
&lt;p&gt;A typical scheme for splitting the data when developing a predictive model is to create an initial split of the data into a training and test set. If resampling is used, it is executed on the training set. A series of binary splits is created. In rsample, we use the term &lt;em&gt;analysis set&lt;/em&gt; for the data that are used to fit the model and the term &lt;em&gt;assessment set&lt;/em&gt; for the set used to compute performance:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;figs/resampling.svg&#34; width=&#34;70%&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;A common method for tuning models is 
&lt;a href=&#34;/learn/work/tune-svm/&#34;&gt;grid search&lt;/a&gt; where a candidate set of tuning parameters is created. The full set of models for every combination of the tuning parameter grid and the resamples is fitted. Each time, the assessment data are used to measure performance and the average value is determined for each tuning parameter.&lt;/p&gt;
&lt;p&gt;The potential problem is that once we pick the tuning parameter associated with the best performance, this performance value is usually quoted as the performance of the model. There is serious potential for &lt;em&gt;optimization bias&lt;/em&gt; since we use the same data to tune the model and to assess performance. This would result in an optimistic estimate of performance.&lt;/p&gt;
&lt;p&gt;Nested resampling uses an additional layer of resampling that separates the tuning activities from the process used to estimate the efficacy of the model. An &lt;em&gt;outer&lt;/em&gt; resampling scheme is used and, for every split in the outer resample, another full set of resampling splits are created on the original analysis set. For example, if 10-fold cross-validation is used on the outside and 5-fold cross-validation on the inside, a total of 500 models will be fit. The parameter tuning will be conducted 10 times and the best parameters are determined from the average of the 5 assessment sets. This process occurs 10 times.&lt;/p&gt;
&lt;p&gt;Once the tuning results are complete, a model is fit to each of the outer resampling splits using the best parameter associated with that resample. The average of the outer method&amp;rsquo;s assessment sets are a unbiased estimate of the model.&lt;/p&gt;
&lt;p&gt;We will simulate some regression data to illustrate the methods. The mlbench package has a function &lt;code&gt;mlbench::mlbench.friedman1()&lt;/code&gt; that can simulate a complex regression data structure from the 
&lt;a href=&#34;https://scholar.google.com/scholar?hl=en&amp;amp;q=%22Multivariate&amp;#43;adaptive&amp;#43;regression&amp;#43;splines%22&amp;amp;btnG=&amp;amp;as_sdt=1%2C7&amp;amp;as_sdtp=&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;original MARS publication&lt;/a&gt;. A training set size of 100 data points are generated as well as a large set that will be used to characterize how well the resampling procedure performed.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(mlbench)
sim_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(n) {
  tmp &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;mlbench.friedman1&lt;/span&gt;(n, sd &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
  tmp &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;cbind&lt;/span&gt;(tmp&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;x, tmp&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;y)
  tmp &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.data.frame&lt;/span&gt;(tmp)
  &lt;span style=&#34;color:#00f&#34;&gt;names&lt;/span&gt;(tmp)&lt;span style=&#34;color:#00f&#34;&gt;[ncol&lt;/span&gt;(tmp)] &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;y&amp;#34;&lt;/span&gt;
  tmp
}

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;9815&lt;/span&gt;)
train_dat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;sim_data&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;)
large_dat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;sim_data&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;^5)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;nested-resampling&#34;&gt;Nested resampling&lt;/h2&gt;
&lt;p&gt;To get started, the types of resampling methods need to be specified. This isn&amp;rsquo;t a large data set, so 5 repeats of 10-fold cross validation will be used as the &lt;em&gt;outer&lt;/em&gt; resampling method for generating the estimate of overall performance. To tune the model, it would be good to have precise estimates for each of the values of the tuning parameter so let&amp;rsquo;s use 25 iterations of the bootstrap. This means that there will eventually be &lt;code&gt;5 * 10 * 25 = 1250&lt;/code&gt; models that are fit to the data &lt;em&gt;per tuning parameter&lt;/em&gt;. These models will be discarded once the performance of the model has been quantified.&lt;/p&gt;
&lt;p&gt;To create the tibble with the resampling specifications:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)
results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;nested_cv&lt;/span&gt;(train_dat, 
                     outside &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vfold_cv&lt;/span&gt;(repeats &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;), 
                     inside &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bootstraps&lt;/span&gt;(times &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;25&lt;/span&gt;))
results
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;nested_cv&amp;#34;  &amp;#34;vfold_cv&amp;#34;   &amp;#34;rset&amp;#34;       &amp;#34;tbl_df&amp;#34;     &amp;#34;tbl&amp;#34;       &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [6] &amp;#34;data.frame&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # Nested resampling:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #  outer: 10-fold cross-validation repeated 5 times&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #  inner: Bootstrap sampling&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 50 x 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits          id      id2    inner_resamples  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;named list&amp;gt;    &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;  &amp;lt;named list&amp;gt;     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold01 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold02 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold03 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold04 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold05 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold06 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold07 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold08 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold09 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [90/10]&amp;gt; Repeat1 Fold10 &amp;lt;tibble [25 × 2]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 40 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The splitting information for each resample is contained in the &lt;code&gt;split&lt;/code&gt; objects. Focusing on the second fold of the first repeat:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;splits[[2]]
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &amp;lt;Training/Validation/Total&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &amp;lt;90/10/100&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;&amp;lt;90/10/100&amp;gt;&lt;/code&gt; indicates the number of observations in the analysis set, assessment set, and the original data.&lt;/p&gt;
&lt;p&gt;Each element of &lt;code&gt;inner_resamples&lt;/code&gt; has its own tibble with the bootstrapping splits.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;inner_resamples[[5]]
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # Bootstrap sampling &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 25 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits          id         &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;list&amp;gt;          &amp;lt;chr&amp;gt;      &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [90/31]&amp;gt; Bootstrap01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [90/33]&amp;gt; Bootstrap02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [90/37]&amp;gt; Bootstrap03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [90/31]&amp;gt; Bootstrap04&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [90/32]&amp;gt; Bootstrap05&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [90/32]&amp;gt; Bootstrap06&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [90/36]&amp;gt; Bootstrap07&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [90/34]&amp;gt; Bootstrap08&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [90/29]&amp;gt; Bootstrap09&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [90/31]&amp;gt; Bootstrap10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 15 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;These are self-contained, meaning that the bootstrap sample is aware that it is a sample of a specific 90% of the data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;inner_resamples[[5]]&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;splits[[1]]
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &amp;lt;Training/Validation/Total&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &amp;lt;90/31/90&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To start, we need to define how the model will be created and measured. Let&amp;rsquo;s use a radial basis support vector machine model via the function &lt;code&gt;kernlab::ksvm&lt;/code&gt;. This model is generally considered to have &lt;em&gt;two&lt;/em&gt; tuning parameters: the SVM cost value and the kernel parameter &lt;code&gt;sigma&lt;/code&gt;. For illustration purposes here, only the cost value will be tuned and the function &lt;code&gt;kernlab::sigest&lt;/code&gt; will be used to estimate &lt;code&gt;sigma&lt;/code&gt; during each model fit. This is automatically done by &lt;code&gt;ksvm&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;After the model is fit to the analysis set, the root-mean squared error (RMSE) is computed on the assessment set. &lt;strong&gt;One important note:&lt;/strong&gt; for this model, it is critical to center and scale the predictors before computing dot products. We don&amp;rsquo;t do this operation here because &lt;code&gt;mlbench.friedman1&lt;/code&gt; simulates all of the predictors to be standardized uniform random variables.&lt;/p&gt;
&lt;p&gt;Our function to fit the model and compute the RMSE is:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(kernlab)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# `object` will be an `rsplit` object from our `results` tibble&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# `cost` is the tuning parameter&lt;/span&gt;
svm_rmse &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(object, cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;) {
  y_col &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;ncol&lt;/span&gt;(object&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;data)
  mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;svm_rbf&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;regression&amp;#34;&lt;/span&gt;, cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;kernlab&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;analysis&lt;/span&gt;(object))
  
  holdout_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(mod, &lt;span style=&#34;color:#00f&#34;&gt;assessment&lt;/span&gt;(object) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; dplyr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;y)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;assessment&lt;/span&gt;(object) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; dplyr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(y))
  &lt;span style=&#34;color:#00f&#34;&gt;rmse&lt;/span&gt;(holdout_pred, truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; y, estimate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred)&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;.estimate
}

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# In some case, we want to parameterize the function over the tuning parameter:&lt;/span&gt;
rmse_wrapper &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(cost, object) &lt;span style=&#34;color:#00f&#34;&gt;svm_rmse&lt;/span&gt;(object, cost)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For the nested resampling, a model needs to be fit for each tuning parameter and each bootstrap split. To do this, create a wrapper:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# `object` will be an `rsplit` object for the bootstrap samples&lt;/span&gt;
tune_over_cost &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(object) {
  &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt; ^ &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;8&lt;/span&gt;, by &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(RMSE &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map_dbl&lt;/span&gt;(cost, rmse_wrapper, object &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; object))
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Since this will be called across the set of outer cross-validation splits, another wrapper is required:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# `object` is an `rsplit` object in `results$inner_resamples` &lt;/span&gt;
summarize_tune_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(object) {
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Return row-bound tibble that has the 25 bootstrap results&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;map_df&lt;/span&gt;(object&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;splits, tune_over_cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# For each value of the tuning parameter, compute the &lt;/span&gt;
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# average RMSE which is the inner bootstrap estimate. &lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;group_by&lt;/span&gt;(cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;summarize&lt;/span&gt;(mean_RMSE &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(RMSE, na.rm &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;),
              n &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;length&lt;/span&gt;(RMSE))
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now that those functions are defined, we can execute all the inner resampling loops:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;tuning_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;inner_resamples, summarize_tune_results) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Alternatively, since these computations can be run in parallel, we can use the furrr package. Instead of using &lt;code&gt;map()&lt;/code&gt;, the function &lt;code&gt;future_map()&lt;/code&gt; parallelizes the iterations using the 
&lt;a href=&#34;https://cran.r-project.org/web/packages/future/vignettes/future-1-overview.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;future package&lt;/a&gt;. The &lt;code&gt;multisession&lt;/code&gt; plan uses the local cores to process the inner resampling loop. The end results are the same as the sequential computations.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(furrr)
&lt;span style=&#34;color:#00f&#34;&gt;plan&lt;/span&gt;(multisession)

tuning_results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;future_map&lt;/span&gt;(results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;inner_resamples, summarize_tune_results) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The object &lt;code&gt;tuning_results&lt;/code&gt; is a list of data frames for each of the 50 outer resamples.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s make a plot of the averaged results to see what the relationship is between the RMSE and the tuning parameters for each of the inner bootstrapping operations:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(scales)

pooled_inner &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; tuning_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; bind_rows

best_cost &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(dat) dat&lt;span style=&#34;color:#00f&#34;&gt;[which.min&lt;/span&gt;(dat&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;mean_RMSE),]

p &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(pooled_inner, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cost, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; mean_RMSE)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;scale_x_continuous&lt;/span&gt;(trans &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;log2&amp;#39;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;xlab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;SVM Cost&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;ylab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Inner RMSE&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;for &lt;/span&gt;(i in &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;length&lt;/span&gt;(tuning_results))
  p &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; p  &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;(data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; tuning_results[[i]], alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;best_cost&lt;/span&gt;(tuning_results[[i]]), pch &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;16&lt;/span&gt;, alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)

p &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; p &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;geom_smooth&lt;/span&gt;(data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; pooled_inner, se &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
p
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/rmse-plot-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Each gray line is a separate bootstrap resampling curve created from a different 90% of the data. The blue line is a LOESS smooth of all the results pooled together.&lt;/p&gt;
&lt;p&gt;To determine the best parameter estimate for each of the outer resampling iterations:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cost_vals &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  tuning_results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;map_df&lt;/span&gt;(best_cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(cost)

results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(results, cost_vals) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;factor&lt;/span&gt;(cost, levels &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;paste&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt; ^ &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;8&lt;/span&gt;, by &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))))

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(results, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cost)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_bar&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;xlab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;SVM Cost&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;scale_x_discrete&lt;/span&gt;(drop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/choose-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Most of the resamples produced an optimal cost value of 2.0, but the distribution is right-skewed due to the flat trend in the resampling profile once the cost value becomes 10 or larger.&lt;/p&gt;
&lt;p&gt;Now that we have these estimates, we can compute the outer resampling results for each of the 50 splits using the corresponding tuning parameter value:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;results &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  results &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(RMSE &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map2_dbl&lt;/span&gt;(splits, cost, svm_rmse))

&lt;span style=&#34;color:#00f&#34;&gt;summary&lt;/span&gt;(results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;RMSE)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    1.57    2.09    2.68    2.69    3.25    4.25&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The estimated RMSE for the model tuning process is 2.69.&lt;/p&gt;
&lt;p&gt;What is the RMSE estimate for the non-nested procedure when only the outer resampling method is used? For each cost value in the tuning grid, 50 SVM models are fit and their RMSE values are averaged. The table of cost values and mean RMSE estimates is used to determine the best cost value. The associated RMSE is the biased estimate.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;not_nested &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(results&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;splits, tune_over_cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  bind_rows

outer_summary &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; not_nested &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;group_by&lt;/span&gt;(cost) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;summarize&lt;/span&gt;(outer_RMSE &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(RMSE), n &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;length&lt;/span&gt;(RMSE))

outer_summary
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 11 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      cost outer_RMSE     n&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1   0.25       3.54    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2   0.5        3.11    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3   1          2.77    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4   2          2.62    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5   4          2.65    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6   8          2.75    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7  16          2.82    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8  32          2.82    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9  64          2.83    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 128          2.83    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 11 256          2.82    50&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(outer_summary, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cost, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; outer_RMSE)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;scale_x_continuous&lt;/span&gt;(trans &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;log2&amp;#39;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;xlab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;SVM Cost&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;ylab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;RMSE&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/not-nested-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;The non-nested procedure estimates the RMSE to be 2.62. Both estimates are fairly close.&lt;/p&gt;
&lt;p&gt;The approximately true RMSE for an SVM model with a cost value of 2.0 can be approximated with the large sample that was simulated at the beginning.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;finalModel &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;ksvm&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_dat, C &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
large_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(finalModel, large_dat[, &lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;ncol&lt;/span&gt;(large_dat)])
&lt;span style=&#34;color:#00f&#34;&gt;sqrt&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;((large_dat&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;y &lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt; large_pred) ^ &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, na.rm &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.71&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The nested procedure produces a closer estimate to the approximate truth but the non-nested estimate is very similar.&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  furrr      * 0.1.0   2018-05-16 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  kernlab    * 0.9-29  2019-11-12 [1] CRAN (R 3.6.0)
#&amp;gt;  mlbench    * 2.1-1   2012-07-10 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  scales     * 1.1.0   2019-11-18 [1] CRAN (R 3.6.0)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Preprocess your data with recipes</title>
      <link>/start/recipes/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/start/recipes/</guid>
      <description>&lt;h2 id=&#34;intro&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;In our 
&lt;a href=&#34;/start/models/&#34;&gt;&lt;em&gt;Build a Model&lt;/em&gt;&lt;/a&gt; article, we learned how to specify and train models with different engines using the 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip package&lt;/a&gt;. In this article, we&amp;rsquo;ll explore another tidymodels package, 
&lt;a href=&#34;https://tidymodels.github.io/recipes/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipes&lt;/a&gt;, which is designed to help you preprocess your data &lt;em&gt;before&lt;/em&gt; training your model. Recipes are built as a series of preprocessing steps, such as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;converting qualitative predictors to indicator variables (also known as dummy variables),&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;transforming data to be on a different scale (e.g., taking the logarithm of a variable),&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;transforming whole groups of predictors together,&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;extracting key features from raw variables (e.g., getting the day of the week out of a date variable),&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;and so on. If you are familiar with R&amp;rsquo;s formula interface, a lot of this might sound familiar and like what a formula already does. Recipes can be used to do many of the same things, but they have a much wider range of possibilities. This article shows how to use recipes for modeling.&lt;/p&gt;
&lt;p&gt;To use code in this article,  you will need to install the following packages: nycflights13, skimr, and tidymodels.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)      &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for the recipes package, along with the rest of tidymodels&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Helper packages&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(nycflights13)    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for flight data&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(skimr)           &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for variable summaries&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;data&#34;&gt;The New York City flight data&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s use the 
&lt;a href=&#34;https://github.com/hadley/nycflights13&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;nycflights13 data&lt;/a&gt; to predict whether a plane arrives more than 30 minutes late. This data set contains information on 325,819 flights departing near New York City in 2013. Let&amp;rsquo;s start by loading the data and making a few changes to the variables:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;123&lt;/span&gt;)

flight_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  flights &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Convert the arrival delay to a factor&lt;/span&gt;
    arr_delay &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;ifelse&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;&amp;gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;30&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;late&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;on_time&amp;#34;&lt;/span&gt;),
    arr_delay &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;factor&lt;/span&gt;(arr_delay),
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# We will use the date (not date-time) in the recipe below&lt;/span&gt;
    date &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.Date&lt;/span&gt;(time_hour)
  ) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Include the weather data&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;inner_join&lt;/span&gt;(weather, by &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;origin&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;time_hour&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Only retain the specific columns we will use&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(dep_time, flight, origin, dest, air_time, distance, 
         carrier, date, arr_delay, time_hour) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Exclude missing data&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;na.omit&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# For creating models, it is better to have qualitative columns&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# encoded as factors (instead of character strings)&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate_if&lt;/span&gt;(is.character, as.factor)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We can see that about 16% of the flights in this data set arrived more than 30 minutes late.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flight_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;count&lt;/span&gt;(arr_delay) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; n&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(n))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   arr_delay      n  prop&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt;      &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 late       52540 0.161&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 on_time   273279 0.839&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Before we start building up our recipe, let&amp;rsquo;s take a quick look at a few specific variables that will be important for both preprocessing and modeling.&lt;/p&gt;
&lt;p&gt;First, notice that the variable we created called &lt;code&gt;arr_delay&lt;/code&gt; is a factor variable; it is important that our outcome variable for training a logistic regression model is a factor.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;glimpse&lt;/span&gt;(flight_data)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Observations: 325,819&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variables: 10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ dep_time  &amp;lt;int&amp;gt; 517, 533, 542, 544, 554, 554, 555, 557, 557, 558, 558, 558,…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ flight    &amp;lt;int&amp;gt; 1545, 1714, 1141, 725, 461, 1696, 507, 5708, 79, 301, 49, 7…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ origin    &amp;lt;fct&amp;gt; EWR, LGA, JFK, JFK, LGA, EWR, EWR, LGA, JFK, LGA, JFK, JFK,…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ dest      &amp;lt;fct&amp;gt; IAH, IAH, MIA, BQN, ATL, ORD, FLL, IAD, MCO, ORD, PBI, TPA,…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ air_time  &amp;lt;dbl&amp;gt; 227, 227, 160, 183, 116, 150, 158, 53, 140, 138, 149, 158, …&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ distance  &amp;lt;dbl&amp;gt; 1400, 1416, 1089, 1576, 762, 719, 1065, 229, 944, 733, 1028…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ carrier   &amp;lt;fct&amp;gt; UA, UA, AA, B6, DL, UA, B6, EV, B6, AA, B6, B6, UA, UA, AA,…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ date      &amp;lt;date&amp;gt; 2013-01-01, 2013-01-01, 2013-01-01, 2013-01-01, 2013-01-01…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ arr_delay &amp;lt;fct&amp;gt; on_time, on_time, late, on_time, on_time, on_time, on_time,…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ time_hour &amp;lt;dttm&amp;gt; 2013-01-01 05:00:00, 2013-01-01 05:00:00, 2013-01-01 05:00…&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Second, there are two variables that we don&amp;rsquo;t want to use as predictors in our model, but that we would like to retain as identification variables that can be used to troubleshoot poorly predicted data points. These are &lt;code&gt;flight&lt;/code&gt;, a numeric value, and &lt;code&gt;time_hour&lt;/code&gt;, a date-time value.&lt;/p&gt;
&lt;p&gt;Third, there are 104 flight destinations contained in &lt;code&gt;dest&lt;/code&gt; and 16 distinct &lt;code&gt;carrier&lt;/code&gt;s.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flight_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  skimr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;skim&lt;/span&gt;(dest, carrier) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;table style=&#39;width: auto;&#39;
        class=&#39;table table-condensed&#39;&gt;
&lt;caption&gt;Table 1: Data summary&lt;/caption&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt;   &lt;/th&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt;   &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Name &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Piped data &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Number of rows &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; 325819 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Number of columns &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; 10 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; _______________________ &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt;  &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Column type frequency: &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt;  &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; factor &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; 2 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; ________________________ &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt;  &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Group variables &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; None &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;Variable type: factor&lt;/strong&gt;&lt;/p&gt;
&lt;table&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt; skim_variable &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; n_missing &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; complete_rate &lt;/th&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt; ordered &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; n_unique &lt;/th&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt; top_counts &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; dest &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 1 &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; FALSE &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 104 &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; ATL: 16771, ORD: 16507, LAX: 15942, BOS: 14948 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; carrier &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 1 &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; FALSE &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 16 &lt;/td&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; UA: 57489, B6: 53715, EV: 50868, DL: 47465 &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Because we&amp;rsquo;ll be using a simple logistic regression model, the variables &lt;code&gt;dest&lt;/code&gt; and &lt;code&gt;carrier&lt;/code&gt; will be converted to 
&lt;a href=&#34;https://bookdown.org/max/FES/creating-dummy-variables-for-unordered-categories.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dummy variables&lt;/a&gt;. However, some of these values do not occur very frequently and this could complicate our analysis. We&amp;rsquo;ll discuss specific steps later in this article that we can add to our recipe to address this issue before modeling.&lt;/p&gt;
&lt;h2 id=&#34;data-split&#34;&gt;Data splitting&lt;/h2&gt;
&lt;p&gt;To get started, let&amp;rsquo;s split this single dataset into two: a &lt;em&gt;training&lt;/em&gt; set and a &lt;em&gt;testing&lt;/em&gt; set. We&amp;rsquo;ll keep most of the rows in the original dataset (subset chosen randomly) in the &lt;em&gt;training&lt;/em&gt; set. The training data will be used to &lt;em&gt;fit&lt;/em&gt; the model, and the &lt;em&gt;testing&lt;/em&gt; set will be used to measure model performance.&lt;/p&gt;
&lt;p&gt;To do this, we can use the 
&lt;a href=&#34;https://tidymodels.github.io/rsample/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rsample&lt;/a&gt; package to create an object that contains the information on &lt;em&gt;how&lt;/em&gt; to split the data, and then two more rsample functions to create data frames for the training and testing sets:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Fix the random numbers by setting the seed &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# This enables the analysis to be reproducible when random numbers are used &lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;555&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Put 3/4 of the data into the training set &lt;/span&gt;
data_split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(flight_data, prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Create data frames for the two sets:&lt;/span&gt;
train_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(data_split)
test_data  &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(data_split)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;recipe&#34;&gt;Create recipe and roles&lt;/h2&gt;
&lt;p&gt;To get started, let&amp;rsquo;s create a recipe for a simple logistic regression model. Before training the model, we can use a recipe to create a few new predictors and conduct some preprocessing required by the model.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s initiate a new recipe:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/recipe.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;recipe()&lt;/code&gt; function&lt;/a&gt; as we used it here has two arguments:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;A &lt;strong&gt;formula&lt;/strong&gt;. Any variable on the left-hand side of the tilde (&lt;code&gt;~&lt;/code&gt;) is considered the model outcome (here, &lt;code&gt;arr_delay&lt;/code&gt;). On the right-hand side of the tilde are the predictors. Variables may be listed by name, or you can use the dot (&lt;code&gt;.&lt;/code&gt;) to indicate all other variables as predictors.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The &lt;strong&gt;data&lt;/strong&gt;. A recipe is associated with the data set used to create the model. This will typically be the &lt;em&gt;training&lt;/em&gt; set, so &lt;code&gt;data = train_data&lt;/code&gt; here. Naming a data set doesn&amp;rsquo;t actually change the data itself; it is only used to catalog the names of the variables and their types, like factors, integers, dates, etc.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Now we can add 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/roles.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;roles&lt;/a&gt; to this recipe. We can use the 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/roles.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;update_role()&lt;/code&gt; function&lt;/a&gt; to let recipes know that &lt;code&gt;flight&lt;/code&gt; and &lt;code&gt;time_hour&lt;/code&gt; are variables with a custom role that we called &lt;code&gt;&amp;quot;ID&amp;quot;&lt;/code&gt; (a role can have any character value). Whereas our formula included all variables in the training set other than &lt;code&gt;arr_delay&lt;/code&gt; as predictors, this tells the recipe to keep these two variables but not use them as either outcomes or predictors.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;update_role&lt;/span&gt;(flight, time_hour, new_role &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ID&amp;#34;&lt;/span&gt;) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This step of adding roles to a recipe is optional; the purpose of using it here is that those two variables can be retained in the data but not included in the model. This can be convenient when, after the model is fit, we want to investigate some poorly predicted value. These ID columns will be available and can be used to try to understand what went wrong.&lt;/p&gt;
&lt;p&gt;To get the current set of variables and roles, use the &lt;code&gt;summary()&lt;/code&gt; function:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;summary&lt;/span&gt;(flights_rec)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 10 x 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    variable  type    role      source  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt;   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 dep_time  numeric predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 flight    numeric ID        original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 origin    nominal predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 dest      nominal predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 air_time  numeric predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 distance  numeric predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 carrier   nominal predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 date      date    predictor original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 time_hour date    ID        original&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 arr_delay nominal outcome   original&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;features&#34;&gt;Create features&lt;/h2&gt;
&lt;p&gt;Now we can start adding steps onto our recipe using the pipe operator. Perhaps it is reasonable for the date of the flight to have an effect on the likelihood of a late arrival. A little bit of &lt;strong&gt;feature engineering&lt;/strong&gt; might go a long way to improving our model. How should the date be encoded into the model? The &lt;code&gt;date&lt;/code&gt; column has an R &lt;code&gt;date&lt;/code&gt; object so including that column &amp;ldquo;as is&amp;rdquo; will mean that the model will convert it to a numeric format equal to the number of days after a reference date:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flight_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;distinct&lt;/span&gt;(date) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(numeric_date &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.numeric&lt;/span&gt;(date)) 
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 364 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   date       numeric_date&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;date&amp;gt;            &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2013-01-01        15706&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 2013-01-02        15707&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 2013-01-03        15708&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 2013-01-04        15709&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 2013-01-05        15710&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 359 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;It&amp;rsquo;s possible that the numeric date variable is a good option for modeling; perhaps the model would benefit from a linear trend between the log-odds of a late arrival and the numeric date variable. However, it might be better to add model terms &lt;em&gt;derived&lt;/em&gt; from the date that have a better potential to be important to the model. For example, we could derive the following meaningful features from the single &lt;code&gt;date&lt;/code&gt; variable:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;the day of the week,&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;the month, and&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;whether or not the date corresponds to a holiday.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Let&amp;rsquo;s do all three of these by adding steps to our recipe:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;update_role&lt;/span&gt;(flight, time_hour, new_role &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ID&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_date&lt;/span&gt;(date, features &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;dow&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;               
  &lt;span style=&#34;color:#00f&#34;&gt;step_holiday&lt;/span&gt;(date, holidays &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; timeDate&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;listHolidays&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;US&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_rm&lt;/span&gt;(date)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;What do each of these steps do?&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;With 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/step_date.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;step_date()&lt;/code&gt;&lt;/a&gt;, we created two new factor columns with the appropriate day of the week and the month.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;With 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/step_holiday.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;step_holiday()&lt;/code&gt;&lt;/a&gt;, we created a binary variable indicating whether the current date is a holiday or not. The argument value of &lt;code&gt;timeDate::listHolidays(&amp;quot;US&amp;quot;)&lt;/code&gt; uses the 
&lt;a href=&#34;https://cran.r-project.org/web/packages/timeDate/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;timeDate package&lt;/a&gt; to list the 17 standard US holidays.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;With 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/step_rm.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;step_rm()&lt;/code&gt;&lt;/a&gt;, we remove the original &lt;code&gt;date&lt;/code&gt; variable since we no longer want it in the model.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Next, we&amp;rsquo;ll turn our attention to the variable types of our predictors. Because we plan to train a logistic regression model, we know that predictors will ultimately need to be numeric, as opposed to factor variables. In other words, there may be a difference in how we store our data (in factors inside a data frame), and how the underlying equations require them (a purely numeric matrix).&lt;/p&gt;
&lt;p&gt;For factors like &lt;code&gt;dest&lt;/code&gt; and &lt;code&gt;origin&lt;/code&gt;, 
&lt;a href=&#34;https://bookdown.org/max/FES/creating-dummy-variables-for-unordered-categories.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;standard practice&lt;/a&gt; is to convert them into &lt;em&gt;dummy&lt;/em&gt; or &lt;em&gt;indicator&lt;/em&gt; variables to make them numeric. These are binary values for each level of the factor. For example, our &lt;code&gt;origin&lt;/code&gt; variable has values of &lt;code&gt;&amp;quot;EWR&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;JFK&amp;quot;&lt;/code&gt;, and &lt;code&gt;&amp;quot;LGA&amp;quot;&lt;/code&gt;. The standard dummy variable encoding, shown below, will create &lt;em&gt;two&lt;/em&gt; numeric columns of the data that are 1 when the originating airport is &lt;code&gt;&amp;quot;JFK&amp;quot;&lt;/code&gt; or &lt;code&gt;&amp;quot;LGA&amp;quot;&lt;/code&gt; and zero otherwise, respectively.&lt;/p&gt;
&lt;table class=&#34;table&#34; style=&#34;width: auto !important; margin-left: auto; margin-right: auto;&#34;&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt; origin &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; origin_JFK &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; origin_LGA &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; EWR &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; JFK &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 1 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; LGA &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 1 &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;But, unlike the standard model formula methods in R, a recipe &lt;strong&gt;does not&lt;/strong&gt; automatically create these dummy variables for you; you&amp;rsquo;ll need to tell your recipe to add this step. This is for two reasons. First, many models do not require 
&lt;a href=&#34;https://bookdown.org/max/FES/categorical-trees.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;numeric predictors&lt;/a&gt;, so dummy variables may not always be preferred. Second, recipes can also be used for purposes outside of modeling, where non-dummy versions of the variables may work better. For example, you may want to make a table or a plot with a variable as a single factor. For those reasons, you need to explicitly tell recipes to create dummy variables using &lt;code&gt;step_dummy()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;update_role&lt;/span&gt;(flight, time_hour, new_role &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ID&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_date&lt;/span&gt;(date, features &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;dow&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_holiday&lt;/span&gt;(date, holidays &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; timeDate&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;listHolidays&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;US&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_rm&lt;/span&gt;(date) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_nominal&lt;/span&gt;(), &lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;all_outcomes&lt;/span&gt;())
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Here, we did something different than before: instead of applying a step to an individual variable, we used 
&lt;a href=&#34;https://tidymodels.github.io/recipes/reference/selections.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;selectors&lt;/a&gt; to apply this recipe step to several variables at once.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The first selector, &lt;code&gt;all_nominal()&lt;/code&gt;, selects all variables that are either factors or characters.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The second selector, &lt;code&gt;-all_outcomes()&lt;/code&gt; removes any outcome variables from this recipe step.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With these two selectors together, our recipe step above translates to:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Create dummy variables for all of the factor or character columns &lt;em&gt;unless&lt;/em&gt; they are outcomes.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;At this stage in the recipe, this step selects the &lt;code&gt;origin&lt;/code&gt;, &lt;code&gt;dest&lt;/code&gt;, and &lt;code&gt;carrier&lt;/code&gt; variables. It also includes two new variables, &lt;code&gt;date_dow&lt;/code&gt; and &lt;code&gt;date_month&lt;/code&gt;, that were created by the earlier &lt;code&gt;step_date()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;More generally, the recipe selectors mean that you don&amp;rsquo;t always have to apply steps to individual variables one at a time. Since a recipe knows the &lt;em&gt;variable type&lt;/em&gt; and &lt;em&gt;role&lt;/em&gt; of each column, they can also be selected (or dropped) using this information.&lt;/p&gt;
&lt;p&gt;We need one final step to add to our recipe. Since &lt;code&gt;carrier&lt;/code&gt; and &lt;code&gt;dest&lt;/code&gt; have some infrequently occurring values, it is possible that dummy variables might be created for values that don&amp;rsquo;t exist in the training set. For example, there is one destination that is only in the test set:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;test_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;distinct&lt;/span&gt;(dest) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;anti_join&lt;/span&gt;(train_data)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Joining, by = &amp;#34;dest&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   dest &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 LEX&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;When the recipe is applied to the training set, a column is made for LEX but it will contain all zeros. This is a &amp;ldquo;zero-variance predictor&amp;rdquo; that has no information within the column. While some R functions will not produce an error for such predictors, it usually causes warnings and other issues. &lt;code&gt;step_zv()&lt;/code&gt; will remove columns from the data when the training set data have a single value, so it is added to the recipe &lt;em&gt;after&lt;/em&gt; &lt;code&gt;step_dummy()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(arr_delay &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;update_role&lt;/span&gt;(flight, time_hour, new_role &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ID&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_date&lt;/span&gt;(date, features &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;dow&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_holiday&lt;/span&gt;(date, holidays &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; timeDate&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;listHolidays&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;US&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_rm&lt;/span&gt;(date) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_nominal&lt;/span&gt;(), &lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;all_outcomes&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_zv&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now we&amp;rsquo;ve created a &lt;em&gt;specification&lt;/em&gt; of what should be done with the data. How do we use the recipe we made?&lt;/p&gt;
&lt;h2 id=&#34;fit-workflow&#34;&gt;Fit a model with a recipe&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s use logistic regression to model the flight data. As we saw in 
&lt;a href=&#34;/start/models/&#34;&gt;&lt;em&gt;Build a Model&lt;/em&gt;&lt;/a&gt;, we start by 
&lt;a href=&#34;/start/models/#build-model&#34;&gt;building a model specification&lt;/a&gt; using the parsnip package:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;lr_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;logistic_reg&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;glm&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We will want to use our recipe across several steps as we train and test our model. We will:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Process the recipe using the training set&lt;/strong&gt;: This involves any estimation or calculations based on the training set. For our recipe, the training set will be used to determine which predictors should be converted to dummy variables and which predictors will have zero-variance in the training set, and should be slated for removal.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Apply the recipe to the training set&lt;/strong&gt;: We create the final predictor set on the training set.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Apply the recipe to the test set&lt;/strong&gt;: We create the final predictor set on the test set. Nothing is recomputed and no information from the test set is used here; the dummy variable and zero-variance results from the training set are applied to the test set.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;To simplify this process, we can use a &lt;em&gt;model workflow&lt;/em&gt;, which pairs a model and recipe together. This is a straightforward approach because different recipes are often needed for different models, so when a model and recipe are bundled, it becomes easier to train and test &lt;em&gt;workflows&lt;/em&gt;. We&amp;rsquo;ll use the 
&lt;a href=&#34;https://tidymodels.github.io/workflows/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;workflows package&lt;/a&gt; from tidymodels to bundle our parsnip model (&lt;code&gt;lr_mod&lt;/code&gt;) with our recipe (&lt;code&gt;flights_rec&lt;/code&gt;).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_wflow &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;workflow&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;add_model&lt;/span&gt;(lr_mod) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;add_recipe&lt;/span&gt;(flights_rec)
flights_wflow
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ══ Workflow ═════════════════════════════════════════════════════════════&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Preprocessor: Recipe&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Model: logistic_reg()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Preprocessor ─────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 Recipe Steps&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ● step_date()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ● step_holiday()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ● step_rm()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ● step_dummy()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ● step_zv()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Model ────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Logistic Regression Model Specification (classification)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Computational engine: glm&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, there is a single function that can be used to prepare the recipe and train the model from the resulting predictors:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  flights_wflow &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; train_data)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This object has the finalized recipe and fitted model objects inside. You may want to extract the model or recipe objects from the workflow. To do this, you can use the helper functions &lt;code&gt;pull_workflow_fit()&lt;/code&gt; and &lt;code&gt;pull_workflow_prepped_recipe()&lt;/code&gt;. For example, here we pull the fitted model object then use the &lt;code&gt;broom::tidy()&lt;/code&gt; function to get a tidy tibble of model coefficients:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_fit &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;pull_workflow_fit&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;tidy&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 157 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   term                estimate std.error statistic  p.value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;                  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 (Intercept)          3.91    2.73           1.43 1.51e- 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 dep_time            -0.00167 0.0000141   -118.   0.      &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 air_time            -0.0439  0.000561     -78.4  0.      &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 distance             0.00686 0.00150        4.57 4.84e- 6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 date_USChristmasDay  1.12    0.173          6.49 8.45e-11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 152 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;predict-workflow&#34;&gt;Use a trained workflow to predict&lt;/h2&gt;
&lt;p&gt;Our goal was to predict whether a plane arrives more than 30 minutes late. We have just:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Built the model (&lt;code&gt;lr_mod&lt;/code&gt;),&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Created a preprocessing recipe (&lt;code&gt;flights_rec&lt;/code&gt;),&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Bundled the model and recipe (&lt;code&gt;flights_wflow&lt;/code&gt;), and&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Trained our workflow using a single call to &lt;code&gt;fit()&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The next step is to use the trained workflow (&lt;code&gt;flights_fit&lt;/code&gt;) to predict with the unseen test data, which we will do with a single call to &lt;code&gt;predict()&lt;/code&gt;. The &lt;code&gt;predict()&lt;/code&gt; method applies the recipe to the new data, then passes them to the fitted model.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(flights_fit, test_data)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 81,454 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .pred_class&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt;      &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 on_time    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 on_time    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 on_time    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 on_time    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 on_time    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 8.145e+04 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Because our outcome variable here is a factor, the output from &lt;code&gt;predict()&lt;/code&gt; returns the predicted class: &lt;code&gt;late&lt;/code&gt; versus &lt;code&gt;on_time&lt;/code&gt;. But, let&amp;rsquo;s say we want the predicted class probabilities for each flight instead. To return those, we can specify &lt;code&gt;type = &amp;quot;prob&amp;quot;&lt;/code&gt; when we use &lt;code&gt;predict()&lt;/code&gt;. We&amp;rsquo;ll also bind the output with some variables from the test data and save them together:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(flights_fit, test_data, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;prob&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(test_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(arr_delay, time_hour, flight)) 

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# The data look like: &lt;/span&gt;
flights_pred
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 81,454 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .pred_late .pred_on_time arr_delay time_hour           flight&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;dbl&amp;gt;         &amp;lt;dbl&amp;gt; &amp;lt;fct&amp;gt;     &amp;lt;dttm&amp;gt;               &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     0.0565         0.944 on_time   2013-01-01 05:00:00   1714&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     0.0264         0.974 on_time   2013-01-01 06:00:00     79&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     0.0481         0.952 on_time   2013-01-01 06:00:00    301&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     0.0325         0.967 on_time   2013-01-01 06:00:00     49&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5     0.0711         0.929 on_time   2013-01-01 06:00:00   1187&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 8.145e+04 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now that we have a tibble with our predicted class probabilities, how will we evaluate the performance of our workflow? We can see from these first few rows that our model predicted these 5 on time flights correctly because the values of &lt;code&gt;.pred_on_time&lt;/code&gt; are &lt;em&gt;p&lt;/em&gt; &amp;gt; .50. But we also know that we have 81,454 rows total to predict. We would like to calculate a metric that tells how well our model predicted late arrivals, compared to the true status of our outcome variable, &lt;code&gt;arr_delay&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s use the area under the 
&lt;a href=&#34;https://bookdown.org/max/FES/measuring-performance.html#class-metrics&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ROC curve&lt;/a&gt; as our metric, computed using &lt;code&gt;roc_curve()&lt;/code&gt; and &lt;code&gt;roc_auc()&lt;/code&gt; from the 
&lt;a href=&#34;https://tidymodels.github.io/yardstick/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;yardstick package&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;To generate a ROC curve, we need the predicted class probabilities for &lt;code&gt;late&lt;/code&gt; and &lt;code&gt;on_time&lt;/code&gt;, which we just calculated in the code chunk above. We can create the ROC curve with these values, using &lt;code&gt;roc_curve()&lt;/code&gt; and then piping to the &lt;code&gt;autoplot()&lt;/code&gt; method:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;roc_curve&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; arr_delay, .pred_late) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;autoplot&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/roc-plot-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Similarly, &lt;code&gt;roc_auc()&lt;/code&gt; estimates the area under the curve:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;flights_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;roc_auc&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; arr_delay, .pred_late)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 roc_auc binary         0.765&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Not too bad! We leave it to the reader to test out this workflow 
&lt;a href=&#34;https://tidymodels.github.io/workflows/reference/add_formula.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;without&lt;/em&gt;&lt;/a&gt; this recipe. You can use &lt;code&gt;workflows::add_formula(arr_delay ~ .)&lt;/code&gt; instead of &lt;code&gt;add_recipe()&lt;/code&gt; (remember to remove the identification variables first!), and see whether our recipe improved our model&amp;rsquo;s ability to predict late arrivals.&lt;/p&gt;
&lt;h2 id=&#34;session-info&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-20                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package      * version date       lib source        
#&amp;gt;  broom        * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials        * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr        * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2      * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer        * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  nycflights13 * 1.0.1   2019-09-16 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip      * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr        * 0.3.4   2020-04-17 [1] CRAN (R 3.6.2)
#&amp;gt;  recipes      * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang          0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample      * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  skimr        * 2.1.1   2020-04-16 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble       * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels   * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune         * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows    * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick    * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Bootstrap resampling and tidy regression models</title>
      <link>/learn/statistics/bootstrap/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/statistics/bootstrap/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: tidymodels and tidyr.&lt;/p&gt;
&lt;p&gt;Combining fitted models in a tidy way is useful for performing bootstrapping or permutation tests. These approaches have been explored before, for instance by 
&lt;a href=&#34;https://rstudio-pubs-static.s3.amazonaws.com/19698_a4c472606e3c43e4b94720506e49bb7b.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Andrew MacDonald here&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/hadley/dplyr/issues/269&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hadley has explored efficient support for bootstrapping&lt;/a&gt; as a potential enhancement to dplyr. The tidymodels package 
&lt;a href=&#34;https://broom.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;broom&lt;/a&gt; fits naturally with 
&lt;a href=&#34;https://dplyr.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dplyr&lt;/a&gt; in performing these analyses.&lt;/p&gt;
&lt;p&gt;Bootstrapping consists of randomly sampling a data set with replacement, then performing the analysis individually on each bootstrapped replicate. The variation in the resulting estimate is then a reasonable approximation of the variance in our estimate.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s say we want to fit a nonlinear model to the weight/mileage relationship in the &lt;code&gt;mtcars&lt;/code&gt; data set.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(mtcars, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(mpg, wt)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-1-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We might use the method of nonlinear least squares (via the &lt;code&gt;nls()&lt;/code&gt; function) to fit a model.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;nlsfit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;nls&lt;/span&gt;(mpg &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; k &lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt; wt &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; b, mtcars, start &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(k &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;))
&lt;span style=&#34;color:#00f&#34;&gt;summary&lt;/span&gt;(nlsfit)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Formula: mpg ~ k/wt + b&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Parameters:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Estimate Std. Error t value Pr(&amp;gt;|t|)    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; k    45.83       4.25   10.79  7.6e-12 ***&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; b     4.39       1.54    2.85   0.0077 ** &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ---&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Residual standard error: 2.77 on 30 degrees of freedom&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of iterations to convergence: 1 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Achieved convergence tolerance: 2.88e-08&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(mtcars, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(wt, mpg)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(nlsfit)))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-2-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;While this does provide a p-value and confidence intervals for the parameters, these are based on model assumptions that may not hold in real data. Bootstrapping is a popular method for providing confidence intervals and predictions that are more robust to the nature of the data.&lt;/p&gt;
&lt;h2 id=&#34;bootstrapping-models&#34;&gt;Bootstrapping models&lt;/h2&gt;
&lt;p&gt;We can use the &lt;code&gt;bootstraps()&lt;/code&gt; function in the rsample package to sample bootstrap replications. First, we construct 2000 bootstrap replicates of the data, each of which has been randomly sampled with replacement. The resulting object is an &lt;code&gt;rset&lt;/code&gt;, which is a data frame with a column of &lt;code&gt;rsplit&lt;/code&gt; objects.&lt;/p&gt;
&lt;p&gt;An &lt;code&gt;rsplit&lt;/code&gt; object has two main components: an analysis data set and an assessment data set, accessible via &lt;code&gt;analysis(rsplit)&lt;/code&gt; and &lt;code&gt;assessment(rsplit)&lt;/code&gt; respectively. For bootstrap samples, the analysis data set is the bootstrap sample itself, and the assessment data set consists of all the out-of-bag samples.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;27&lt;/span&gt;)
boots &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;bootstraps&lt;/span&gt;(mtcars, times &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;, apparent &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
boots
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # Bootstrap sampling with apparent sample &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,001 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits          id           &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;list&amp;gt;          &amp;lt;chr&amp;gt;        &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [32/13]&amp;gt; Bootstrap0001&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [32/10]&amp;gt; Bootstrap0002&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [32/13]&amp;gt; Bootstrap0003&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [32/11]&amp;gt; Bootstrap0004&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [32/9]&amp;gt;  Bootstrap0005&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [32/10]&amp;gt; Bootstrap0006&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [32/11]&amp;gt; Bootstrap0007&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [32/13]&amp;gt; Bootstrap0008&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [32/11]&amp;gt; Bootstrap0009&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [32/11]&amp;gt; Bootstrap0010&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 1,991 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Let&amp;rsquo;s create a helper function to fit an &lt;code&gt;nls()&lt;/code&gt; model on each bootstrap sample, and then use &lt;code&gt;purrr::map()&lt;/code&gt; to apply this function to all the bootstrap samples at once. Similarly, we create a column of tidy coefficient information by unnesting.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;fit_nls_on_bootstrap &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(split) {
    &lt;span style=&#34;color:#00f&#34;&gt;nls&lt;/span&gt;(mpg &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; k &lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt; wt &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; b, &lt;span style=&#34;color:#00f&#34;&gt;analysis&lt;/span&gt;(split), start &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(k &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;))
}

boot_models &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  boots &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(model &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(splits, fit_nls_on_bootstrap),
         coef_info &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(model, tidy))

&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidyr)
boot_coefs &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  boot_models &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;unnest&lt;/span&gt;(coef_info)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The unnested coefficient information contains a summary of each replication combined in a single data frame:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;boot_coefs
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4,002 x 8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits         id           model term  estimate std.error statistic  p.value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;list&amp;gt;         &amp;lt;chr&amp;gt;        &amp;lt;lis&amp;gt; &amp;lt;chr&amp;gt;    &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [32/13… Bootstrap00… &amp;lt;nls&amp;gt; k        42.1       4.05     10.4  1.91e-11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [32/13… Bootstrap00… &amp;lt;nls&amp;gt; b         5.39      1.43      3.78 6.93e- 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [32/10… Bootstrap00… &amp;lt;nls&amp;gt; k        49.9       5.66      8.82 7.82e-10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [32/10… Bootstrap00… &amp;lt;nls&amp;gt; b         3.73      1.92      1.94 6.13e- 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [32/13… Bootstrap00… &amp;lt;nls&amp;gt; k        37.8       2.68     14.1  9.01e-15&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [32/13… Bootstrap00… &amp;lt;nls&amp;gt; b         6.73      1.17      5.75 2.78e- 6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [32/11… Bootstrap00… &amp;lt;nls&amp;gt; k        45.6       4.45     10.2  2.70e-11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [32/11… Bootstrap00… &amp;lt;nls&amp;gt; b         4.75      1.62      2.93 6.38e- 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [32/9]&amp;gt; Bootstrap00… &amp;lt;nls&amp;gt; k        43.6       4.63      9.41 1.85e-10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [32/9]&amp;gt; Bootstrap00… &amp;lt;nls&amp;gt; b         5.89      1.68      3.51 1.44e- 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 3,992 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;confidence-intervals&#34;&gt;Confidence intervals&lt;/h2&gt;
&lt;p&gt;We can then calculate confidence intervals (using what is called the 
&lt;a href=&#34;https://www.uvm.edu/~dhowell/StatPages/Randomization%20Tests/ResamplingWithR/BootstMeans/bootstrapping_means.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;percentile method&lt;/a&gt;):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;percentile_intervals &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;int_pctl&lt;/span&gt;(boot_models, coef_info)
percentile_intervals
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   term   .lower .estimate .upper .alpha .method   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 b      0.0475      4.12   7.31   0.05 percentile&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 k     37.6        46.7   59.8    0.05 percentile&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Or we can use histograms to get a more detailed idea of the uncertainty in each estimate:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(boot_coefs, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(estimate)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_histogram&lt;/span&gt;(bins &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;30&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;facet_wrap&lt;/span&gt;( &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; term, scales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;free&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_vline&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(xintercept &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .lower), data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; percentile_intervals, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;blue&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_vline&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(xintercept &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .upper), data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; percentile_intervals, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;blue&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-6-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;The rsample package also has functions for 
&lt;a href=&#34;https://tidymodels.github.io/rsample/reference/int_pctl.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;other types of confidence intervals&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;possible-model-fits&#34;&gt;Possible model fits&lt;/h2&gt;
&lt;p&gt;We can use &lt;code&gt;augment()&lt;/code&gt; to visualize the uncertainty in the fitted curve. Since there are so many bootstrap samples, we&amp;rsquo;ll only show a sample of the model fits in our visualization:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;boot_aug &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  boot_models &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;sample_n&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;200&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(augmented &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(model, augment)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;unnest&lt;/span&gt;(augmented)

boot_aug
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6,400 x 8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits         id            model coef_info         mpg    wt .fitted .resid&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;list&amp;gt;         &amp;lt;chr&amp;gt;         &amp;lt;lis&amp;gt; &amp;lt;list&amp;gt;          &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  16.4  4.07    15.6  0.829&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  19.7  2.77    21.9 -2.21 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  19.2  3.84    16.4  2.84 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  21.4  2.78    21.8 -0.437&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  26    2.14    27.8 -1.75 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  33.9  1.84    32.0  1.88 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  32.4  2.2     27.0  5.35 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  30.4  1.62    36.1 -5.70 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  21.5  2.46    24.4 -2.86 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [32/11… Bootstrap1644 &amp;lt;nls&amp;gt; &amp;lt;tibble [2 × 5…  26    2.14    27.8 -1.75 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 6,390 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(boot_aug, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(wt, mpg)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .fitted, group &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; id), alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.2&lt;/span&gt;, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;blue&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-8-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;With only a few small changes, we could easily perform bootstrapping with other kinds of predictive or hypothesis testing models, since the &lt;code&gt;tidy()&lt;/code&gt; and &lt;code&gt;augment()&lt;/code&gt; functions works for many statistical outputs. As another example, we could use &lt;code&gt;smooth.spline()&lt;/code&gt;, which fits a cubic smoothing spline to data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;fit_spline_on_bootstrap &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(split) {
    data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;analysis&lt;/span&gt;(split)
    &lt;span style=&#34;color:#00f&#34;&gt;smooth.spline&lt;/span&gt;(data&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;wt, data&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;mpg, df &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
}

boot_splines &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  boots &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;sample_n&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;200&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(spline &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(splits, fit_spline_on_bootstrap),
         aug_train &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(spline, augment))

splines_aug &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  boot_splines &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;unnest&lt;/span&gt;(aug_train)

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(splines_aug, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x, y)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .fitted, group &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; id), alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.2&lt;/span&gt;, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;blue&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-9-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tidyr      * 1.0.2   2020-01-24 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Evaluate your model with resampling</title>
      <link>/start/resampling/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/start/resampling/</guid>
      <description>&lt;h2 id=&#34;intro&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;So far, we have 
&lt;a href=&#34;/start/models/&#34;&gt;built a model&lt;/a&gt; and 
&lt;a href=&#34;/start/recipes/&#34;&gt;preprocessed data with a recipe&lt;/a&gt;. We also introduced 
&lt;a href=&#34;/start/recipes/#fit-workflow&#34;&gt;workflows&lt;/a&gt; as a way to bundle a 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip model&lt;/a&gt; and 
&lt;a href=&#34;https://tidymodels.github.io/recipes/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipe&lt;/a&gt; together. Once we have a model trained, we need a way to measure how well that model predicts new data. This tutorial explains how to characterize model performance based on &lt;strong&gt;resampling&lt;/strong&gt; statistics.&lt;/p&gt;
&lt;p&gt;To use code in this article,  you will need to install the following packages: modeldata, ranger, and tidymodels.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels) &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for the rsample package, along with the rest of tidymodels&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Helper packages&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(modeldata)  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for the cells data&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;data&#34;&gt;The cell image data&lt;/h2&gt;
&lt;p&gt;Let&amp;rsquo;s use data from 
&lt;a href=&#34;http://www.biomedcentral.com/1471-2105/8/340&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Hill, LaPan, Li, and Haney (2007)&lt;/a&gt;, available in the 
&lt;a href=&#34;https://cran.r-project.org/web/packages/modeldata/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;modeldata package&lt;/a&gt;, to predict cell image segmentation quality with resampling. To start, we load this data into R:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(cells, package &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;modeldata&amp;#34;&lt;/span&gt;)
cells
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,019 x 58&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   case  class angle_ch_1 area_ch_1 avg_inten_ch_1 avg_inten_ch_2 avg_inten_ch_3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt; &amp;lt;fct&amp;gt;      &amp;lt;dbl&amp;gt;     &amp;lt;int&amp;gt;          &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 Test  PS        143.         185           15.7           4.95           9.55&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 Train PS        134.         819           31.9         207.            69.9 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 Train WS        107.         431           28.0         116.            63.9 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 Train PS         69.2        298           19.5         102.            28.2 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 Test  PS          2.89       285           24.3         112.            20.5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,014 more rows, and 51 more variables: avg_inten_ch_4 &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   convex_hull_area_ratio_ch_1 &amp;lt;dbl&amp;gt;, convex_hull_perim_ratio_ch_1 &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   diff_inten_density_ch_1 &amp;lt;dbl&amp;gt;, diff_inten_density_ch_3 &amp;lt;dbl&amp;gt;, …&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We have data for 2019 cells, with 58 variables. The main outcome variable of interest for us here is called &lt;code&gt;class&lt;/code&gt;, which you can see is a factor. But before we jump into predicting the &lt;code&gt;class&lt;/code&gt; variable, we need to understand it better. Below is a brief primer on cell image segmentation.&lt;/p&gt;
&lt;h3 id=&#34;predicting-image-segmentation-quality&#34;&gt;Predicting image segmentation quality&lt;/h3&gt;
&lt;p&gt;Some biologists conduct experiments on cells. In drug discovery, a particular type of cell can be treated with either a drug or control and then observed to see what the effect is (if any). A common approach for this kind of measurement is cell imaging. Different parts of the cells can be colored so that the locations of a cell can be determined.&lt;/p&gt;
&lt;p&gt;For example, in top panel of this image of five cells, the green color is meant to define the boundary of the cell (coloring something called the cytoskeleton) while the blue color defines the nucleus of the cell.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;img/cells.png&#34; width=&#34;70%&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Using these colors, the cells in an image can be &lt;em&gt;segmented&lt;/em&gt; so that we know which pixels belong to which cell. If this is done well, the cell can be measured in different ways that are important to the biology. Sometimes the shape of the cell matters and different mathematical tools are used to summarize characteristics like the size or &amp;ldquo;oblongness&amp;rdquo; of the cell.&lt;/p&gt;
&lt;p&gt;The bottom panel shows some segmentation results. Cells 1 and 5 are fairly well segmented. However, cells 2 to 4 are bunched up together because the segmentation was not very good. The consequence of bad segmentation is data contamination; when the biologist analyzes the shape or size of these cells, the data are inaccurate and could lead to the wrong conclusion.&lt;/p&gt;
&lt;p&gt;A cell-based experiment might involve millions of cells so it is unfeasible to visually assess them all. Instead, a subsample can be created and these cells can be manually labeled by experts as either poorly segmented (&lt;code&gt;PS&lt;/code&gt;) or well-segmented (&lt;code&gt;WS&lt;/code&gt;). If we can predict these labels accurately, the larger data set can be improved by filtering out the cells most likely to be poorly segmented.&lt;/p&gt;
&lt;h3 id=&#34;back-to-the-cells-data&#34;&gt;Back to the cells data&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;cells&lt;/code&gt; data has &lt;code&gt;class&lt;/code&gt; labels for 2019 cells — each cell is labeled as either poorly segmented (&lt;code&gt;PS&lt;/code&gt;) or well-segmented (&lt;code&gt;WS&lt;/code&gt;). Each also has a total of 56 predictors based on automated image analysis measurements. For example, &lt;code&gt;avg_inten_ch_1&lt;/code&gt; is the mean intensity of the data contained in the nucleus, &lt;code&gt;area_ch_1&lt;/code&gt; is the total size of the cell, and so on (some predictors are fairly arcane in nature).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cells
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,019 x 58&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   case  class angle_ch_1 area_ch_1 avg_inten_ch_1 avg_inten_ch_2 avg_inten_ch_3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt; &amp;lt;fct&amp;gt;      &amp;lt;dbl&amp;gt;     &amp;lt;int&amp;gt;          &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 Test  PS        143.         185           15.7           4.95           9.55&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 Train PS        134.         819           31.9         207.            69.9 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 Train WS        107.         431           28.0         116.            63.9 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 Train PS         69.2        298           19.5         102.            28.2 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 Test  PS          2.89       285           24.3         112.            20.5 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,014 more rows, and 51 more variables: avg_inten_ch_4 &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   convex_hull_area_ratio_ch_1 &amp;lt;dbl&amp;gt;, convex_hull_perim_ratio_ch_1 &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   diff_inten_density_ch_1 &amp;lt;dbl&amp;gt;, diff_inten_density_ch_3 &amp;lt;dbl&amp;gt;, …&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The rates of the classes are somewhat imbalanced; there are more poorly segmented cells than well-segmented cells:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cells &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;count&lt;/span&gt;(class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; n&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(n))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   class     n  prop&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt; &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 PS     1300 0.644&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 WS      719 0.356&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;data-split&#34;&gt;Data splitting&lt;/h2&gt;
&lt;p&gt;In our previous 
&lt;a href=&#34;/start/recipes/#data-split&#34;&gt;&lt;em&gt;Preprocess your data with recipes&lt;/em&gt;&lt;/a&gt; article, we started by splitting our data. It is common when beginning a modeling project to 
&lt;a href=&#34;https://bookdown.org/max/FES/data-splitting.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;separate the data set&lt;/a&gt; into two partitions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The &lt;em&gt;training set&lt;/em&gt; is used to estimate parameters, compare models and feature engineering techniques, tune models, etc.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The &lt;em&gt;test set&lt;/em&gt; is held in reserve until the end of the project, at which point there should only be one or two models under serious consideration. It is used as an unbiased source for measuring final model performance.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;There are different ways to create these partitions of the data. The most common approach is to use a random sample. Suppose that one quarter of the data were reserved for the test set. Random sampling would randomly select 25% for the test set and use the remainder for the training set. We can use the 
&lt;a href=&#34;https://tidymodels.github.io/rsample/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rsample&lt;/a&gt; package for this purpose.&lt;/p&gt;
&lt;p&gt;Since random sampling uses random numbers, it is important to set the random number seed. This ensures that the random numbers can be reproduced at a later time (if needed).&lt;/p&gt;
&lt;p&gt;The function &lt;code&gt;rsample::initial_split()&lt;/code&gt; takes the original data and saves the information on how to make the partitions. In the original analysis, the authors made their own training/test set and that information is contained in the column &lt;code&gt;case&lt;/code&gt;. To demonstrate how to make a split, we&amp;rsquo;ll remove this column before we make our own split:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;123&lt;/span&gt;)
cell_split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(cells &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;case), 
                            strata &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Here we used the 
&lt;a href=&#34;https://tidymodels.github.io/rsample/reference/initial_split.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;strata&lt;/code&gt; argument&lt;/a&gt;, which conducts a stratified split. This ensures that, despite the imbalance we noticed in our &lt;code&gt;class&lt;/code&gt; variable, our training and test data sets will keep roughly the same proportions of poorly and well-segmented cells as in the original data. After the &lt;code&gt;initial_split&lt;/code&gt;, the &lt;code&gt;training()&lt;/code&gt; and &lt;code&gt;testing()&lt;/code&gt; functions return the actual data sets.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cell_train &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(cell_split)
cell_test  &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(cell_split)

&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(cell_train)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1515&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(cell_train)&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(cells)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 0.7503715&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# training set proportions by class&lt;/span&gt;
cell_train &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;count&lt;/span&gt;(class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; n&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(n))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   class     n  prop&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt; &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 PS      975 0.644&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 WS      540 0.356&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# test set proportions by class&lt;/span&gt;
cell_test &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;count&lt;/span&gt;(class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; n&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(n))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   class     n  prop&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;fct&amp;gt; &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 PS      325 0.645&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 WS      179 0.355&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The majority of the modeling work is then conducted on the training set data.&lt;/p&gt;
&lt;h2 id=&#34;modeling&#34;&gt;Modeling&lt;/h2&gt;
&lt;p&gt;
&lt;a href=&#34;https://en.wikipedia.org/wiki/Random_forest&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Random forest models&lt;/a&gt; are 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Ensemble_learning&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ensembles&lt;/a&gt; of 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Decision_tree&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;decision trees&lt;/a&gt;. A large number of decision tree models are created for the ensemble based on slightly different versions of the training set. When creating the individual decision trees, the fitting process encourages them to be as diverse as possible. The collection of trees are combined into the random forest model and, when a new sample is predicted, the votes from each tree are used to calculate the final predicted value for the new sample. For categorical outcome variables like &lt;code&gt;class&lt;/code&gt; in our &lt;code&gt;cells&lt;/code&gt; data example, the majority vote across all the trees in the random forest determines the predicted class for the new sample.&lt;/p&gt;
&lt;p&gt;One of the benefits of a random forest model is that it is very low maintenance;  it requires very little preprocessing of the data and the default parameters tend to give reasonable results. For that reason, we won&amp;rsquo;t create a recipe for the &lt;code&gt;cells&lt;/code&gt; data.&lt;/p&gt;
&lt;p&gt;At the same time, the number of trees in the ensemble should be large (in the thousands) and this makes the model moderately expensive to compute.&lt;/p&gt;
&lt;p&gt;To fit a random forest model on the training set, let&amp;rsquo;s use the 
&lt;a href=&#34;https://tidymodels.github.io/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip&lt;/a&gt; package with the 
&lt;a href=&#34;https://cran.r-project.org/web/packages/ranger/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ranger&lt;/a&gt; engine. We first define the model that we want to create:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;rand_forest&lt;/span&gt;(trees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1000&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;ranger&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;set_mode&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;classification&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Starting with this parsnip model object, the &lt;code&gt;fit()&lt;/code&gt; function can be used with a model formula. Since random forest models use random numbers, we again set the seed prior to computing:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;234&lt;/span&gt;)
rf_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  rf_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cell_train)
rf_fit
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; parsnip model object&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Fit time:  2.4s &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Ranger result&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  ranger::ranger(formula = formula, data = data, num.trees = ~1000,      num.threads = 1, verbose = FALSE, seed = sample.int(10^5,          1), probability = TRUE) &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Type:                             Probability estimation &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of trees:                  1000 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Sample size:                      1515 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Number of independent variables:  56 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Mtry:                             7 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Target node size:                 10 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variable importance mode:         none &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Splitrule:                        gini &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; OOB prediction error (Brier s.):  0.1218873&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This new &lt;code&gt;rf_fit&lt;/code&gt; object is our fitted model, trained on our training data set.&lt;/p&gt;
&lt;h2 id=&#34;performance&#34;&gt;Estimating performance&lt;/h2&gt;
&lt;p&gt;During a modeling project, we might create a variety of different models. To choose between them, we need to consider how well these models do, as measured by some performance statistics. In our example in this article, some options we could use are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;the area under the Receiver Operating Characteristic (ROC) curve, and&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;overall classification accuracy.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The ROC curve uses the class probability estimates to give us a sense of performance across the entire set of potential probability cutoffs. Overall accuracy uses the hard class predictions to measure performance. The hard class predictions tell us whether our model predicted &lt;code&gt;PS&lt;/code&gt; or &lt;code&gt;WS&lt;/code&gt; for each cell. But, behind those predictions, the model is actually estimating a probability. A simple 50% probability cutoff is used to categorize a cell as poorly segmented.&lt;/p&gt;
&lt;p&gt;The 
&lt;a href=&#34;https://tidymodels.github.io/yardstick/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;yardstick package&lt;/a&gt; has functions for computing both of these measures called &lt;code&gt;roc_auc()&lt;/code&gt; and &lt;code&gt;accuracy()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;At first glance, it might seem like a good idea to use the training set data to compute these statistics. (This is actually a very bad idea.) Let&amp;rsquo;s see what happens if we try this. To evaluate performance based on the training set, we call the &lt;code&gt;predict()&lt;/code&gt; method to get both types of predictions (i.e. probabilities and hard class predictions).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_training_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(rf_fit, cell_train) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(rf_fit, cell_train, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;prob&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Add the true outcome data back in&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(cell_train &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
              &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(class))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Using the yardstick functions, this model has spectacular results, so spectacular that you might be starting to get suspicious:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_training_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# training set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;roc_auc&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_PS)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 roc_auc binary          1.00&lt;/span&gt;
rf_training_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# training set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;accuracy&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric  .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 accuracy binary         0.993&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now that we have this model with exceptional performance, we proceed to the test set. Unfortunately, we discover that, although our results aren&amp;rsquo;t bad, they are certainly worse than what we initially thought based on predicting the training set:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_testing_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(rf_fit, cell_test) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(rf_fit, cell_test, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;prob&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(cell_test &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(class))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_testing_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                   &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# test set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;roc_auc&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_PS)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 roc_auc binary         0.909&lt;/span&gt;
rf_testing_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                   &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# test set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;accuracy&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric  .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 accuracy binary         0.837&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;what-happened-here&#34;&gt;What happened here?&lt;/h3&gt;
&lt;p&gt;There are several reasons why training set statistics like the ones shown in this section can be unrealistically optimistic:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Models like random forests, neural networks, and other black-box methods can essentially memorize the training set. Re-predicting that same set should always result in nearly perfect results.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The training set does not have the capacity to be a good arbiter of performance. It is not an independent piece of information; predicting the training set can only reflect what the model already knows.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;To understand that second point better, think about an analogy from teaching. Suppose you give a class a test, then give them the answers, then provide the same test. The student scores on the &lt;em&gt;second&lt;/em&gt; test do not accurately reflect what they know about the subject; these scores would probably be higher than their results on the first test.&lt;/p&gt;
&lt;h2 id=&#34;resampling&#34;&gt;Resampling to the rescue&lt;/h2&gt;
&lt;p&gt;Resampling methods, such as cross-validation and the bootstrap, are empirical simulation systems. They create a series of data sets similar to the training/testing split discussed previously; a subset of the data are used for creating the model and a different subset is used to measure performance. Resampling is always used with the &lt;em&gt;training set&lt;/em&gt;. This schematic from 
&lt;a href=&#34;https://bookdown.org/max/FES/resampling.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kuhn and Johnson (2019)&lt;/a&gt; illustrates data usage for resampling methods:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;img/resampling.svg&#34; width=&#34;85%&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;In the first level of this diagram, you see what happens when you use &lt;code&gt;rsample::initial_split()&lt;/code&gt;, which splits the original data into training and test sets. Then, the training set is chosen for resampling, and the test set is held out.&lt;/p&gt;
&lt;p&gt;Let&amp;rsquo;s use 10-fold cross-validation (CV) in this example. This method randomly allocates the 1515 cells in the training set to 10 groups of roughly equal size, called &amp;ldquo;folds&amp;rdquo;. For the first iteration of resampling, the first fold of about 151 cells are held out for the purpose of measuring performance. This is similar to a test set but, to avoid confusion, we call these data the &lt;em&gt;assessment set&lt;/em&gt; in the tidymodels framework.&lt;/p&gt;
&lt;p&gt;The other 90% of the data (about 1363 cells) are used to fit the model. Again, this sounds similar to a training set, so in tidymodels we call this data the &lt;em&gt;analysis set&lt;/em&gt;. This model, trained on the analysis set, is applied to the assessment set to generate predictions, and performance statistics are computed based on those predictions.&lt;/p&gt;
&lt;p&gt;In this example, 10-fold CV moves iteratively through the folds and leaves a different 10% out each time for model assessment. At the end of this process, there are 10 sets of performance statistics that were created on 10 data sets that were not used in the modeling process. For the cell example, this means 10 accuracies and 10 areas under the ROC curve. While 10 models were created, these are not used further; we do not keep the models themselves trained on these folds because their only purpose is calculating performance metrics.&lt;/p&gt;
&lt;p&gt;The final resampling estimates for the model are the &lt;strong&gt;averages&lt;/strong&gt; of the performance statistics replicates. For example, suppose for our data the results were:&lt;/p&gt;
&lt;table class=&#34;table&#34; style=&#34;width: auto !important; margin-left: auto; margin-right: auto;&#34;&gt;
 &lt;thead&gt;
  &lt;tr&gt;
   &lt;th style=&#34;text-align:left;&#34;&gt; resample &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; accuracy &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; roc_auc &lt;/th&gt;
   &lt;th style=&#34;text-align:right;&#34;&gt; assessment size &lt;/th&gt;
  &lt;/tr&gt;
 &lt;/thead&gt;
&lt;tbody&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold01 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.7828947 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8419206 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 152 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold02 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8092105 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8939982 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 152 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold03 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8486842 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9174923 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 152 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold04 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8355263 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8941946 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 152 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold05 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8684211 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9063232 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 152 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold06 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8410596 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9136661 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 151 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold07 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8807947 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9368932 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 151 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold08 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.7814570 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8890798 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 151 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold09 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8145695 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9075369 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 151 &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
   &lt;td style=&#34;text-align:left;&#34;&gt; Fold10 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.8675497 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 0.9310806 &lt;/td&gt;
   &lt;td style=&#34;text-align:right;&#34;&gt; 151 &lt;/td&gt;
  &lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;From these resampling statistics, the final estimate of performance for this random forest model would be 0.903 for the area under the ROC curve and 0.833 for accuracy.&lt;/p&gt;
&lt;p&gt;These resampling statistics are an effective method for measuring model performance &lt;em&gt;without&lt;/em&gt; predicting the training set directly as a whole.&lt;/p&gt;
&lt;h2 id=&#34;fit-resamples&#34;&gt;Fit a model with resampling&lt;/h2&gt;
&lt;p&gt;To generate these results, the first step is to create a resampling object using rsample. There are 
&lt;a href=&#34;https://tidymodels.github.io/rsample/reference/index.html#section-resampling-methods&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;several resampling methods&lt;/a&gt; implemented in rsample; cross-validation folds can be created using &lt;code&gt;vfold_cv()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;345&lt;/span&gt;)
folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vfold_cv&lt;/span&gt;(cell_train, v &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)
folds
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #  10-fold cross-validation &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 10 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits             id    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;named list&amp;gt;       &amp;lt;chr&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [1.4K/152]&amp;gt; Fold01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [1.4K/152]&amp;gt; Fold02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [1.4K/152]&amp;gt; Fold03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [1.4K/152]&amp;gt; Fold04&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [1.4K/152]&amp;gt; Fold05&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [1.4K/151]&amp;gt; Fold06&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [1.4K/151]&amp;gt; Fold07&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [1.4K/151]&amp;gt; Fold08&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [1.4K/151]&amp;gt; Fold09&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [1.4K/151]&amp;gt; Fold10&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The list column for &lt;code&gt;splits&lt;/code&gt; contains the information on which rows belong in the analysis and assessment sets. There are functions that can be used to extract the individual resampled data called &lt;code&gt;analysis()&lt;/code&gt; and &lt;code&gt;assessment()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;However, the tune package contains high-level functions that can do the required computations to resample a model for the purpose of measuring performance. You have several options for building an object for resampling:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Resample a model specification preprocessed with a formula or 
&lt;a href=&#34;/start/recipes/&#34;&gt;recipe&lt;/a&gt;, or&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Resample a 
&lt;a href=&#34;https://tidymodels.github.io/workflows/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;workflow()&lt;/code&gt;&lt;/a&gt; that bundles together a model specification and formula/recipe.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For this example, let&amp;rsquo;s use a &lt;code&gt;workflow()&lt;/code&gt; that bundles together the random forest model and a formula, since we are not using a recipe. Whichever of these options you use, the syntax to &lt;code&gt;fit_resamples()&lt;/code&gt; is very similar to &lt;code&gt;fit()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_wf &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;workflow&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;add_model&lt;/span&gt;(rf_mod) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;add_formula&lt;/span&gt;(class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; .)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;456&lt;/span&gt;)
rf_fit_rs &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  rf_wf &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;fit_resamples&lt;/span&gt;(folds)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_fit_rs
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #  10-fold cross-validation &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 10 x 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits             id     .metrics         .notes          &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  * &amp;lt;list&amp;gt;             &amp;lt;chr&amp;gt;  &amp;lt;list&amp;gt;           &amp;lt;list&amp;gt;          &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [1.4K/152]&amp;gt; Fold01 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [1.4K/152]&amp;gt; Fold02 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [1.4K/152]&amp;gt; Fold03 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [1.4K/152]&amp;gt; Fold04 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [1.4K/152]&amp;gt; Fold05 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [1.4K/151]&amp;gt; Fold06 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [1.4K/151]&amp;gt; Fold07 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [1.4K/151]&amp;gt; Fold08 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [1.4K/151]&amp;gt; Fold09 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [1.4K/151]&amp;gt; Fold10 &amp;lt;tibble [2 × 3]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The results are similar to the &lt;code&gt;folds&lt;/code&gt; results with some extra columns. The column &lt;code&gt;.metrics&lt;/code&gt; contains the performance statistics created from the 10 assessment sets. These can be manually unnested but the tune package contains a number of simple functions that can extract these data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;collect_metrics&lt;/span&gt;(rf_fit_rs)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric  .estimator  mean     n std_err&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 accuracy binary     0.833    10 0.0111 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 roc_auc  binary     0.903    10 0.00842&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Think about these values we now have for accuracy and AUC. These performance metrics are now more realistic (i.e. lower) than our ill-advised first attempt at computing performance metrics in the section above. If we wanted to try different model types for this data set, we could more confidently compare performance metrics computed using resampling to choose between models. Also, remember that at the end of our project, we return to our test set to estimate final model performance. We have looked at this once already before we started using resampling, but let&amp;rsquo;s remind ourselves of the results:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rf_testing_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                   &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# test set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;roc_auc&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_PS)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 roc_auc binary         0.909&lt;/span&gt;
rf_testing_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;                   &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# test set predictions&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;accuracy&lt;/span&gt;(truth &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; class, .pred_class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric  .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 accuracy binary         0.837&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The performance metrics from the test set are much closer to the performance metrics computed using resampling than our first (&amp;ldquo;bad idea&amp;rdquo;) attempt. Resampling allows us to simulate how well our model will perform on new data, and the test set acts as the final, unbiased check for our model&amp;rsquo;s performance.&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-21                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  modeldata  * 0.0.1   2019-12-06 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.4   2020-04-17 [1] CRAN (R 3.6.2)
#&amp;gt;  ranger     * 0.12.1  2020-01-10 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Iterative Bayesian optimization of a classification model</title>
      <link>/learn/work/bayes-opt/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/work/bayes-opt/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: kernlab, modeldata, tidymodels, and tidyr.&lt;/p&gt;
&lt;p&gt;Many of the examples for model tuning focus on 
&lt;a href=&#34;/learn/work/tune-svm/&#34;&gt;grid search&lt;/a&gt;. For that method, all the candidate tuning parameter combinations are defined prior to evaluation. Alternatively, &lt;em&gt;iterative search&lt;/em&gt; can be used to analyze the existing tuning parameter results and then &lt;em&gt;predict&lt;/em&gt; which tuning parameters to try next.&lt;/p&gt;
&lt;p&gt;There are a variety of methods for iterative search and the focus in this article is on &lt;em&gt;Bayesian optimization&lt;/em&gt;. For more information on this method, these resources might be helpful:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C7&amp;amp;q=Practical&amp;#43;Bayesian&amp;#43;Optimization&amp;#43;of&amp;#43;Machine&amp;#43;Learning&amp;#43;Algorithms&amp;amp;btnG=&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Practical bayesian optimization of machine learning algorithms&lt;/em&gt;&lt;/a&gt; (2012). J Snoek, H Larochelle, and RP Adams. Advances in neural information.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://www.cs.toronto.edu/~rgrosse/courses/csc411_f18/tutorials/tut8_adams_slides.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;A Tutorial on Bayesian Optimization for Machine Learning&lt;/em&gt;&lt;/a&gt; (2018). R Adams.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;http://www.gaussianprocess.org/gpml/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Gaussian Processes for Machine Learning&lt;/em&gt;&lt;/a&gt; (2006). C E Rasmussen and C Williams.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C7&amp;amp;q=%22Bayesian&amp;#43;Optimization%22&amp;amp;btnG=&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Other articles!&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;cell-segmenting-revisited&#34;&gt;Cell segmenting revisited&lt;/h2&gt;
&lt;p&gt;To demonstrate this approach to tuning models, let&amp;rsquo;s return to the cell segmentation data from the 
&lt;a href=&#34;/start/resampling/&#34;&gt;Getting Started&lt;/a&gt; article on resampling:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(modeldata)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Load data&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(cells)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2369&lt;/span&gt;)
tr_te_split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(cells &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;case), prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
cell_train &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(tr_te_split)
cell_test  &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(tr_te_split)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1697&lt;/span&gt;)
folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vfold_cv&lt;/span&gt;(cell_train, v &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;the-tuning-scheme&#34;&gt;The tuning scheme&lt;/h2&gt;
&lt;p&gt;Since the predictors are highly correlated, we can used a recipe to convert the original predictors to principal component scores. There is also slight class imbalance in these data; about 64% of the data are poorly segmented. To mitigate this, the data will be down-sampled at the end of the pre-processing so that the number of poorly and well segmented cells occur with equal frequency. We can use a recipe for all this pre-processing, but the number of principal components will need to be &lt;em&gt;tuned&lt;/em&gt; so that we have enough (but not too many) representations of the data.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cell_pre_proc &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cell_train) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_YeoJohnson&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_pca&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;(), num_comp &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tune&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_downsample&lt;/span&gt;(class)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;In this analysis, we will use a support vector machine to model the data. Let&amp;rsquo;s use a radial basis function (RBF) kernel and tune its main parameter ($\sigma$). Additionally, the main SVM parameter, the cost value, also needs optimization.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;svm_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;svm_rbf&lt;/span&gt;(mode &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;classification&amp;#34;&lt;/span&gt;, cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tune&lt;/span&gt;(), rbf_sigma &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tune&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;kernlab&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;These two objects (the recipe and model) will be combined into a single object via the &lt;code&gt;workflow()&lt;/code&gt; function from the 
&lt;a href=&#34;https://tidymodels.github.io/workflows/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;workflows&lt;/a&gt; package; this object will be used in the optimization process.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;svm_wflow &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;workflow&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;add_model&lt;/span&gt;(svm_mod) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;add_recipe&lt;/span&gt;(cell_pre_proc)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;From this object, we can derive information about what parameters are slated to be tuned. A parameter set is derived by:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;svm_set &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;parameters&lt;/span&gt;(svm_wflow)
svm_set
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Collection of 3 parameters for tuning&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;         id parameter type object class&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       cost           cost    nparam[+]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  rbf_sigma      rbf_sigma    nparam[+]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   num_comp       num_comp    nparam[+]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The default range for the number of PCA components is rather small for this data set. A member of the parameter set can be modified using the &lt;code&gt;update()&lt;/code&gt; function. Let&amp;rsquo;s constrain the search to one to twenty components by updating the &lt;code&gt;num_comp&lt;/code&gt; parameter. Additionally, the lower bound of this parameter is set to zero which specifies that the original predictor set should also be evaluated (i.e., with no PCA step at all):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;svm_set &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  svm_set &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;update&lt;/span&gt;(num_comp &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;num_comp&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0L&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;20L&lt;/span&gt;)))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;sequential-tuning&#34;&gt;Sequential tuning&lt;/h2&gt;
&lt;p&gt;Bayesian optimization is a sequential method that uses a model to predict new candidate parameters for assessment. When scoring potential parameter value, the mean and variance of performance are predicted. The strategy used to define how these two statistical quantities are used is defined by an &lt;em&gt;acquisition function&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;For example, one approach for scoring new candidates is to use a confidence bound. Suppose accuracy is being optimized. For a metric that we want to maximize, a lower confidence bound can be used. The multiplier on the standard error (denoted as &lt;code&gt;\(\kappa\)&lt;/code&gt;) is a value that can be used to make trade-offs between &lt;strong&gt;exploration&lt;/strong&gt; and &lt;strong&gt;exploitation&lt;/strong&gt;.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Exploration&lt;/strong&gt; means that the search will consider candidates in untested space.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Exploitation&lt;/strong&gt; focuses in areas where the previous best results occurred.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The variance predicted by the Bayesian model is mostly spatial variation; the value will be large for candidate values that are not close to values that have already been evaluated. If the standard error multiplier is high, the search process will be more likely to avoid areas without candidate values in the vicinity.&lt;/p&gt;
&lt;p&gt;We&amp;rsquo;ll use another acquisition function, &lt;em&gt;expected improvement&lt;/em&gt;, that determines which candidates are likely to be helpful relative to the current best results. This is the default acquisition function. More information on these functions can be found in the 
&lt;a href=&#34;https://tidymodels.github.io/tune/articles/acquisition_functions.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;package vignette for acquisition functions&lt;/a&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;12&lt;/span&gt;)
search_res &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  svm_wflow &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;tune_bayes&lt;/span&gt;(
    resamples &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; folds,
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# To use non-default parameter ranges&lt;/span&gt;
    param_info &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; svm_set,
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Generate five at semi-random to start&lt;/span&gt;
    initial &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;,
    iter &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;50&lt;/span&gt;,
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# How to measure performance?&lt;/span&gt;
    metrics &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;metric_set&lt;/span&gt;(roc_auc),
    control &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;control_bayes&lt;/span&gt;(no_improve &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;30&lt;/span&gt;, verbose &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
  )
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &amp;gt;  Generating a set of 5 initial parameter results&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Initialization complete&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Optimizing roc_auc using the expected improvement&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 1 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.58, rbf_sigma=1.54e-09, num_comp=12&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8624 (+/-0.00897)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 2 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0251, rbf_sigma=6.36e-06, num_comp=16&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8606 (+/-0.00908)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 3 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=23, rbf_sigma=1.02e-10, num_comp=7&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8634 (+/-0.00923)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 4 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0894, rbf_sigma=1.09e-10, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8494 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 5 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.402, rbf_sigma=0.413, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8236 (+/-0.00885)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 6 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=24, rbf_sigma=0.942, num_comp=8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8054 (+/-0.0114)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 7 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=30.3, rbf_sigma=2.25e-06, num_comp=13&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8622 (+/-0.009)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 8 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=25, rbf_sigma=1.07e-10, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8655 (+/-0.00848)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 9 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=2.1, rbf_sigma=5.29e-06, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8494 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 10 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8655 (@iter 0)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=9.87, rbf_sigma=0.000395, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ♥ Newest results:	roc_auc=0.8681 (+/-0.00898)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 11 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8681 (@iter 10)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.073, rbf_sigma=0.000585, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8509 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 12 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8681 (@iter 10)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00101, rbf_sigma=1.29e-07, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8494 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 13 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8681 (@iter 10)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0553, rbf_sigma=0.000291, num_comp=12&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8625 (+/-0.00898)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 14 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8681 (@iter 10)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=11.8, rbf_sigma=0.00143, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ♥ Newest results:	roc_auc=0.8691 (+/-0.00837)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 15 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8691 (@iter 14)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0915, rbf_sigma=0.03, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ♥ Newest results:	roc_auc=0.8728 (+/-0.00842)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 16 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0289, rbf_sigma=8.48e-09, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8655 (+/-0.00848)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 17 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0021, rbf_sigma=0.0109, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8696 (+/-0.00881)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 18 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.461, rbf_sigma=0.908, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7732 (+/-0.0168)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 19 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00132, rbf_sigma=8.1e-08, num_comp=11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8621 (+/-0.00933)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 20 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=20.2, rbf_sigma=1.64e-09, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8494 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 21 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00173, rbf_sigma=0.126, num_comp=11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8721 (+/-0.00749)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 22 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00853, rbf_sigma=0.989, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7369 (+/-0.0313)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 23 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00673, rbf_sigma=1.55e-10, num_comp=17&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.787 (+/-0.0485)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 24 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.871, rbf_sigma=1.72e-10, num_comp=19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.864 (+/-0.00842)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 25 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=3.8, rbf_sigma=6.24e-10, num_comp=19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.864 (+/-0.00842)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 26 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=5.2, rbf_sigma=0.791, num_comp=1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7319 (+/-0.0173)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 27 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.213, rbf_sigma=9.11e-10, num_comp=20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8655 (+/-0.00848)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 28 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=6.99, rbf_sigma=3.03e-10, num_comp=0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8494 (+/-0.0116)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 29 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00102, rbf_sigma=0.344, num_comp=5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8631 (+/-0.0079)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 30 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=20.3, rbf_sigma=1.28e-05, num_comp=3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8503 (+/-0.0112)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 31 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0012, rbf_sigma=3.75e-05, num_comp=7&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8634 (+/-0.00923)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 32 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0142, rbf_sigma=2.58e-10, num_comp=1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7015 (+/-0.0374)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 33 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00411, rbf_sigma=0.557, num_comp=1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.747 (+/-0.0173)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 34 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.161, rbf_sigma=0.167, num_comp=1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7541 (+/-0.0177)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 35 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=2.48, rbf_sigma=0.783, num_comp=19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7748 (+/-0.014)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 36 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0138, rbf_sigma=0.624, num_comp=19&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7938 (+/-0.0117)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 37 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00341, rbf_sigma=1.11e-10, num_comp=2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7311 (+/-0.0404)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 38 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00113, rbf_sigma=1.48e-10, num_comp=14&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7888 (+/-0.0489)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 39 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=17.1, rbf_sigma=1.71e-10, num_comp=9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8638 (+/-0.00874)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 40 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=13.3, rbf_sigma=0.968, num_comp=17&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7691 (+/-0.0158)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 41 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0026, rbf_sigma=0.995, num_comp=3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8496 (+/-0.0093)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 42 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=23.6, rbf_sigma=0.856, num_comp=13&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.7972 (+/-0.0144)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 43 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00142, rbf_sigma=7.1e-06, num_comp=18&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8593 (+/-0.00882)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 44 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=31.4, rbf_sigma=1.7e-10, num_comp=15&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8616 (+/-0.00899)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 45 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8728 (@iter 15)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=31.4, rbf_sigma=0.0118, num_comp=6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ♥ Newest results:	roc_auc=0.8779 (+/-0.00726)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 46 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8779 (@iter 45)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=11, rbf_sigma=0.718, num_comp=10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8247 (+/-0.0103)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 47 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8779 (@iter 45)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=27.1, rbf_sigma=3.61e-06, num_comp=8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8645 (+/-0.00874)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 48 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8779 (@iter 45)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=20.4, rbf_sigma=1.23e-10, num_comp=4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8513 (+/-0.0109)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 49 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8779 (@iter 45)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.0011, rbf_sigma=0.677, num_comp=16&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8075 (+/-0.0119)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Iteration 50 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Current best:		roc_auc=0.8779 (@iter 45)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Gaussian process model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Generating 5000 candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Predicted candidates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i cost=0.00133, rbf_sigma=0.592, num_comp=14&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; i Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ Estimating performance&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ⓧ Newest results:	roc_auc=0.8311 (+/-0.014)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The resulting tibble is a stacked set of rows of the rsample object with an additional column for the iteration number:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;search_res
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #  10-fold cross-validation &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 510 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    splits             id     .metrics         .notes           .iter&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  * &amp;lt;list&amp;gt;             &amp;lt;chr&amp;gt;  &amp;lt;list&amp;gt;           &amp;lt;list&amp;gt;           &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;lt;split [1.4K/152]&amp;gt; Fold01 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;lt;split [1.4K/152]&amp;gt; Fold02 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;lt;split [1.4K/152]&amp;gt; Fold03 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;lt;split [1.4K/152]&amp;gt; Fold04 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;lt;split [1.4K/152]&amp;gt; Fold05 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;lt;split [1.4K/151]&amp;gt; Fold06 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;lt;split [1.4K/151]&amp;gt; Fold07 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;lt;split [1.4K/151]&amp;gt; Fold08 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;lt;split [1.4K/151]&amp;gt; Fold09 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;lt;split [1.4K/151]&amp;gt; Fold10 &amp;lt;tibble [5 × 6]&amp;gt; &amp;lt;tibble [0 × 1]&amp;gt;     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 500 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As with grid search, we can summarize the results over resamples:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;estimates &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;collect_metrics&lt;/span&gt;(search_res) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;arrange&lt;/span&gt;(.iter)

estimates
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 55 x 9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        cost rbf_sigma num_comp .iter .metric .estimator  mean     n std_err&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;    &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1  0.00207  1.56e- 5       10     0 roc_auc binary     0.864    10 0.00888&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2  0.0304   6.41e- 9        5     0 roc_auc binary     0.859    10 0.00922&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3  0.348    4.43e- 2        1     0 roc_auc binary     0.757    10 0.0177 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4  1.45     2.04e- 3       15     0 roc_auc binary     0.865    10 0.00962&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 15.5      1.28e- 7       20     0 roc_auc binary     0.865    10 0.00848&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6  0.580    1.54e- 9       12     1 roc_auc binary     0.862    10 0.00897&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7  0.0251   6.36e- 6       16     2 roc_auc binary     0.861    10 0.00908&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 23.0      1.02e-10        7     3 roc_auc binary     0.863    10 0.00923&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9  0.0894   1.09e-10        0     4 roc_auc binary     0.849    10 0.0116 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10  0.402    4.13e- 1       20     5 roc_auc binary     0.824    10 0.00885&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 45 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The best performance of the initial set of candidate values was &lt;code&gt;AUC = 0.865 &lt;/code&gt;. The best results were achieved at iteration 45 with a corresponding AUC value of 0.878. The five best results are:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;show_best&lt;/span&gt;(search_res, metric &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;roc_auc&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5 x 9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       cost rbf_sigma num_comp .iter .metric .estimator  mean     n std_err&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;    &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 31.4       0.0118         6    45 roc_auc binary     0.878    10 0.00726&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2  0.0915    0.0300        20    15 roc_auc binary     0.873    10 0.00842&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3  0.00173   0.126         11    21 roc_auc binary     0.872    10 0.00749&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4  0.00210   0.0109        20    17 roc_auc binary     0.870    10 0.00881&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 11.8       0.00143       20    14 roc_auc binary     0.869    10 0.00837&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;A plot of the search iterations can be created via:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;autoplot&lt;/span&gt;(search_res, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;performance&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/bo-plot-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;There are many parameter combinations have roughly equivalent results.&lt;/p&gt;
&lt;p&gt;How did the parameters change over iterations? Since two of the parameters are usually treated on the log scale, we can use &lt;code&gt;mutate()&lt;/code&gt; to transform them, and then construct a plot using ggplot2:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidyr)

&lt;span style=&#34;color:#00f&#34;&gt;collect_metrics&lt;/span&gt;(search_res) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-.&lt;/span&gt;metric,&lt;span style=&#34;color:#666&#34;&gt;-.&lt;/span&gt;estimator,&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;mean,&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;n,&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;std_err) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(cost &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(cost), 
         rbf_sigma &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(rbf_sigma)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(cols &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-.i&lt;/span&gt;ter),
               names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;parameter&amp;#34;&lt;/span&gt;,
               values_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;value&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .iter, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; value)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;facet_wrap&lt;/span&gt;( &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; parameter, scales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;free_y&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/bo-param-plot-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  kernlab    * 0.9-29  2019-11-12 [1] CRAN (R 3.6.0)
#&amp;gt;  modeldata  * 0.0.1   2019-12-06 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang      * 0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tidyr      * 1.0.2   2020-01-24 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Hypothesis testing using resampling and tidy data</title>
      <link>/learn/statistics/infer/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/statistics/infer/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;This article only requires the tidymodels package.&lt;/p&gt;
&lt;p&gt;The tidymodels package 
&lt;a href=&#34;https://tidymodels.github.io/infer/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;infer&lt;/a&gt; implements an expressive grammar to perform statistical inference that coheres with the &lt;code&gt;tidyverse&lt;/code&gt; design framework. Rather than providing methods for specific statistical tests, this package consolidates the principles that are shared among common hypothesis tests into a set of 4 main verbs (functions), supplemented with many utilities to visualize and extract information from their outputs.&lt;/p&gt;
&lt;p&gt;Regardless of which hypothesis test we&amp;rsquo;re using, we&amp;rsquo;re still asking the same kind of question:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Is the effect or difference in our observed data real, or due to chance?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;To answer this question, we start by assuming that the observed data came from some world where &amp;ldquo;nothing is going on&amp;rdquo; (i.e. the observed effect was simply due to random chance), and call this assumption our &lt;strong&gt;null hypothesis&lt;/strong&gt;. (In reality, we might not believe in the null hypothesis at all; the null hypothesis is in opposition to the &lt;strong&gt;alternate hypothesis&lt;/strong&gt;, which supposes that the effect present in the observed data is actually due to the fact that &amp;ldquo;something is going on.&amp;quot;) We then calculate a &lt;strong&gt;test statistic&lt;/strong&gt; from our data that describes the observed effect. We can use this test statistic to calculate a &lt;strong&gt;p-value&lt;/strong&gt;, giving the probability that our observed data could come about if the null hypothesis was true. If this probability is below some pre-defined &lt;strong&gt;significance level&lt;/strong&gt; &lt;code&gt;\(\alpha\)&lt;/code&gt;, then we can reject our null hypothesis.&lt;/p&gt;
&lt;p&gt;If you are new to hypothesis testing, take a look at&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;a href=&#34;https://moderndive.com/9-hypothesis-testing.html#understanding-ht&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Section 9.2 of &lt;em&gt;Statistical Inference via Data Science&lt;/em&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;The American Statistical Association&amp;rsquo;s recent 
&lt;a href=&#34;https://doi.org/10.1080/00031305.2016.1154108&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;statement on p-values&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The workflow of this package is designed around these ideas. Starting from some data set,&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;specify()&lt;/code&gt; allows you to specify the variable, or relationship between variables, that you&amp;rsquo;re interested in,&lt;/li&gt;
&lt;li&gt;&lt;code&gt;hypothesize()&lt;/code&gt; allows you to declare the null hypothesis,&lt;/li&gt;
&lt;li&gt;&lt;code&gt;generate()&lt;/code&gt; allows you to generate data reflecting the null hypothesis, and&lt;/li&gt;
&lt;li&gt;&lt;code&gt;calculate()&lt;/code&gt; allows you to calculate a distribution of statistics from the generated data to form the null distribution.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Throughout this vignette, we make use of &lt;code&gt;gss&lt;/code&gt;, a data set available in infer containing a sample of 500 observations of 11 variables from the &lt;em&gt;General Social Survey&lt;/em&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels) &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Includes the infer package&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# load in the data set&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(gss)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# take a look at its structure&lt;/span&gt;
dplyr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;glimpse&lt;/span&gt;(gss)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Observations: 3,000&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Variables: 11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ year    &amp;lt;dbl&amp;gt; 2008, 2006, 1985, 1987, 2006, 1986, 1977, 1998, 2012, 1982, 1…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ age     &amp;lt;dbl&amp;gt; 37, 29, 58, 40, 39, 37, 53, 41, 55, 47, 36, 75, 22, 19, 34, 5…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ sex     &amp;lt;fct&amp;gt; male, female, male, male, female, male, female, male, male, m…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ college &amp;lt;fct&amp;gt; no degree, no degree, degree, degree, no degree, no degree, n…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ partyid &amp;lt;fct&amp;gt; dem, dem, ind, rep, dem, dem, dem, rep, ind, rep, rep, rep, r…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ hompop  &amp;lt;dbl&amp;gt; 4, 3, 3, 5, 5, 5, 4, 1, 5, 4, 5, 2, 3, 2, 5, 2, 5, 7, 1, 3, 4…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ hours   &amp;lt;dbl&amp;gt; 50, NA, 60, 84, 40, 50, NA, 60, NA, 40, 20, NA, 40, 40, 20, 5…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ income  &amp;lt;ord&amp;gt; $25000 or more, lt $1000, $25000 or more, $25000 or more, $60…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ class   &amp;lt;fct&amp;gt; working class, middle class, middle class, middle class, NA, …&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ finrela &amp;lt;fct&amp;gt; below average, below average, far above average, far below av…&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $ weight  &amp;lt;dbl&amp;gt; 0.875, 0.430, 1.554, 1.010, 0.859, 1.007, 0.988, 0.550, 3.496…&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Each row is an individual survey response, containing some basic demographic information on the respondent as well as some additional variables. See &lt;code&gt;?gss&lt;/code&gt; for more information on the variables included and their source. Note that this data (and our examples on it) are for demonstration purposes only, and will not necessarily provide accurate estimates unless weighted properly. For these examples, let&amp;rsquo;s suppose that this data set is a representative sample of a population we want to learn about: American adults.&lt;/p&gt;
&lt;h2 id=&#34;specify-variables&#34;&gt;Specify variables&lt;/h2&gt;
&lt;p&gt;The &lt;code&gt;specify()&lt;/code&gt; function can be used to specify which of the variables in the data set you&amp;rsquo;re interested in. If you&amp;rsquo;re only interested in, say, the &lt;code&gt;age&lt;/code&gt; of the respondents, you might write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; age)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: age (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,988 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      age&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1    37&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2    29&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3    58&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4    40&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5    39&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6    37&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7    53&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8    41&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9    55&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10    47&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,978 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;On the front end, the output of &lt;code&gt;specify()&lt;/code&gt; just looks like it selects off the columns in the dataframe that you&amp;rsquo;ve specified. What do we see if we check the class of this object, though?&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; age) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;class&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;infer&amp;#34;      &amp;#34;tbl_df&amp;#34;     &amp;#34;tbl&amp;#34;        &amp;#34;data.frame&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We can see that the infer class has been appended on top of the dataframe classes; this new class stores some extra metadata.&lt;/p&gt;
&lt;p&gt;If you&amp;rsquo;re interested in two variables (&lt;code&gt;age&lt;/code&gt; and &lt;code&gt;partyid&lt;/code&gt;, for example) you can &lt;code&gt;specify()&lt;/code&gt; their relationship in one of two (equivalent) ways:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# as a formula&lt;/span&gt;
gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(age &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; partyid)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: age (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Explanatory: partyid (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,963 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      age partyid&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt; &amp;lt;fct&amp;gt;  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1    37 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2    29 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3    58 ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4    40 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5    39 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6    37 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7    53 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8    41 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9    55 ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10    47 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,953 more rows&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# with the named arguments&lt;/span&gt;
gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; age, explanatory &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; partyid)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: age (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Explanatory: partyid (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,963 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      age partyid&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt; &amp;lt;fct&amp;gt;  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1    37 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2    29 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3    58 ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4    40 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5    39 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6    37 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7    53 dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8    41 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9    55 ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10    47 rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,953 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you&amp;rsquo;re doing inference on one proportion or a difference in proportions, you will need to use the &lt;code&gt;success&lt;/code&gt; argument to specify which level of your &lt;code&gt;response&lt;/code&gt; variable is a success. For instance, if you&amp;rsquo;re interested in the proportion of the population with a college degree, you might use the following code:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# specifying for inference on proportions&lt;/span&gt;
gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; college, success &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;degree&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: college (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,990 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    college  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;fct&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 degree   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 degree   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 no degree&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,980 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;declare-the-hypothesis&#34;&gt;Declare the hypothesis&lt;/h2&gt;
&lt;p&gt;The next step in the infer pipeline is often to declare a null hypothesis using &lt;code&gt;hypothesize()&lt;/code&gt;. The first step is to supply one of &amp;ldquo;independence&amp;rdquo; or &amp;ldquo;point&amp;rdquo; to the &lt;code&gt;null&lt;/code&gt; argument. If your null hypothesis assumes independence between two variables, then this is all you need to supply to &lt;code&gt;hypothesize()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(college &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; partyid, success &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;degree&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: college (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Explanatory: partyid (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Null Hypothesis: independence&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2,967 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    college   partyid&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;fct&amp;gt;     &amp;lt;fct&amp;gt;  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 no degree dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 no degree dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 degree    ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 degree    rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 no degree dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 no degree dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 no degree dem    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 no degree rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 no degree ind    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 no degree rep    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 2,957 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you&amp;rsquo;re doing inference on a point estimate, you will also need to provide one of &lt;code&gt;p&lt;/code&gt; (the true proportion of successes, between 0 and 1), &lt;code&gt;mu&lt;/code&gt; (the true mean), &lt;code&gt;med&lt;/code&gt; (the true median), or &lt;code&gt;sigma&lt;/code&gt; (the true standard deviation). For instance, if the null hypothesis is that the mean number of hours worked per week in our population is 40, we would write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; hours) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, mu &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;40&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: hours (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Null Hypothesis: point&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1,756 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    hours&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2    60&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3    84&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4    40&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5    50&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6    60&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7    40&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8    20&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9    40&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10    40&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 1,746 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Again, from the front-end, the dataframe outputted from &lt;code&gt;hypothesize()&lt;/code&gt; looks almost exactly the same as it did when it came out of &lt;code&gt;specify()&lt;/code&gt;, but infer now &amp;ldquo;knows&amp;rdquo; your null hypothesis.&lt;/p&gt;
&lt;h2 id=&#34;generate-the-distribution&#34;&gt;Generate the distribution&lt;/h2&gt;
&lt;p&gt;Once we&amp;rsquo;ve asserted our null hypothesis using &lt;code&gt;hypothesize()&lt;/code&gt;, we can construct a null distribution based on this hypothesis. We can do this using one of several methods, supplied in the &lt;code&gt;type&lt;/code&gt; argument:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;bootstrap&lt;/code&gt;: A bootstrap sample will be drawn for each replicate, where a sample of size equal to the input sample size is drawn (with replacement) from the input sample data.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;permute&lt;/code&gt;: For each replicate, each input value will be randomly reassigned (without replacement) to a new output value in the sample.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;simulate&lt;/code&gt;: A value will be sampled from a theoretical distribution with parameters specified in &lt;code&gt;hypothesize()&lt;/code&gt; for each replicate. (This option is currently only applicable for testing point estimates.)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Continuing on with our example above, about the average number of hours worked a week, we might write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; hours) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, mu &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;40&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;bootstrap&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: hours (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Null Hypothesis: point&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 8,780,000 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # Groups:   replicate [5,000]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    replicate hours&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1         1  42.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2         1  54.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3         1  29.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4         1  39.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5         1  39.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6         1  54.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7         1  39.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8         1  24.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9         1  42.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10         1  23.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 8,779,990 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;In the above example, we take 5000 bootstrap samples to form our null distribution.&lt;/p&gt;
&lt;p&gt;To generate a null distribution for the independence of two variables, we could also randomly reshuffle the pairings of explanatory and response variables to break any existing association. For instance, to generate 5000 replicates that can be used to create a null distribution under the assumption that political party affiliation is not affected by age:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(partyid &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; age) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;permute&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Response: partyid (factor)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Explanatory: age (numeric)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Null Hypothesis: independence&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 14,815,000 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # Groups:   replicate [5,000]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    partyid   age replicate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;fct&amp;gt;   &amp;lt;dbl&amp;gt;     &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 dem        37         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 dem        29         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 rep        58         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 rep        40         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 other      39         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 ind        37         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 ind        53         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 ind        41         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 dem        55         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 rep        47         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 14,814,990 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;calculate-statistics&#34;&gt;Calculate statistics&lt;/h2&gt;
&lt;p&gt;Depending on whether you&amp;rsquo;re carrying out computation-based inference or theory-based inference, you will either supply &lt;code&gt;calculate()&lt;/code&gt; with the output of &lt;code&gt;generate()&lt;/code&gt; or &lt;code&gt;hypothesize()&lt;/code&gt;, respectively. The function, for one, takes in a &lt;code&gt;stat&lt;/code&gt; argument, which is currently one of &lt;code&gt;&amp;quot;mean&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;median&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;sum&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;sd&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;prop&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;count&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;diff in means&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;diff in medians&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;diff in props&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;Chisq&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;F&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;t&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;z&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;slope&amp;quot;&lt;/code&gt;, or &lt;code&gt;&amp;quot;correlation&amp;quot;&lt;/code&gt;. For example, continuing our example above to calculate the null distribution of mean hours worked per week:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; hours) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, mu &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;40&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;bootstrap&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;mean&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5,000 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    replicate  stat&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1         1  40.1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2         2  39.8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3         3  39.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4         4  39.9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5         5  40.3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6         6  40.1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7         7  40.2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8         8  40.4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9         9  39.8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10        10  39.9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 4,990 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The output of &lt;code&gt;calculate()&lt;/code&gt; here shows us the sample statistic (in this case, the mean) for each of our 1000 replicates. If you&amp;rsquo;re carrying out inference on differences in means, medians, or proportions, or &lt;code&gt;\(t\)&lt;/code&gt; and &lt;code&gt;\(z\)&lt;/code&gt; statistics, you will need to supply an &lt;code&gt;order&lt;/code&gt; argument, giving the order in which the explanatory variables should be subtracted. For instance, to find the difference in mean age of those that have a college degree and those that don&amp;rsquo;t, we might write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(age &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; college) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;permute&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;diff in means&amp;#34;&lt;/span&gt;, order &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;degree&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;no degree&amp;#34;&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5,000 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    replicate    stat&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1         1 -0.0914&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2         2 -0.0354&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3         3  0.112 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4         4 -0.665 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5         5 -1.32  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6         6 -1.01  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7         7 -1.41  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8         8 -0.0506&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9         9  0.247 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10        10 -0.214 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 4,990 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;other-utilities&#34;&gt;Other utilities&lt;/h2&gt;
&lt;p&gt;The infer package also offers several utilities to extract meaning out of summary statistics and null distributions; the package provides functions to visualize where a statistic is relative to a distribution (with &lt;code&gt;visualize()&lt;/code&gt;), calculate p-values (with &lt;code&gt;get_p_value()&lt;/code&gt;), and calculate confidence intervals (with &lt;code&gt;get_confidence_interval()&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;To illustrate, we&amp;rsquo;ll go back to the example of determining whether the mean number of hours worked per week is 40 hours.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# find the point estimate&lt;/span&gt;
point_estimate &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; hours) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;mean&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning: Removed 1244 rows containing missing values.&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# generate a null distribution&lt;/span&gt;
null_dist &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; hours) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, mu &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;40&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;bootstrap&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;mean&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning: Removed 1244 rows containing missing values.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;(Notice the warning: &lt;code&gt;Removed 1244 rows containing missing values.&lt;/code&gt; This would be worth noting if you were actually carrying out this hypothesis test.)&lt;/p&gt;
&lt;p&gt;Our point estimate 40.772 seems &lt;em&gt;pretty&lt;/em&gt; close to 40, but a little bit different. We might wonder if this difference is just due to random chance, or if the mean number of hours worked per week in the population really isn&amp;rsquo;t 40.&lt;/p&gt;
&lt;p&gt;We could initially just visualize the null distribution.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;null_dist &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;()
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Where does our sample&amp;rsquo;s observed statistic lie on this distribution? We can use the &lt;code&gt;obs_stat&lt;/code&gt; argument to specify this.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;null_dist &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(obs_stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; point_estimate, direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;two_sided&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize2-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Notice that infer has also shaded the regions of the null distribution that are as (or more) extreme than our observed statistic. (Also, note that we now use the &lt;code&gt;+&lt;/code&gt; operator to apply the &lt;code&gt;shade_p_value()&lt;/code&gt; function. This is because &lt;code&gt;visualize()&lt;/code&gt; outputs a plot object from ggplot2 instead of a dataframe, and the &lt;code&gt;+&lt;/code&gt; operator is needed to add the p-value layer to the plot object.) The red bar looks like it&amp;rsquo;s slightly far out on the right tail of the null distribution, so observing a sample mean of 40.772 hours would be somewhat unlikely if the mean was actually 40 hours. How unlikely, though?&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# get a two-tailed p-value&lt;/span&gt;
p_value &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; null_dist &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;get_p_value&lt;/span&gt;(obs_stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; point_estimate, direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;two_sided&amp;#34;&lt;/span&gt;)

p_value
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   p_value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1  0.0216&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;It looks like the p-value is 0.022, which is pretty small&amp;mdash;if the true mean number of hours worked per week was actually 40, the probability of our sample mean being this far (0.772 hours) from 40 would be 0.022. This may or may not be statistically significantly different, depending on the significance level &lt;code&gt;\(\alpha\)&lt;/code&gt; you decided on &lt;em&gt;before&lt;/em&gt; you ran this analysis. If you had set &lt;code&gt;\(\alpha = .05\)&lt;/code&gt;, then this difference would be statistically significant, but if you had set &lt;code&gt;\(\alpha = .01\)&lt;/code&gt;, then it would not be.&lt;/p&gt;
&lt;p&gt;To get a confidence interval around our estimate, we can write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# start with the null distribution&lt;/span&gt;
null_dist &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# calculate the confidence interval around the point estimate&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;get_confidence_interval&lt;/span&gt;(point_estimate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; point_estimate,
                          &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# at the 95% confidence level&lt;/span&gt;
                          level &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;.95&lt;/span&gt;,
                          &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# using the standard error&lt;/span&gt;
                          type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;se&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   lower upper&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1  40.1  41.4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As you can see, 40 hours per week is not contained in this interval, which aligns with our previous conclusion that this finding is significant at the confidence level &lt;code&gt;\(\alpha = .05\)&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&#34;theoretical-methods&#34;&gt;Theoretical methods&lt;/h2&gt;
&lt;p&gt;The infer package also provides functionality to use theoretical methods for &lt;code&gt;&amp;quot;Chisq&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;F&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;t&amp;quot;&lt;/code&gt; test statistics.&lt;/p&gt;
&lt;p&gt;Generally, to find a null distribution using theory-based methods, use the same code that you would use to find the null distribution using randomization-based methods, but skip the &lt;code&gt;generate()&lt;/code&gt; step. For example, if we wanted to find a null distribution for the relationship between age (&lt;code&gt;age&lt;/code&gt;) and party identification (&lt;code&gt;partyid&lt;/code&gt;) using randomization, we could write:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;null_f_distn &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(age &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; partyid) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;permute&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;F&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To find the null distribution using theory-based methods, instead, skip the &lt;code&gt;generate()&lt;/code&gt; step entirely:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;null_f_distn_theoretical &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(age &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; partyid) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
   &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;F&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We&amp;rsquo;ll calculate the observed statistic to make use of in the following visualizations; this procedure is the same, regardless of the methods used to find the null distribution.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;F_hat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; gss &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(age &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; partyid) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;F&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, instead of just piping the null distribution into &lt;code&gt;visualize()&lt;/code&gt;, as we would do if we wanted to visualize the randomization-based null distribution, we also need to provide &lt;code&gt;method = &amp;quot;theoretical&amp;quot;&lt;/code&gt; to &lt;code&gt;visualize()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;(null_f_distn_theoretical, method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;theoretical&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(obs_stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; F_hat, direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-4-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;To get a sense of how the theory-based and randomization-based null distributions relate, we can pipe the randomization-based null distribution into &lt;code&gt;visualize()&lt;/code&gt; and also specify &lt;code&gt;method = &amp;quot;both&amp;quot;&lt;/code&gt;&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;(null_f_distn, method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;both&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(obs_stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; F_hat, direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-5-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;That&amp;rsquo;s it! This vignette covers most all of the key functionality of infer. See &lt;code&gt;help(package = &amp;quot;infer&amp;quot;)&lt;/code&gt; for a full list of functions and vignettes.&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Multivariate analysis using partial least squares</title>
      <link>/learn/models/pls/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/models/pls/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;To use the code in this article, you will need to install the following packages: modeldata, pls, tidymodels, and tidyr.&lt;/p&gt;
&lt;p&gt;&amp;ldquo;Multivariate analysis&amp;rdquo; usually refers to multiple &lt;em&gt;outcomes&lt;/em&gt; being modeled, analyzed, and/or predicted. There are multivariate versions of many common statistical tools. For example, suppose there was a data set with columns &lt;code&gt;y1&lt;/code&gt; and &lt;code&gt;y2&lt;/code&gt; representing two outcomes to be predicted. The &lt;code&gt;lm()&lt;/code&gt; function would look something like:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;lm&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;cbind&lt;/span&gt;(y1, y2) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; dat)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This &lt;code&gt;cbind()&lt;/code&gt; call is pretty awkward and is a consequence of how the traditional formula infrastructure works. The recipes package is a lot easier to work with! This article demonstrates how to model multiple outcomes.&lt;/p&gt;
&lt;p&gt;The data that we&amp;rsquo;ll use has three outcomes. From &lt;code&gt;?modeldata::meats&lt;/code&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;ldquo;These data are recorded on a Tecator Infratec Food and Feed Analyzer working in the wavelength range 850 - 1050 nm by the Near Infrared Transmission (NIT) principle. Each sample contains finely chopped pure meat with different moisture, fat and protein contents.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;ldquo;For each meat sample the data consists of a 100 channel spectrum of absorbances and the contents of moisture (water), fat and protein. The absorbance is &lt;code&gt;-log10&lt;/code&gt; of the transmittance measured by the spectrometer. The three contents, measured in percent, are determined by analytic chemistry.&amp;rdquo;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The goal is to predict the proportion of the three substances using the chemistry test. There can often be a high degree of between-variable correlations in predictors, and that is certainly the case here.&lt;/p&gt;
&lt;p&gt;To start, let&amp;rsquo;s take the two data matrices (called &lt;code&gt;endpoints&lt;/code&gt; and &lt;code&gt;absorp&lt;/code&gt;) and bind them together in a data frame:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(modeldata)
&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(meats)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The three &lt;em&gt;outcomes&lt;/em&gt; have fairly high correlations also.&lt;/p&gt;
&lt;h2 id=&#34;preprocessing-the-data&#34;&gt;Preprocessing the data&lt;/h2&gt;
&lt;p&gt;If the outcomes can be predicted using a linear model, partial least squares (PLS) is an ideal method. PLS models the data as a function of a set of unobserved &lt;em&gt;latent&lt;/em&gt; variables that are derived in a manner similar to principal component analysis (PCA).&lt;/p&gt;
&lt;p&gt;PLS, unlike PCA, also incorporates the outcome data when creating the PLS components. Like PCA, it tries to maximize the variance of the predictors that are explained by the components but it also tries to simultaneously maximize the correlation between those components and the outcomes. In this way, PLS &lt;em&gt;chases&lt;/em&gt; variation of the predictors and outcomes.&lt;/p&gt;
&lt;p&gt;Since we are working with variances and covariances, we need to standardize the data. The recipe will center and scale all of the variables.&lt;/p&gt;
&lt;p&gt;Many base R functions that deal with multivariate outcomes using a formula require the use of &lt;code&gt;cbind()&lt;/code&gt; on the left-hand side of the formula to work with the traditional formula methods. In tidymodels, recipes do not; the outcomes can be symbolically &amp;ldquo;added&amp;rdquo; together on the left-hand side:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;norm_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(water &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; fat &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; protein &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meats) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;everything&lt;/span&gt;()) 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Before we can finalize the PLS model, the number of PLS components to retain must be determined. This can be done using performance metrics such as the root mean squared error. However, we can also calculate the proportion of variance explained by the components for the &lt;em&gt;predictors and each of the outcomes&lt;/em&gt;. This allows an informed choice to be made based on the level of evidence that the situation requires.&lt;/p&gt;
&lt;p&gt;Since the data set isn&amp;rsquo;t large, let&amp;rsquo;s use resampling to measure these proportions. With ten repeats of 10-fold cross-validation, we build the PLS model on 90% of the data and evaluate on the heldout 10%. For each of the 100 models, we extract and save the proportions.&lt;/p&gt;
&lt;p&gt;The folds can be created using the 
&lt;a href=&#34;https://tidymodels.github.io/rsample/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rsample&lt;/a&gt; package and the recipe can be estimated for each resample using the 
&lt;a href=&#34;https://tidymodels.github.io/rsample/reference/prepper.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;prepper()&lt;/code&gt;&lt;/a&gt; function:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;57343&lt;/span&gt;)
folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vfold_cv&lt;/span&gt;(meats, repeats &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)

folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  folds &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(recipes &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(splits, prepper, recipe &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; norm_rec))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;partial-least-squares&#34;&gt;Partial least squares&lt;/h2&gt;
&lt;p&gt;The complicated parts for moving forward are:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Formatting the predictors and outcomes into the format that the pls package requires, and&lt;/li&gt;
&lt;li&gt;Estimating the proportions.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;For the first part, the standardized outcomes and predictors need to be formatted into two separate matrices. Since we used &lt;code&gt;retain = TRUE&lt;/code&gt; when prepping the recipes, we can use the &lt;code&gt;juice()&lt;/code&gt; function. To save the data as a matrix, the option &lt;code&gt;composition = &amp;quot;matrix&amp;quot;&lt;/code&gt; will avoid saving the data as tibbles and use the required format.&lt;/p&gt;
&lt;p&gt;The pls package expects a simple formula to specify the model, but each side of the formula should &lt;em&gt;represent a matrix&lt;/em&gt;. In other words, we need a data set with two columns where each column is a matrix. The secret to doing this is to &amp;ldquo;protect&amp;rdquo; the two matrices using &lt;code&gt;I()&lt;/code&gt; when adding them to the data frame.&lt;/p&gt;
&lt;p&gt;The calculation for the proportion of variance explained is straightforward for the predictors; the function &lt;code&gt;pls::explvar()&lt;/code&gt; will compute that. For the outcomes, the process is more complicated. A ready-made function to compute these is not obvious but there is some code inside of the summary function to do the computation (see below).&lt;/p&gt;
&lt;p&gt;The function &lt;code&gt;get_var_explained()&lt;/code&gt; shown here will do all these computations and return a data frame with columns &lt;code&gt;components&lt;/code&gt;, &lt;code&gt;source&lt;/code&gt; (for the predictors, water, etc), and the &lt;code&gt;proportion&lt;/code&gt; of variance that is explained by the components.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(pls)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidyr)

get_var_explained &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(recipe, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt;) {
  
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Extract the predictors and outcomes into their own matrices&lt;/span&gt;
  y_mat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;(recipe, composition &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;matrix&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#00f&#34;&gt;all_outcomes&lt;/span&gt;())
  x_mat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;(recipe, composition &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;matrix&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;())
  
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# The pls package prefers the data in a data frame where the outcome&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# and predictors are in _matrices_. To make sure this is formatted&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# properly, use the `I()` function to inhibit `data.frame()` from making&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# all the individual columns. `pls_format` should have two columns.&lt;/span&gt;
  pls_format &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;data.frame&lt;/span&gt;(
    endpoints &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;I&lt;/span&gt;(y_mat),
    measurements &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;I&lt;/span&gt;(x_mat)
  )
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Fit the model&lt;/span&gt;
  mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;plsr&lt;/span&gt;(endpoints &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; measurements, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; pls_format)
  
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Get the proportion of the predictor variance that is explained&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# by the model for different number of components. &lt;/span&gt;
  xve &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;explvar&lt;/span&gt;(mod)&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt; 

  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# To do the same for the outcome, it is more complex. This code &lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# was extracted from pls:::summary.mvr. &lt;/span&gt;
  explained &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;drop&lt;/span&gt;(pls&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;R2&lt;/span&gt;(mod, estimate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;train&amp;#34;&lt;/span&gt;, intercept &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;val) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# transpose so that components are in rows&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;t&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
    &lt;span style=&#34;color:#00f&#34;&gt;as_tibble&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Add the predictor proportions&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(predictors &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;cumsum&lt;/span&gt;(xve) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.vector&lt;/span&gt;(),
           components &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;seq_along&lt;/span&gt;(xve)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
    &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Put into a tidy format that is tall&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(
      cols &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;components),
      names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;source&amp;#34;&lt;/span&gt;,
      values_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;proportion&amp;#34;&lt;/span&gt;
    )
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We compute this data frame for each resample and save the results in the different columns.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  folds &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(var &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(recipes, get_var_explained),
         var &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;unname&lt;/span&gt;(var))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To extract and aggregate these data, simple row binding can be used to stack the data vertically. Most of the action happens in the first 15 components so let&amp;rsquo;s filter the data and compute the &lt;em&gt;average&lt;/em&gt; proportion.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;variance_data &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;bind_rows&lt;/span&gt;(folds[[&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;var&amp;#34;&lt;/span&gt;]]) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;filter&lt;/span&gt;(components &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;15&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;group_by&lt;/span&gt;(components, source) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;summarize&lt;/span&gt;(proportion &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(proportion))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The plot below shows that, if the protein measurement is important, you might require 10 or so components to achieve a good representation of that outcome. Note that the predictor variance is captured extremely well using a single component. This is due to the high degree of correlation in those data.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(variance_data, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; components, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; proportion, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; source)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() 
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/plot-1.svg&#34; width=&#34;100%&#34; /&gt;&lt;/p&gt;
&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  modeldata  * 0.0.1   2019-12-06 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  pls        * 2.7-2   2019-10-01 [1] CRAN (R 3.6.0)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tidyr      * 1.0.2   2020-01-24 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>Statistical analysis of contingency tables</title>
      <link>/learn/statistics/xtabs/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/learn/statistics/xtabs/</guid>
      <description>&lt;h2 id=&#34;introduction&#34;&gt;Introduction&lt;/h2&gt;
&lt;p&gt;This article only requires that you have the tidymodels package installed.&lt;/p&gt;
&lt;p&gt;In this vignette, we&amp;rsquo;ll walk through conducting a &lt;code&gt;\(\chi^2\)&lt;/code&gt; (chi-squared) test of independence and a chi-squared goodness of fit test using infer. We&amp;rsquo;ll start out with a chi-squared test of independence, which can be used to test the association between two categorical variables. Then, we&amp;rsquo;ll move on to a chi-squared goodness of fit test, which tests how well the distribution of one categorical variable can be approximated by some theoretical distribution.&lt;/p&gt;
&lt;p&gt;Throughout this vignette, we&amp;rsquo;ll make use of the &lt;code&gt;ad_data&lt;/code&gt; data set (available in the modeldata package, which is part of tidymodels). This data set is related to cognitive impairment in 333 patients from 
&lt;a href=&#34;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3079734/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Craig-Schapiro &lt;em&gt;et al&lt;/em&gt; (2011)&lt;/a&gt;. See &lt;code&gt;?ad_data&lt;/code&gt; for more information on the variables included and their source. One of the main research questions in these data were how a person&amp;rsquo;s genetics related to the Apolipoprotein E gene affect their cognitive skills. The data shows:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels) &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Includes the infer package&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(ad_data, package &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;modeldata&amp;#34;&lt;/span&gt;)
ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(Genotype, Class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 333 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    Genotype Class   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;fct&amp;gt;    &amp;lt;fct&amp;gt;   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 E3E3     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 E3E4     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 E3E4     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 E3E4     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 E3E3     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 E4E4     Impaired&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 E2E3     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 E2E3     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 E3E3     Control &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 E2E3     Impaired&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 323 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The three main genetic variants are called E2, E3, and E4. The values in &lt;code&gt;Genotype&lt;/code&gt; represent the genetic makeup of patients based on what they inherited from their parents (i.e, a value of &amp;ldquo;E2E4&amp;rdquo; means E2 from one parent and E4 from the other).&lt;/p&gt;
&lt;h2 id=&#34;test-of-independence&#34;&gt;Test of independence&lt;/h2&gt;
&lt;p&gt;To carry out a chi-squared test of independence, we&amp;rsquo;ll examine the association between their cognitive ability (impaired and healthy) and the genetic makeup. This is what the relationship looks like in the sample data:&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;figs/plot-indep-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;If there were no relationship, we would expect to see the purple bars reaching to the same length, regardless of cognitive ability. Are the differences we see here, though, just due to random noise?&lt;/p&gt;
&lt;p&gt;First, to calculate the observed statistic, we can use &lt;code&gt;specify()&lt;/code&gt; and &lt;code&gt;calculate()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# calculate the observed statistic&lt;/span&gt;
observed_indep_statistic &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(Genotype &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Chisq&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The observed &lt;code&gt;\(\chi^2\)&lt;/code&gt; statistic is 21.577. Now, we want to compare this statistic to a null distribution, generated under the assumption that these variables are not actually related, to get a sense of how likely it would be for us to see this observed statistic if there were actually no association between cognitive ability and genetics.&lt;/p&gt;
&lt;p&gt;We can &lt;code&gt;generate()&lt;/code&gt; the null distribution in one of two ways: using randomization or theory-based methods. The randomization approach permutes the response and explanatory variables, so that each person&amp;rsquo;s genetics is matched up with a random cognitive rating from the sample in order to break up any association between the two.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# generate the null distribution using randomization&lt;/span&gt;
null_distribution_simulated &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(Genotype &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;permute&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Chisq&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that, in the line &lt;code&gt;specify(Genotype ~ Class)&lt;/code&gt; above, we could use the equivalent syntax &lt;code&gt;specify(response = Genotype, explanatory = Class)&lt;/code&gt;. The same goes in the code below, which generates the null distribution using theory-based methods instead of randomization.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# generate the null distribution by theoretical approximation&lt;/span&gt;
null_distribution_theoretical &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(Genotype &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# note that we skip the generation step here!&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Chisq&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To get a sense for what these distributions look like, and where our observed statistic falls, we can use &lt;code&gt;visualize()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# visualize the null distribution and test statistic!&lt;/span&gt;
null_distribution_simulated &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(observed_indep_statistic,
                direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize-indep-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We could also visualize the observed statistic against the theoretical null distribution. Note that we skip the &lt;code&gt;generate()&lt;/code&gt; and &lt;code&gt;calculate()&lt;/code&gt; steps when using the theoretical approach, and that we now need to provide &lt;code&gt;method = &amp;quot;theoretical&amp;quot;&lt;/code&gt; to &lt;code&gt;visualize()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# visualize the theoretical null distribution and test statistic!&lt;/span&gt;
ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(Genotype &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;independence&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;(method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;theoretical&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(observed_indep_statistic,
                direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize-indep-theor-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;To visualize both the randomization-based and theoretical null distributions to get a sense of how the two relate, we can pipe the randomization-based null distribution into &lt;code&gt;visualize()&lt;/code&gt;, and further provide &lt;code&gt;method = &amp;quot;both&amp;quot;&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# visualize both null distributions and the test statistic!&lt;/span&gt;
null_distribution_simulated &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;(method &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;both&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(observed_indep_statistic,
                direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize-indep-both-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Either way, it looks like our observed test statistic would be fairly unlikely if there were actually no association between cognition and genotype. More exactly, we can calculate the p-value:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# calculate the p value from the observed statistic and null distribution&lt;/span&gt;
p_value_independence &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; null_distribution_simulated &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;get_p_value&lt;/span&gt;(obs_stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; observed_indep_statistic,
              direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)

p_value_independence
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    p_value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 0.000600&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Thus, if there were really no relationship between cognition and genotype, the probability that we would see a statistic as or more extreme than 21.577 is approximately 6\times 10^{-4}.&lt;/p&gt;
&lt;p&gt;Note that, equivalently to the steps shown above, the package supplies a wrapper function, &lt;code&gt;chisq_test&lt;/code&gt;, to carry out Chi-Squared tests of independence on tidy data. The syntax goes like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;chisq_test&lt;/span&gt;(ad_data, Genotype &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; Class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   statistic chisq_df  p_value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       &amp;lt;dbl&amp;gt;    &amp;lt;int&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1      21.6        5 0.000630&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;goodness-of-fit&#34;&gt;Goodness of fit&lt;/h2&gt;
&lt;p&gt;Now, moving on to a chi-squared goodness of fit test, we&amp;rsquo;ll take a look at just the genotype data. Many papers have investigated the relationship of Apolipoprotein E to diseases. For example, 
&lt;a href=&#34;https://annals.org/aim/article-abstract/717641/meta-analysis-apolipoprotein-e-genotypes-risk-coronary-heart-disease&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Song &lt;em&gt;et al&lt;/em&gt; (2004)&lt;/a&gt; conducted a meta-analysis of numerous studies that looked at this gene and heart disease. In their paper, they describe the frequency of the different genotypes across many samples. For the cognition study, it might be interesting to see if our sample of genotypes was consistent with this literature (treating the rates, for this analysis, as known).&lt;/p&gt;
&lt;p&gt;The rates of the meta-analysis and our observed data are:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Song, Y., Stampfer, M. J., &amp;amp; Liu, S. (2004). Meta-Analysis: Apolipoprotein E &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Genotypes and Risk for Coronary Heart Disease. Annals of Internal Medicine, &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# 141(2), 137.&lt;/span&gt;
meta_rates &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E2E2&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.71&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E2E3&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;11.4&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E2E4&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2.32&lt;/span&gt;,
                &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E3E3&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;61.0&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E3E4&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;22.6&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;E4E4&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2.22&lt;/span&gt;)
meta_rates &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; meta_rates&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(meta_rates) &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# these add up to slightly &amp;gt; 100%&lt;/span&gt;

obs_rates &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;(ad_data&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;Genotype)&lt;span style=&#34;color:#666&#34;&gt;/&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(ad_data)
&lt;span style=&#34;color:#00f&#34;&gt;round&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;cbind&lt;/span&gt;(obs_rates, meta_rates) &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      obs_rates meta_rates&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E2E2       0.6       0.71&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E2E3      11.1      11.37&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E2E4       2.4       2.31&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E3E3      50.1      60.85&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E3E4      31.8      22.54&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; E4E4       3.9       2.21&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Suppose our null hypothesis is that &lt;code&gt;Genotype&lt;/code&gt; follows the same frequency distribution as the meta-analysis. Lets now test whether this difference in distributions is statistically significant.&lt;/p&gt;
&lt;p&gt;First, to carry out this hypothesis test, we would calculate our observed statistic.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# calculating the null distribution&lt;/span&gt;
observed_gof_statistic &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Genotype) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, p &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meta_rates) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Chisq&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The observed statistic is 23.384. Now, generating a null distribution, by just dropping in a call to &lt;code&gt;generate()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# generating a null distribution&lt;/span&gt;
null_distribution_gof &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; ad_data &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;specify&lt;/span&gt;(response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Genotype) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;hypothesize&lt;/span&gt;(null &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;point&amp;#34;&lt;/span&gt;, p &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meta_rates) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;generate&lt;/span&gt;(reps &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5000&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;simulate&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;calculate&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Chisq&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Again, to get a sense for what these distributions look like, and where our observed statistic falls, we can use &lt;code&gt;visualize()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# visualize the null distribution and test statistic!&lt;/span&gt;
null_distribution_gof &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;visualize&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;shade_p_value&lt;/span&gt;(observed_gof_statistic,
                direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/visualize-indep-gof-1.svg&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;This statistic seems like it would be unlikely if our rates were the same as the rates from the meta-analysis! How unlikely, though? Calculating the p-value:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# calculate the p-value&lt;/span&gt;
p_value_gof &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; null_distribution_gof &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;get_p_value&lt;/span&gt;(observed_gof_statistic,
              direction &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;greater&amp;#34;&lt;/span&gt;)

p_value_gof
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   p_value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1  0.0008&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Thus, if each genotype occurred at the same rate as the Song paper, the probability that we would see a distribution like the one we did is approximately 8\times 10^{-4}.&lt;/p&gt;
&lt;p&gt;Again, equivalently to the steps shown above, the package supplies a wrapper function, &lt;code&gt;chisq_test&lt;/code&gt;, to carry out chi-squared goodness of fit tests on tidy data. The syntax goes like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;chisq_test&lt;/span&gt;(ad_data, response &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Genotype, p &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meta_rates)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   statistic chisq_df  p_value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1      23.4        5 0.000285&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;session-information&#34;&gt;Session information&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────
#&amp;gt;  setting  value                       
#&amp;gt;  version  R version 3.6.2 (2019-12-12)
#&amp;gt;  os       macOS Mojave 10.14.6        
#&amp;gt;  system   x86_64, darwin15.6.0        
#&amp;gt;  ui       X11                         
#&amp;gt;  language (EN)                        
#&amp;gt;  collate  en_US.UTF-8                 
#&amp;gt;  ctype    en_US.UTF-8                 
#&amp;gt;  tz       America/Denver              
#&amp;gt;  date     2020-04-17                  
#&amp;gt; 
#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────
#&amp;gt;  package    * version date       lib source        
#&amp;gt;  broom      * 0.5.5   2020-02-29 [1] CRAN (R 3.6.0)
#&amp;gt;  dials      * 0.0.6   2020-04-03 [1] CRAN (R 3.6.2)
#&amp;gt;  dplyr      * 0.8.5   2020-03-07 [1] CRAN (R 3.6.0)
#&amp;gt;  ggplot2    * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)
#&amp;gt;  infer      * 0.5.1   2019-11-19 [1] CRAN (R 3.6.0)
#&amp;gt;  parsnip    * 0.1.0   2020-04-09 [1] CRAN (R 3.6.2)
#&amp;gt;  purrr      * 0.3.3   2019-10-18 [1] CRAN (R 3.6.0)
#&amp;gt;  recipes    * 0.1.10  2020-03-18 [1] CRAN (R 3.6.0)
#&amp;gt;  rlang        0.4.5   2020-03-01 [1] CRAN (R 3.6.0)
#&amp;gt;  rsample    * 0.0.6   2020-03-31 [1] CRAN (R 3.6.2)
#&amp;gt;  tibble     * 2.1.3   2019-06-06 [1] CRAN (R 3.6.2)
#&amp;gt;  tidymodels * 0.1.0   2020-02-16 [1] CRAN (R 3.6.0)
#&amp;gt;  tune       * 0.1.0   2020-04-02 [1] CRAN (R 3.6.2)
#&amp;gt;  workflows  * 0.1.1   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt;  yardstick  * 0.0.6   2020-03-17 [1] CRAN (R 3.6.0)
#&amp;gt; 
#&amp;gt; [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
&lt;/code&gt;&lt;/pre&gt;</description>
    </item>
    
    <item>
      <title>recipes 0.1.13</title>
      <link>/blog/2020/06/recipes-0-1-13/</link>
      <pubDate>Thu, 25 Jun 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/06/recipes-0-1-13/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re very chuffed to announce the release of 
&lt;a href=&#34;https://recipes.tidymodels.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipes&lt;/a&gt; 0.1.13. recipes is an alternative method for creating and preprocessing design matrices that can be used for modeling or visualization.&lt;/p&gt;
&lt;p&gt;You can install it from CRAN with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;recipes&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You can see a full list of changes in the 
&lt;a href=&#34;https://recipes.tidymodels.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;release notes&lt;/a&gt;. There are some improvements and changes to talk about.&lt;/p&gt;
&lt;h1 id=&#34;general-changes&#34;&gt;General changes&lt;/h1&gt;
&lt;p&gt;First, &lt;code&gt;step_filter()&lt;/code&gt;, &lt;code&gt;step_slice()&lt;/code&gt;, &lt;code&gt;step_sample()&lt;/code&gt;, and &lt;code&gt;step_naomit()&lt;/code&gt; had their defaults for &lt;code&gt;skip&lt;/code&gt; changed to &lt;code&gt;TRUE&lt;/code&gt;. In the vast majority of applications, these steps should not be applied to the test or assessment sets.&lt;/p&gt;
&lt;p&gt;Also, &lt;code&gt;step_upsample()&lt;/code&gt; and &lt;code&gt;step_downsample()&lt;/code&gt; are soft deprecated in recipes as they are now available in the 
&lt;a href=&#34;https://tidymodels.github.io/themis/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;themis package&lt;/a&gt;. They will be removed in the next version.&lt;/p&gt;
&lt;p&gt;Finally, for the new version of dplyr, the selectors &lt;code&gt;all_of()&lt;/code&gt; and &lt;code&gt;any_of()&lt;/code&gt; can now be used in step selections.&lt;/p&gt;
&lt;h1 id=&#34;feature-extraction-improvements&#34;&gt;Feature extraction improvements&lt;/h1&gt;
&lt;p&gt;In the 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Feature_extraction&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;feature extraction&lt;/em&gt;&lt;/a&gt; category, there are two improvements. First, the &lt;code&gt;tidy()&lt;/code&gt; method for &lt;code&gt;step_pca()&lt;/code&gt; can return the percentage of variation accounted for by each 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Principal_component_analysis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;PCA component&lt;/a&gt;. For example:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Many highly correlated numeric predictors:&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(meats, package &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;modeldata&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2383&lt;/span&gt;)
split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(meats)
meat_tr &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(split)
meat_te &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(split)

pca_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(water &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; fat &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; protein &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meat_tr) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_pca&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;(), num_comp &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;, id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pca&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;()

var_info &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tidy&lt;/span&gt;(pca_rec, id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pca&amp;#34;&lt;/span&gt;, type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;variance&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;(var_info&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;terms)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;pre&gt;&lt;code&gt;## 
## cumulative percent variance         cumulative variance 
##                         100                         100 
##            percent variance                    variance 
##                         100                         100
&lt;/code&gt;&lt;/pre&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;var_info &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  dplyr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;filter&lt;/span&gt;(terms &lt;span style=&#34;color:#666&#34;&gt;==&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;percent variance&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; component, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; value)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_bar&lt;/span&gt;(stat &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;identity&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;xlim&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ylab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;% of Total Variation&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figure/pca-tidy-1.svg&#34; alt=&#34;plot of chunk pca-tidy&#34;&gt;&lt;/p&gt;
&lt;p&gt;Another change in this version of recipes is that &lt;code&gt;step_pls()&lt;/code&gt; has received an upgrade. 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Partial_least_squares_regression&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Partial least squares&lt;/a&gt; (PLS) is similar to PCA but takes the outcome(s) into account.&lt;/p&gt;
&lt;p&gt;Previously, it used the 
&lt;a href=&#34;https://github.com/bhmevik/pls&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pls package&lt;/a&gt; to do the computations. That&amp;rsquo;s a great package but it lacks two important features: allow for a categorical outcome value (e.g. &amp;ldquo;pls-da&amp;rdquo; for &lt;em&gt;discriminant analysis&lt;/em&gt;) or allow for sparsity in the coefficients. Sparsity would facilitate simpler and perhaps more robust models.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;step_pls()&lt;/code&gt; now uses the Bioconductor 
&lt;a href=&#34;https://www.bioconductor.org/packages/release/bioc/html/mixOmics.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;mixOmics package&lt;/a&gt;. As such, the outcome data can now be a factor and a new argument &lt;code&gt;predictor_prop&lt;/code&gt; is used for sparsity. That argument specifies the maximum proportion of partial least squares loadings that will be &lt;em&gt;non-zero&lt;/em&gt; (per component) during training. Newly prepped recipes will use this package but previously created recipes still use the pls package.
For our previous example, let&amp;rsquo;s look at the protein outcome and build a recipe:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;pls_rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(water &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; fat &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; protein &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; meat_tr) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;step_pls&lt;/span&gt;(
    &lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;(),
    outcome &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vars&lt;/span&gt;(protein),
    num_comp &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;,
    predictor_prop &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.75&lt;/span&gt;,
    id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pls&amp;#34;&lt;/span&gt;
  ) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;()

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# for new data: &lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;bake&lt;/span&gt;(pls_rec, meat_te, protein, &lt;span style=&#34;color:#00f&#34;&gt;starts_with&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;PLS&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  tidyr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(cols &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;protein),
                      names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;component&amp;#34;&lt;/span&gt;,
                      values_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;values&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; values, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; protein)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;facet_wrap&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; component, scale &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;free_x&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;xlab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;PLS Score&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figure/pls-1.svg&#34; title=&#34;plot of chunk pls&#34; alt=&#34;plot of chunk pls&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What are the PLS coefficients from this?&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;tidy&lt;/span&gt;(pls_rec, id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pls&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; component, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; terms, fill &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; value)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_tile&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;scale_fill_gradient2&lt;/span&gt;(
    low &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;#B2182B&amp;#34;&lt;/span&gt;,
    mid &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;white&amp;#34;&lt;/span&gt;,
    high &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;#2166AC&amp;#34;&lt;/span&gt;,
    limits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-0.4&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;0.4&lt;/span&gt;)
  ) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;theme&lt;/span&gt;(axis.text.y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;element_blank&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; 
  &lt;span style=&#34;color:#00f&#34;&gt;ylab&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Predictors&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figure/pls-coef-1.svg&#34; title=&#34;plot of chunk pls-coef&#34; alt=&#34;plot of chunk pls-coef&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;The third component has the largest coefficients and the largest effect on predicting the percentage of protein. This is consistent with the scatter plot above. The blocks of white in the heatmap above are coefficients effected by the sparsity argument.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>tidyr 1.1.0</title>
      <link>/blog/2020/05/tidyr-1.1.0/</link>
      <pubDate>Tue, 26 May 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/05/tidyr-1.1.0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re delighted to announce that 
&lt;a href=&#34;http://tidyr.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidyr&lt;/a&gt; 1.1.0 is now available from CRAN. tidyr provides a set of tools for transforming data frames to and from tidy data, where each variable is a column and each observation is a row. Tidy data is a convention for matching the semantics and structure of your data that makes using the rest of the tidyverse (and many other R packages) much easier.&lt;/p&gt;
&lt;p&gt;You can install install tidyr with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;tidyr&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This release doesn&amp;rsquo;t include any major new excitement but it includes a whole passel of minor improvements building on the major changes in 
&lt;a href=&#34;https://www.tidyverse.org/blog/2019/09/tidyr-1-0-0/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidyr 1.0.0&lt;/a&gt;, and generally making everything easier to use and a bit more flexible. In this blog post, I&amp;rsquo;ll give a quick run down on new pivoting features; see the 
&lt;a href=&#34;https://github.com/tidyverse/tidyr/releases/tag/v1.1.0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;full release announcement&lt;/a&gt; for the details of other changes.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidyr)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;pivot_longer&#34;&gt;&lt;code&gt;pivot_longer()&lt;/code&gt;&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_longer()&lt;/code&gt; gains a new &lt;code&gt;names_transform&lt;/code&gt; argument that allows you to
transform column names before they turn into data. For example, you can
use this new argument along with 
&lt;a href=&#34;https://readr.tidyverse.org/reference/parse_number.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;readr::parse_number()&lt;/code&gt;&lt;/a&gt; to
parse column names that really should be numbers:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, wk1 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, wk2 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, wk3 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;9&lt;/span&gt;, wk4 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;25&lt;/span&gt;)
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(
  cols &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;starts_with&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;wk&amp;#34;&lt;/span&gt;),
  names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;week&amp;#34;&lt;/span&gt;,
  names_transform &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(week &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; readr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;parse_number),
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id  week value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1     1     0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1     2     4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     1     3     9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     1     4    25&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_longer()&lt;/code&gt; can now discard uninformative column names by setting
&lt;code&gt;names_to = character()&lt;/code&gt;, thanks to idea and implementation from

&lt;a href=&#34;https://github.com/mitchelloharawild&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mitch O&amp;rsquo;Hara Wild&lt;/a&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, fruitful_panda &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, angry_aardvark &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;)  
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;id, names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;character&lt;/span&gt;())
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1     3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1     5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     2     4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2     6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_longer()&lt;/code&gt; no longer creates a &lt;code&gt;.copy&lt;/code&gt; variable in the presence of
duplicate column names. This makes it more consistent with the handling
of non-unique pivot specifications.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;, .name_repair &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;minimal&amp;#34;&lt;/span&gt;)  
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;id)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id name  value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1 x         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1 x         4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     2 x         2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2 x         5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5     3 x         3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6     3 x         6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_longer()&lt;/code&gt; automatically disambiguates non-unique outputs, which can
occur when the input variables include some additional component that you
don&amp;rsquo;t care about and want to discard. You can discard parts of column names
either with &lt;code&gt;names_pattern&lt;/code&gt; or with &lt;code&gt;NA&lt;/code&gt; in &lt;code&gt;names_to&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, x_1 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, y_2 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;, y_3 &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;9&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;11&lt;/span&gt;)
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;id, names_pattern &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;(.)_.&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 9 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id name  value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1 x         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1 y         4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     1 y         9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2 x         2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5     2 y         5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6     2 y        10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 7     3 x         3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 8     3 y         6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 9     3 y        11&lt;/span&gt;
    
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;id, names_sep &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;_&amp;#34;&lt;/span&gt;, names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;name&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;NA&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 9 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id name  value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1 x         1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1 y         4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     1 y         9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2 x         2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5     2 y         5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6     2 y        10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 7     3 x         3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 8     3 y         6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 9     3 y        11&lt;/span&gt;
    
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_longer&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;id, names_sep &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;_&amp;#34;&lt;/span&gt;, names_to &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;.value&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;NA&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      id     x     y&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1     1     4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     1    NA     9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     2     2     5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2    NA    10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5     3     3     6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6     3    NA    11&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;pivot_wider&#34;&gt;&lt;code&gt;pivot_wider()&lt;/code&gt;&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_wider()&lt;/code&gt; gains a &lt;code&gt;names_sort&lt;/code&gt; argument which allows you to sort
column names in order. The default, &lt;code&gt;FALSE&lt;/code&gt;, orders columns by their
first appearance. I&amp;rsquo;m considering changing the default value to &lt;code&gt;TRUE&lt;/code&gt;
in a future version.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(
  day_int &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;),
  day_fac &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;factor&lt;/span&gt;(day_int, labels &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Mon&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Tue&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Wed&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Thu&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Fri&amp;#34;&lt;/span&gt;))
)
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_wider&lt;/span&gt;(
  names_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; day_fac, 
  values_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; day_int
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Thu   Wed   Fri   Mon   Tue&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     4     3     5     1     2&lt;/span&gt;
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_wider&lt;/span&gt;(
  names_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; day_fac,
  names_sort &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;,
  values_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; day_int
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Mon   Tue   Wed   Thu   Fri&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1     2     3     4     5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_wider()&lt;/code&gt; gains a &lt;code&gt;names_glue&lt;/code&gt; argument that allows you to construct
output column names with a glue specification when &lt;code&gt;names_to&lt;/code&gt; includes
multiple columns.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(
  first &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;a&amp;#34;&lt;/span&gt;,
  second &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;1&amp;#34;&lt;/span&gt;,
  third &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;X&amp;#34;&lt;/span&gt;,
  val &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
)
df &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;pivot_wider&lt;/span&gt;(
  names_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(first, second, third), 
  values_from &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; val,
  names_glue &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;{first}.{second}_{third}&amp;#34;&lt;/span&gt;
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   a.1_X&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pivot_wider()&lt;/code&gt; arguments &lt;code&gt;values_fn&lt;/code&gt; and &lt;code&gt;values_fill&lt;/code&gt; can now be single
values; you now only need to use a named list if you want to use different
values for different value columns. You&amp;rsquo;ll also get better errors if
they&amp;rsquo;re not of the correct type.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Finally, both &lt;code&gt;pivot_wider()&lt;/code&gt; and &lt;code&gt;pivot_longer()&lt;/code&gt; are considerably more
performant, thanks largely to improvements in the underlying vctrs code
by 
&lt;a href=&#34;http://github.com/DavisVaughan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Davis Vaughn&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;Thanks to all 135 people who contributed to this version of tidyr by discussing issues and suggesting new code! 
&lt;a href=&#34;https://github.com/abichat&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@abichat&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/abiyug&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@abiyug&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/adisarid&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@adisarid&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ahmohamed&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ahmohamed&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/akikirinrin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@akikirinrin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/albertotb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@albertotb&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/alex-pax&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@alex-pax&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/amirmazmi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@amirmazmi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/andtheWings&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@andtheWings&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ashiklom&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ashiklom&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/atusy&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@atusy&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bertrandh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@bertrandh&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/BillBlanc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@BillBlanc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/billdenney&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@billdenney&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/BrianDiggs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@BrianDiggs&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bushdanielkwajaffa&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@bushdanielkwajaffa&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cderv&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cderv&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/CGMossa&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@CGMossa&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cgoo4&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cgoo4&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/charliejhadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@charliejhadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/chester-gan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@chester-gan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cimentadaj&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cimentadaj&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cjvanlissa&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cjvanlissa&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cloversleaves&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cloversleaves&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/colearendt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@colearendt&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dah33&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dah33&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DanOvando&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DanOvando&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dapperjapper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dapperjapper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/daranzolin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@daranzolin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/davidhunterwalsh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@davidhunterwalsh&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/davisadamw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@davisadamw&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DavisVaughan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DavisVaughan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dchiu911&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dchiu911&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dpastoor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dpastoor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dpeterson71&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dpeterson71&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dpprdan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dpprdan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/eantworth&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@eantworth&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/earcanal&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@earcanal&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/echasnovski&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@echasnovski&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/enixam&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@enixam&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ericgunnink&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ericgunnink&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/florianm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@florianm&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fmmattioni&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fmmattioni&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/franzbischoff&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@franzbischoff&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/GegznaV&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@GegznaV&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/geotheory&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@geotheory&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ggrothendieck&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ggrothendieck&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gregorp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gregorp&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/HanOostdijk&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@HanOostdijk&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/henry090&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@henry090&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/iago-pssjd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@iago-pssjd&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ifellows&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ifellows&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/infotroph&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@infotroph&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jam1015&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jam1015&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jannikbuhr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jannikbuhr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jasonpcasey&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jasonpcasey&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jeffreypullin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jeffreypullin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jennybc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jennybc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jenren&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jenren&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/JenspederM&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@JenspederM&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jeonghyunwoo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jeonghyunwoo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jjnote&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jjnote&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jmh530&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jmh530&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/JohnCoene&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@JohnCoene&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/joshua-theisen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@joshua-theisen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/JosiahParry&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@JosiahParry&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jthomasmock&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jthomasmock&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jwilliman&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jwilliman&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kaneplusplus&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kaneplusplus&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kaybenleroll&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kaybenleroll&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kent37&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kent37&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kiernann&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kiernann&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/krlmlr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@krlmlr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lionel-&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lionel-&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Ljupch0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Ljupch0&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lymanmark&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lymanmark&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maelle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maelle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/majazaloznik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@majazaloznik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mattantaliss&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mattantaliss&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mattwarkentin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mattwarkentin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maurolepore&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maurolepore&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/md0u80c9&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@md0u80c9&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mgirlich&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mgirlich&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MikeEdinger&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MikeEdinger&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikemahoney218&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mikemahoney218&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikmart&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mikmart&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mitchelloharawild&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mitchelloharawild&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/moodymudskipper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@moodymudskipper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/msberends&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@msberends&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/msgoussi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@msgoussi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mstackhouse&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mstackhouse&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MyKo101&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MyKo101&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nacnudus&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nacnudus&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/namelessjon&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@namelessjon&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ndrewGele&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ndrewGele&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Nicktz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Nicktz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/npjc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@npjc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/osorensen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@osorensen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/PathosEthosLogos&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@PathosEthosLogos&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/philipp-baumann&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@philipp-baumann&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/PMSeitzer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@PMSeitzer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/psychelzh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@psychelzh&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/randomgambit&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@randomgambit&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/riinuots&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@riinuots&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/romagnolid&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@romagnolid&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/romainfrancois&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@romainfrancois&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rvino&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rvino&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/salim-b&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@salim-b&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/shanepiesik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@shanepiesik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/shannonpileggi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@shannonpileggi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/sharleenw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@sharleenw&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/siddharthprabhu&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@siddharthprabhu&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/simazhi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@simazhi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/skr5k&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@skr5k&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/skydavis435&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@skydavis435&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/smingerson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@smingerson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/smithjd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@smithjd&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/srnnkls&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@srnnkls&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stragu&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stragu&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stufield&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stufield&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tangcxx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@tangcxx&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tdhock&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@tdhock&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/the-Zian&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@the-Zian&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tomhopper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@tomhopper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/topepo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@topepo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wgrundlingh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wgrundlingh&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wibeasley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wibeasley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/william3031&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@william3031&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wmoldham&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wmoldham&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wolski&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wolski&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/xkdog&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@xkdog&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/xtimbeau&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@xtimbeau&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/yusuzech&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yusuzech&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>rules 0.0.1</title>
      <link>/blog/2020/05/rules-0-0-1/</link>
      <pubDate>Thu, 21 May 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/05/rules-0-0-1/</guid>
      <description>&lt;p&gt;We are happy to announce the release of the 
&lt;a href=&#34;https://rules.tidymodels.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rules package&lt;/a&gt; on 
&lt;a href=&#34;https://cran.r-project.org/package=rules&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CRAN&lt;/a&gt;. rules is another &amp;ldquo;parsnip-adjacent&amp;rdquo; package that enables a specific class of models within the tidymodels infrastructure.  rules currently contains three models:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;C5_rules()&lt;/code&gt;: classification rule sets based on the C5.0 model.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;cubist_rules()&lt;/code&gt;: regression rules using Cubist.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;rule_fit()&lt;/code&gt;: classification or regression rules using the RuleFit model.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If you aren&amp;rsquo;t familiar with rule-based models, there is a 
&lt;a href=&#34;https://rviews.rstudio.com/2020/05/21/modern-rule-based-models/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;companion blog post&lt;/a&gt; that summarizes how they work.&lt;/p&gt;
&lt;p&gt;Install rules from CRAN like so:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;rules&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Then attach it for use via:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(rules)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Here&amp;rsquo;s an example of creating Cubist regression rules via the 
&lt;a href=&#34;https://tune.tidymodels.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;parsnip package&lt;/a&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tidymodels)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Attaching packages ──────────────────────────────────── tidymodels 0.1.0 ──&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ broom     0.5.6      ✓ recipes   0.1.12&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ dials     0.0.6      ✓ rsample   0.0.6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ dplyr     0.8.5      ✓ tibble    3.0.1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ ggplot2   3.3.0      ✓ tune      0.1.0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ infer     0.5.1      ✓ workflows 0.1.1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ parsnip   0.1.1      ✓ yardstick 0.0.6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ✓ purrr     0.3.4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ── Conflicts ─────────────────────────────────────── tidymodels_conflicts() ──&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x purrr::accumulate() masks foreach::accumulate()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x purrr::discard()    masks scales::discard()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x dplyr::filter()     masks stats::filter()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x dplyr::lag()        masks stats::lag()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x ggplot2::margin()   masks dials::margin()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x recipes::step()     masks stats::step()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x purrr::when()       masks foreach::when()&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(rules)

&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(car_prices, package &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;modeldata&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;9932&lt;/span&gt;)
car_split &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;initial_split&lt;/span&gt;(car_prices)
car_tr &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;training&lt;/span&gt;(car_split)
car_te &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;  &lt;span style=&#34;color:#00f&#34;&gt;testing&lt;/span&gt;(car_split)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# A single rule set:&lt;/span&gt;
cubist_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;cubist_rules&lt;/span&gt;(neighbors &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;7&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Cubist&amp;#34;&lt;/span&gt;)

cubist_fit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  cubist_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; car_tr)

&lt;span style=&#34;color:#00f&#34;&gt;summary&lt;/span&gt;(cubist_fit&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;fit)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Call:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; cubist.default(x = x, y = y, committees = 1)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Cubist [Release 2.07 GPL Edition]  Wed May 20 21:39:22 2020&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ---------------------------------&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Target attribute `outcome&amp;#39;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Read 603 cases (18 attributes) from undefined.data&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Model:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 1: [210 cases, mean 4.116360, range 3.94295 to 4.2505, est err 0.030756]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;lt;= 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Saab &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.115185 + 0.12 Saab - 3.5e-06 Mileage + 0.017 Cylinder&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 0.087 hatchback - 0.029 Chevy + 0.046 wagon + 0.028 Leather&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.041 Cadillac - 0.024 sedan + 0.027 convertible&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.006 Doors + 0.012 Buick&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 2: [8 cases, mean 4.207121, range 4.13308 to 4.26696, est err 0.006589]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Saturn &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 3.88624 + 0.057 Cylinder + 0.2 Saab + 0.141 Cadillac&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 3.8e-06 Mileage - 0.054 sedan + 0.094 convertible&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 0.085 hatchback + 0.019 Doors + 0.04 Buick + 0.014 Cruise&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.01 Leather + 0.007 Sound + 0.007 Saturn&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 3: [33 cases, mean 4.229076, range 4.16741 to 4.29184, est err 0.012903]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cruise &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.265627 - 3.7e-06 Mileage + 0.039 Chevy&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 4: [94 cases, mean 4.272727, range 4.18913 to 4.4427, est err 0.034717]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Mileage &amp;gt; 3946&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Doors &amp;gt; 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cruise &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Buick &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cadillac &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Saturn &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.037203 + 0.051 Cylinder - 4.3e-06 Mileage + 0.061 Saab&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.044 Cadillac - 0.016 sedan + 0.029 convertible&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 0.026 hatchback + 0.006 Doors - 0.009 Chevy + 0.012 Buick&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.004 Cruise&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 5: [57 cases, mean 4.314541, range 4.17208 to 4.42864, est err 0.049758]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Buick &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.389884 - 3e-06 Mileage&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 6: [9 cases, mean 4.341528, range 4.23957 to 4.66962, est err 0.036309]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Mileage &amp;lt;= 3946&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cadillac &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 3.439093 + 5.28e-05 Mileage + 0.129 Cylinder&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 7: [43 cases, mean 4.354487, range 4.1778 to 4.60071, est err 0.031792]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Doors &amp;lt;= 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cruise &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	convertible &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 3.40984 + 0.13 Cylinder + 0.116 Chevy - 2.7e-06 Mileage&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.037 Sound + 0.031 Leather&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 8: [85 cases, mean 4.462877, range 4.34723 to 4.58348, est err 0.023398]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Saab &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.522928 - 3.4e-06 Mileage + 0.064 Saab - 0.021 Doors&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 0.035 sedan + 0.009 Cylinder + 0.022 Cadillac&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          - 0.024 hatchback + 0.015 convertible - 0.004 Chevy&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          + 0.006 Buick&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 9: [60 cases, mean 4.592824, range 4.44778 to 4.84976, est err 0.041948]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cadillac &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.774347 - 0.103 Doors + 0.036 Cylinder - 3.4e-06 Mileage&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   Rule 10: [7 cases, mean 4.625017, range 4.58911 to 4.6727, est err 0.006627]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     if&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cylinder &amp;gt; 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Cadillac &amp;lt;= 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	convertible &amp;gt; 0&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     then&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	outcome = 4.693132 - 3.9e-06 Mileage&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Evaluation on training data (603 cases):&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Average  |error|           0.032526&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Relative |error|               0.23&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;     Correlation coefficient        0.97&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	Attribute usage:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	  Conds  Model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   67%    84%    Cylinder&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   49%    66%    Saab&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   28%    66%    Cadillac&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   28%    17%    Cruise&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   25%    66%    Buick&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   23%    75%    Doors&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   17%   100%    Mileage&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	   17%     1%    Saturn&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	    8%    66%    convertible&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          77%    Chevy&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          66%    hatchback&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          66%    sedan&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          43%    Leather&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	          35%    wagon&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 	           8%    Sound&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Time: 0.0 secs&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(cubist_fit, car_te &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;-&lt;/span&gt;Price))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 201 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    .pred&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1  4.32&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2  4.49&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3  4.54&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4  4.54&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5  4.43&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6  4.43&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7  4.46&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8  4.44&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9  4.37&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10  4.48&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 191 more rows&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The functions also work with the 
&lt;a href=&#34;https://tune.tidymodels.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tune package&lt;/a&gt;. To optimize our model, the number of committees (similar to boosting iterations) and the number of nearest-neighbors are the primary parameters for tuning.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;cb_grid &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;expand.grid&lt;/span&gt;(committees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;30&lt;/span&gt;, neighbors &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;7&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;9&lt;/span&gt;))

&lt;span style=&#34;color:#00f&#34;&gt;set.seed&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;8226&lt;/span&gt;)
car_folds &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vfold_cv&lt;/span&gt;(car_tr)

cubist_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;cubist_rules&lt;/span&gt;(neighbors &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tune&lt;/span&gt;(), committees &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tune&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;set_engine&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Cubist&amp;#34;&lt;/span&gt;)

car_tune_res &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  cubist_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;tune_grid&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., resamples &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; car_folds, grid &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; cb_grid)

car_tune_res &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;collect_metrics&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;filter&lt;/span&gt;(.metric &lt;span style=&#34;color:#666&#34;&gt;==&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;rmse&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(neighbors &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;factor&lt;/span&gt;(neighbors)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; committees, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; mean, col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; neighbors)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_line&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;scale_color_brewer&lt;/span&gt;(palette &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Dark2&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;theme&lt;/span&gt;(legend.position &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;top&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;/blog/2020-05-rules-0-0-1/index_files/figure-html/cb-tune-1.png&#34; width=&#34;700px&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;
&lt;span style=&#34;color:#00f&#34;&gt;show_best&lt;/span&gt;(car_tune_res, metric &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;rmse&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 5 x 7&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   committees neighbors .metric .estimator   mean     n std_err&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;int&amp;gt;     &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;       &amp;lt;dbl&amp;gt; &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1          9         9 rmse    standard   0.0395    10 0.00133&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2          5         9 rmse    standard   0.0395    10 0.00132&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3         11         9 rmse    standard   0.0395    10 0.00133&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4         13         9 rmse    standard   0.0395    10 0.00132&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5          8         9 rmse    standard   0.0395    10 0.00131&lt;/span&gt;

smallest_rmse &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select_best&lt;/span&gt;(car_tune_res, metric &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;rmse&amp;#34;&lt;/span&gt;)
smallest_rmse
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   committees neighbors&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;        &amp;lt;int&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1          9         9&lt;/span&gt;

final_cb_mod &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  cubist_mod &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;finalize_model&lt;/span&gt;(smallest_rmse) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;fit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Price) &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; ., data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; car_tr)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;It appears that the benefit of using committees occurs in the first 10 iterations. The nearest-neighbor adjustment was important to obtaining good performance.&lt;/p&gt;
&lt;p&gt;The test set results look good and are consistent with the resampling estimate of RMSE:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;test_pred &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;predict&lt;/span&gt;(final_cb_mod, car_te) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;bind_cols&lt;/span&gt;(car_te &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;select&lt;/span&gt;(Price)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(Price &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;log10&lt;/span&gt;(Price))

test_pred &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;rmse&lt;/span&gt;(Price, .pred)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   .metric .estimator .estimate&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 rmse    standard      0.0382&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;ggplot&lt;/span&gt;(test_pred, &lt;span style=&#34;color:#00f&#34;&gt;aes&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; .pred, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; Price)) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_abline&lt;/span&gt;(col &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;green&amp;#34;&lt;/span&gt;, lty &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;geom_point&lt;/span&gt;(alpha &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;coord_fixed&lt;/span&gt;(ratio &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;/blog/2020-05-rules-0-0-1/index_files/figure-html/cb-test-1.png&#34; width=&#34;700px&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;d like to thank 
&lt;a href=&#34;https://github.com/holub008&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Karl Holub&lt;/a&gt; for making the 
&lt;a href=&#34;https://github.com/holub008/xrf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;xrf package&lt;/a&gt; and accepting my PRs and changes.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Updates to ragg and systemfonts</title>
      <link>/blog/2020/05/updates-to-ragg-and-systemfonts/</link>
      <pubDate>Fri, 15 May 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/05/updates-to-ragg-and-systemfonts/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re thrilled to announce the new releases of ragg and systemfonts. ragg is a package that provides a modern set of raster devices as alternatives to the ones you know from the grDevices package (e.g. &lt;code&gt;png()&lt;/code&gt;). systemfonts is a low level package that provides font-related utilities to graphic devices, most importantly detecting and matching fonts to the correct font files across systems.&lt;/p&gt;
&lt;p&gt;While these are not the most flashy of releases, we thought it would be a good time to discuss some of our thoughts on the future of graphics in R, especially related to fonts.&lt;/p&gt;
&lt;h2 id=&#34;ragg-020&#34;&gt;ragg 0.2.0&lt;/h2&gt;
&lt;p&gt;ragg is our take on a modern set of raster devices for R, based based on the AGG 2D graphics library which provides both 
&lt;a href=&#34;https://ragg.r-lib.org/articles/ragg_quality.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;high quality&lt;/a&gt; and 
&lt;a href=&#34;https://ragg.r-lib.org/articles/ragg_performance.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;high performance&lt;/a&gt; output (you can read more about it in 
&lt;a href=&#34;https://www.tidyverse.org/blog/2019/07/ragg-0-1-0/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the 0.1.0 release post&lt;/a&gt;). It also provides high quality font support and automatically finds fonts installed on your computer.&lt;/p&gt;
&lt;p&gt;This release has resolved many rendering bugs and include one major new features: the addition of a jpeg device (&lt;code&gt;agg_jpeg()&lt;/code&gt;) that produces jpegs. jpeg is a lossy image format (i.e. it decreases the image quality to reduce file size) as opposed to png which uses a lossless compression scheme. That means that jpeg is usually not the best choice for most R graphics, but it does excel in compressing image data, so for this type of plot it can offer a very high compression with almost no noticeable degradation in quality:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(ragg)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning: package &amp;#39;ragg&amp;#39; was built under R version 3.6.2&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(fs)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning: package &amp;#39;fs&amp;#39; was built under R version 3.6.2&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Creating some image data&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(ambient)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(dplyr)
noise &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;long_grid&lt;/span&gt;(
    x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, length.out &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;), 
    y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;seq&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, length.out &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;)
  ) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(
    z &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;fracture&lt;/span&gt;(
      noise &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; gen_worley, 
      fractal &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; billow, 
      octaves &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, 
      gain &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; .x &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.75&lt;/span&gt;,
      frequency &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; .x &lt;span style=&#34;color:#666&#34;&gt;*&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;,
      x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; x, 
      y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; y,
      value &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;distance&amp;#34;&lt;/span&gt;,
      seed &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;12&lt;/span&gt;
    )
  )

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# plot using png&lt;/span&gt;
pngfile &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;path&lt;/span&gt;(knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;fig_path&lt;/span&gt;(),  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;png_image.png&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;agg_png&lt;/span&gt;(pngfile, &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;plot&lt;/span&gt;(noise, z)
&lt;span style=&#34;color:#00f&#34;&gt;invisible&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;dev.off&lt;/span&gt;())
knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;include_graphics&lt;/span&gt;(pngfile)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-1-1/png_image.png&#34; width=&#34;700px&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# plot using jpeg&lt;/span&gt;
jpegfile &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;path&lt;/span&gt;(knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;fig_path&lt;/span&gt;(),  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;jpeg_image.jpeg&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;agg_jpeg&lt;/span&gt;(jpegfile, &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;plot&lt;/span&gt;(noise, z)
&lt;span style=&#34;color:#00f&#34;&gt;invisible&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;dev.off&lt;/span&gt;())

knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;include_graphics&lt;/span&gt;(jpegfile)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-2-1/jpeg_image.jpeg&#34; width=&#34;700px&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;As you can hopefully agree on, there is no noticeable quality difference between the two. What about file size?&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;file_size&lt;/span&gt;(pngfile)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 944K&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;file_size&lt;/span&gt;(jpegfile)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 387K&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We observe more than a 50% file size reduction. Again, this is only worth it when saving image-type plots with high-frequency variations. &lt;code&gt;agg_png()&lt;/code&gt; should be your go-to device for most data visualizations.&lt;/p&gt;
&lt;h2 id=&#34;systemfonts-020&#34;&gt;systemfonts 0.2.0&lt;/h2&gt;
&lt;p&gt;systemfonts is a package that spun out of the development of ragg. Its main claim-to-fame is its ability to find the correct font file for a specific font and style. This release has improved a lot of the underlying implementation and added a bunch of new features that may interest you.&lt;/p&gt;
&lt;h3 id=&#34;why-is-text-so-hard&#34;&gt;Why is text so hard?&lt;/h3&gt;
&lt;p&gt;Before we delve into the features of systemfonts it may be good to understand why fonts don&amp;rsquo;t just work. At its core text and fonts are just very messy, with differences between operating systems and font file formats to name some of the challenges. (For a humorous overview consult the 
&lt;a href=&#34;https://gankra.github.io/blah/text-hates-you/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Text Rendering Hates You&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;There are simply so many issues related to how finding the font files, reading the content, figuring out the correct glyphs for a character string, and determining the correct position of each glyph, that most applications use high-level solutions such as those provided by the operating system or e.g. 
&lt;a href=&#34;https://www.pango.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Pango&lt;/a&gt;. This is not really an option for graphic devices as text is used in many other ways than the standard &amp;ldquo;text box&amp;rdquo; use that the high-level solutions are tailored to. All-in-all this makes it rather difficult to do fully correct text rendering on graphics device.&lt;/p&gt;
&lt;h3 id=&#34;new-features&#34;&gt;New features&lt;/h3&gt;
&lt;p&gt;The core functionality of systemfonts (finding font files) has been expanded with this release. It now makes heavy use of caching to speed up the font queries that may happen many hundreds of times for a single plot. This gives a nice speed boost, but comes at a cost: if a font is installed while your R session is running, you&amp;rsquo;ll need to explicitly run &lt;code&gt;systemfonts::reset_font_cache()&lt;/code&gt; in order to flush the cache.&lt;/p&gt;
&lt;p&gt;systemfonts has also gained the ability to register fonts that are not installed in the usual place so that e.g. packages can provide a new set of fonts. This is done with the &lt;code&gt;register_font()&lt;/code&gt; function. The font registration can also be used to get access to fonts that are not easily targeted with the limited information you can pass through the graphic engine in R (family, bold-or-not, italic-or-not), or if you want to overwrite the system-specific mappings of &lt;code&gt;&#39;sans&#39;&lt;/code&gt;, &lt;code&gt;&#39;serif&#39;&lt;/code&gt;, and &lt;code&gt;&#39;mono&#39;&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Apart from font file matching, systemfonts now also provides the basic infrastructure for text shaping, that is, figuring out the placement of glyphs on a page, taking size, kerning, etc. into account. It further provides efficient functions for calculating string widths and glyph dimensions, something needed if you choose to implement a graphic device.&lt;/p&gt;
&lt;p&gt;The text shaping, font information, and glyph metrics are available from R, although the intended use is primarily from C:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(systemfonts)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning: package &amp;#39;systemfonts&amp;#39; was built under R version 3.6.2&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;font_info&lt;/span&gt;(family &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;spectral&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 22&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   path  index family style italic bold  monospace kerning color scalable&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt; &amp;lt;lgl&amp;gt;  &amp;lt;lgl&amp;gt; &amp;lt;lgl&amp;gt;     &amp;lt;lgl&amp;gt;   &amp;lt;lgl&amp;gt; &amp;lt;lgl&amp;gt;   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 /Sys…     0 Helve… Regu… FALSE  FALSE FALSE     FALSE   FALSE TRUE    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 12 more variables: vertical &amp;lt;lgl&amp;gt;, n_glyphs &amp;lt;int&amp;gt;, n_sizes &amp;lt;int&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   n_charmaps &amp;lt;int&amp;gt;, bbox &amp;lt;list&amp;gt;, max_ascend &amp;lt;dbl&amp;gt;, max_descend &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   max_advance_width &amp;lt;dbl&amp;gt;, max_advance_height &amp;lt;dbl&amp;gt;, lineheight &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   underline_pos &amp;lt;dbl&amp;gt;, underline_size &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;glyph_info&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;glyphs&amp;#34;&lt;/span&gt;, family &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;spectral&amp;#34;&lt;/span&gt;, size &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;14&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 6 x 9&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   glyph index width height x_bearing y_bearing x_advance y_advance bbox     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt; &amp;lt;list&amp;gt;   &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 g        74     7     10         0         7         8         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 l        79     3     10         0        10         3         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 y        92     7     10         0         7         7         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 p        83     8     10         0         7         8         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 5 h        75     7     10         0        10         8         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 6 s        86     7      7         0         7         7         0 &amp;lt;dbl [4]&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;text &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;This is an example of the current text shaping engine build in to systemfonts. It does automatic text wrapping and support advanced layout features such as first-line and hanging indent. It also support setting &amp;#34;&lt;/span&gt;,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;tracking&amp;#34;&lt;/span&gt;,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34; on some word and generally mixing text with different &amp;#34;&lt;/span&gt;,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;fonts, &amp;#34;&lt;/span&gt;,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;sizes,&amp;#34;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34; and styles.&amp;#34;&lt;/span&gt;
) 
family &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;charter&amp;#39;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;charter&amp;#39;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;charter&amp;#39;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;exo 2&amp;#39;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;charter&amp;#39;&lt;/span&gt;, 
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;charter&amp;#39;&lt;/span&gt;
)
italic &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt;  &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;), &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
bold &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;), &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
style &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;plain&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;), &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;bold.italic&amp;#34;&lt;/span&gt;)
size &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;14&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;), &lt;span style=&#34;color:#666&#34;&gt;24&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;14&lt;/span&gt;)
shape &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;shape_string&lt;/span&gt;(
  text, 
  id &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, 
  family &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; family,
  italic &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; italic,
  bold &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; bold,
  size &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; size,
  tracking &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;400&lt;/span&gt;, &lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)),
  width &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4.5&lt;/span&gt;,
  indent &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.2&lt;/span&gt;,
  hjust &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;,
  vjust &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;,
  res &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;300&lt;/span&gt;
)

shape
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $shape&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 299 x 7&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    glyph index metric_id string_id x_offset y_offset x_midpoint&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;     &amp;lt;int&amp;gt;     &amp;lt;int&amp;gt;    &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1 &amp;#34;T&amp;#34;      55         1         1   -133.      47.7       4.2 &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2 &amp;#34;h&amp;#34;      75         1         1   -125.      47.7       3.96&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3 &amp;#34;i&amp;#34;      76         1         1   -117.      47.7       1.92&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4 &amp;#34;s&amp;#34;      86         1         1   -113.      47.7       2.76&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5 &amp;#34; &amp;#34;       3         1         1   -108.      47.7       1.92&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6 &amp;#34;i&amp;#34;      76         1         1   -104.      47.7       1.92&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7 &amp;#34;s&amp;#34;      86         1         1    -99.8     47.7       2.76&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8 &amp;#34; &amp;#34;       3         1         1    -94.3     47.7       1.92&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9 &amp;#34;a&amp;#34;      68         1         1    -90.5     47.7       3.48&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10 &amp;#34;n&amp;#34;      81         1         1    -83.5     47.7       3.96&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 289 more rows&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $metrics&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 11&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   string width height left_bearing right_bearing top_bearing bottom_bearing&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;chr&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;        &amp;lt;dbl&amp;gt;         &amp;lt;dbl&amp;gt;       &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 This …  1350   582.           -1            15        13.8           1.02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # … with 4 more variables: left_border &amp;lt;dbl&amp;gt;, top_border &amp;lt;dbl&amp;gt;, pen_x &amp;lt;dbl&amp;gt;,&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; #   pen_y &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(grid)

pngfile &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;path&lt;/span&gt;(knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;fig_path&lt;/span&gt;(),  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;png_image.png&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;agg_png&lt;/span&gt;(pngfile, &lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2.5&lt;/span&gt;, unit &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;in&amp;#39;&lt;/span&gt;, res &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;300&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning in agg_png(pngfile, 6, 2.5, unit = &amp;#34;in&amp;#34;, res = 300): partial argument&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; match of &amp;#39;unit&amp;#39; to &amp;#39;units&amp;#39;&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;grid.text&lt;/span&gt;(
  shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;glyph, 
  x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;unit&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;npc&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;unit&lt;/span&gt;(shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;x_offset, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pt&amp;#34;&lt;/span&gt;), 
  y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;unit&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;npc&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;unit&lt;/span&gt;(shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;y_offset, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pt&amp;#34;&lt;/span&gt;),
  hjust &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;,
  vjust &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;,
  gp &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;gpar&lt;/span&gt;(
    fontfamily &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; family[shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;string_id],
    fontface &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; style[shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;string_id],
    fontsize &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; size[shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;shape&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;string_id]
  )
)
&lt;span style=&#34;color:#00f&#34;&gt;invisible&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;dev.off&lt;/span&gt;())
knitr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;include_graphics&lt;/span&gt;(pngfile)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;img src=&#34;figs/unnamed-chunk-7-1/png_image.png&#34; width=&#34;700px&#34; style=&#34;display: block; margin: auto;&#34; /&gt;&lt;/p&gt;
&lt;p&gt;When you see this you may immediately think of the work Claus Wilke has done with 
&lt;a href=&#34;https://wilkelab.org/gridtext/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;gridtext&lt;/a&gt;. We are both committed to improving the rich text support in R, and are attacking the problem from different directions. Claus has worked on doing per-word shaping, whereas the text shaping done in systemfonts are done glyph-by-glyph. Both approaches have their merits, and we are currently at a stage where we are trying to figure out how to best solve this in R. At the time of writing there is no doubt that gridtext is more complete and stable, but doing glyph-based shaping offers possibilities that are not possible with gridtext, e.g. setting tracking and flowing text along paths. All of this is highly experimental still, and there will need to be some changes in the graphic engine to make glyph-based rendering performant.&lt;/p&gt;
&lt;h3 id=&#34;the-font-ier&#34;&gt;The font-ier&lt;/h3&gt;
&lt;p&gt;A big (for some) issue that I haven&amp;rsquo;t touched upon is support for ligatures. A &lt;strong&gt;ligature&lt;/strong&gt; is a glyph that represents multiple characters, as when &amp;ldquo;f&amp;rdquo; and &amp;ldquo;i&amp;rdquo; are substituted for the single glyph &amp;ldquo;fi&amp;rdquo;. Ligatures are often used as flourishes in Western scripts but are essential for other scripts such as Arabic. Further, ligatures power emojis which have become an integral part of modern communication.&lt;/p&gt;
&lt;p&gt;While we don&amp;rsquo;t currently have support for ligatures, there&amp;rsquo;s no doubt that we need it. The exact mechanism for this is still up in the air because ligatures must be resolved at the font rendering level. Each font file may provide their own unique ligatures and the mapping from multiple characters to a single glyph can be encoded into a number of different substitution table formats in the font file. Because of this complexity it is often handled by a dedicated library such as 
&lt;a href=&#34;https://www.freedesktop.org/wiki/Software/HarfBuzz/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;HarfBuzz&lt;/a&gt;. This could of course be wrapped by an R package but since ligatures can not be encoded with unicode (for the most part), there is no way of precomputing the substitution and pass it on to the graphic devices. Whether ligature support can retroactively be added to old graphic devices is thus still up in the air.&lt;/p&gt;
&lt;p&gt;Text rendering in R is a delicate collaboration between graphic devices, the graphic engine, and additional support packages such as systemfonts and gridtext, but thankfully the focus and will is there in all parts to push this forward.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Self-cleaning test fixtures</title>
      <link>/blog/2020/04/self-cleaning-test-fixtures/</link>
      <pubDate>Mon, 27 Apr 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/04/self-cleaning-test-fixtures/</guid>
      <description>&lt;p&gt;&lt;em&gt;Adapted from an internal presentation to the tidyverse team, which tells you something about the target reader. The primary audience for this post is R programmers and, especially, package developers. The problems identified and solved here are pretty niche! People who are mostly interested in R as a data analysis tool may not have direct use for this material. But the post offers something for anyone curious about the hazards of side effects and the various ways we can leave the world as you found it.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;test-hygiene&#34;&gt;Test hygiene&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;Take nothing but memories, leave nothing but footprints.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;― Chief Si&amp;rsquo;ahl&lt;/p&gt;
&lt;p&gt;Ideally a test should leave the world exactly as it found it. Examples of things you might do inside a test and, therefore, need to undo:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Create a file or directory&lt;/li&gt;
&lt;li&gt;Create a resource on an external system&lt;/li&gt;
&lt;li&gt;Set an R option&lt;/li&gt;
&lt;li&gt;Set an environment variable&lt;/li&gt;
&lt;li&gt;Change working directory&lt;/li&gt;
&lt;li&gt;Change an aspect of the tested package&amp;rsquo;s state&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Scrupulous attention to cleanup is more than just courtesy or being fastidious. It is also self-serving. The state of the world after test &lt;code&gt;i&lt;/code&gt; is the starting state for test &lt;code&gt;i + 1&lt;/code&gt;. Tests that change state willy-nilly eventually end up interfering with each other in ways that can be very difficult to debug. Most tests are written with an implicit assumption about the starting state, usually whatever &lt;em&gt;tabula rasa&lt;/em&gt; means for the target domain of your package. If you accumulate enough sloppy tests, you will eventually find yourself asking the programming equivalent of questions like &amp;ldquo;Who forgot to turn off the oven?&amp;rdquo; and &amp;ldquo;Who didn&amp;rsquo;t clean up after the dog?&amp;quot;.&lt;/p&gt;
&lt;p&gt;First, we lay some foundations that aren&amp;rsquo;t obviously related to tests, but just trust that we&amp;rsquo;ll get there eventually.&lt;/p&gt;
&lt;h2 id=&#34;the-onexit-pattern&#34;&gt;The &lt;code&gt;on.exit()&lt;/code&gt; pattern&lt;/h2&gt;
&lt;p&gt;If you want to clean up after yourself, how should you actually do it?&lt;/p&gt;
&lt;p&gt;The first function to know about is base R&amp;rsquo;s 
&lt;a href=&#34;https://rdrr.io/r/base/on.exit.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;on.exit()&lt;/code&gt;&lt;/a&gt;. You use it inside a function. In the function body, every time you do something that should be undone &lt;strong&gt;on exit&lt;/strong&gt;, you immediately register the cleanup code with &lt;code&gt;on.exit(expr, add = TRUE)&lt;/code&gt;&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a href=&#34;#fn:1&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;Here&amp;rsquo;s a &lt;code&gt;sloppy()&lt;/code&gt; function that prints a number with a specific number of significant digits, by adjusting an R option.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;sloppy &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(x, sig_digits) {
  &lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sig_digits)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(x)
}

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.141593&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;sloppy&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.1&lt;/span&gt;

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Notice how &lt;code&gt;pi&lt;/code&gt; prints differently before and after the call to &lt;code&gt;sloppy()&lt;/code&gt;. Calling &lt;code&gt;sloppy()&lt;/code&gt; has a side effect: it changes the &lt;code&gt;digits&lt;/code&gt; option globally, not just within its own scope of operations. This is what we want to avoid.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Don&amp;rsquo;t worry, I&amp;rsquo;m restoring global state (specifically, the &lt;code&gt;digits&lt;/code&gt; option) behind the scenes here.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Here&amp;rsquo;s how to do better with &lt;code&gt;on.exit()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;neat &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(x, sig_digits) {
  op &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sig_digits)
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(op), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(x)
}

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.141593&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;neat&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.1&lt;/span&gt;

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.141593&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The use of &lt;code&gt;on.exit()&lt;/code&gt; ensures that &lt;code&gt;neat()&lt;/code&gt; leaves &lt;code&gt;digits&lt;/code&gt; the way that it found it. &lt;code&gt;on.exit()&lt;/code&gt; also works when you exit the function abnormally, i.e. due to error. This is why it&amp;rsquo;s a better choice than any do-it-yourself solution.&lt;/p&gt;
&lt;p&gt;But I promised to talk about tests! Never fear, &lt;code&gt;on.exit()&lt;/code&gt; also works inside a test.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(testthat)

&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.718282&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;test_that&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;on.exit() works in a test&amp;#34;&lt;/span&gt;, {
  op &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(op), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
  out &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;format&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
  &lt;span style=&#34;color:#00f&#34;&gt;expect_equal&lt;/span&gt;(out, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;2.7&amp;#34;&lt;/span&gt;)
  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# printing just for the benefit of the reader&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(out) 
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2.7&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.718282&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;on.exit()&lt;/code&gt; is a very useful function and provides enough inspiration for an entire package: withr (
&lt;a href=&#34;http://withr.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;withr.r-lib.org&lt;/a&gt;), which is a Swiss army knife for managing state in very flexible ways. It&amp;rsquo;s what I usually use, in functions and tests, for situations like that above.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;For more background, the section about 
&lt;a href=&#34;https://adv-r.hadley.nz/functions.html#on-exit&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Exit handlers&lt;/a&gt; in Advanced R is a good reference. The 
&lt;a href=&#34;https://github.com/r-lib/cleancall#readme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;cleancall package&lt;/a&gt; addresses a similar problem, but in the C code of an R package. cleancall is introduced in the blog post 
&lt;a href=&#34;https://www.tidyverse.org/blog/2019/05/resource-cleanup-in-c-and-the-r-api/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Resource Cleanup in C and the R API&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;
&lt;h2 id=&#34;withrdefer&#34;&gt;&lt;code&gt;withr::defer()&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;withr::defer()&lt;/code&gt; is a more general version of &lt;code&gt;on.exit()&lt;/code&gt;. It can run cleanup for any environment, but defaults to the environment it was called in. Therefore, it works like &lt;code&gt;on.exit()&lt;/code&gt; inside a function &amp;ndash; an extremely important special case &amp;ndash; but the added flexibility means you can use it in more situations.&lt;/p&gt;
&lt;p&gt;Below I compare &lt;code&gt;on.exit()&lt;/code&gt; and &lt;code&gt;withr::defer()&lt;/code&gt; and I put the code inside &lt;code&gt;local()&lt;/code&gt;, instead of inside a function. This is meant to reinforce that cleanup can be relevant beyond function execution environments. It also gives you another tool to play with, in addition to toy functions and tests, in your own explorations of how to manage scope.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(withr)

&lt;span style=&#34;color:#00f&#34;&gt;local&lt;/span&gt;({
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;first&amp;#34;&lt;/span&gt;))
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;last&amp;#34;&lt;/span&gt;))  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# this clobbers `print(&amp;#34;first&amp;#34;)` :(&lt;/span&gt;
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;last&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;local&lt;/span&gt;({
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;first&amp;#34;&lt;/span&gt;), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;last&amp;#34;&lt;/span&gt;), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;first&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;last&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;local&lt;/span&gt;({
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;first&amp;#34;&lt;/span&gt;), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;, after &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;last&amp;#34;&lt;/span&gt;), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;, after &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;last&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;first&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;local&lt;/span&gt;({
  &lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;first&amp;#34;&lt;/span&gt;))
  &lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;last&amp;#34;&lt;/span&gt;))
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;last&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;first&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This showcases the nice ergonomics of &lt;code&gt;defer()&lt;/code&gt;: each call &lt;em&gt;adds&lt;/em&gt; to the list of deferred tasks (vs. replaces) and, by default, adds to the &lt;em&gt;front&lt;/em&gt; of the stack (vs. the back). As you&amp;rsquo;ll see below, this turns out to matter in most real world usage&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a href=&#34;#fn:2&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;h2 id=&#34;withrlocal_&#34;&gt;&lt;code&gt;withr::local_*()&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;on.exit()&lt;/code&gt; usage has a very predictable, slightly clunky pattern. In &lt;code&gt;neat()&lt;/code&gt;, it looks like:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;op &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sig_digits)
&lt;span style=&#34;color:#00f&#34;&gt;on.exit&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(op), add &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)      
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The first statement accomplishes two things at once: it sets the &lt;code&gt;digits&lt;/code&gt; option and captures its original value in &lt;code&gt;op&lt;/code&gt;. The second statement schedules the restoration of &lt;code&gt;digits&lt;/code&gt;. This order of operations is encouraged by the design of &lt;code&gt;options()&lt;/code&gt;, which returns the original values when it&amp;rsquo;s used as a setter.&lt;/p&gt;
&lt;p&gt;Here is the more general (and possibly slightly safer) approach: First, capture the current state. Then, immediately schedule the eventual restoration of this original state, so that this is arranged before any additional calls are made that could exit, e.g. throw an error. Last, make the desired state change.&lt;/p&gt;
&lt;p&gt;What if such a maneuver happens all over your package and you want to write a helper?&lt;/p&gt;
&lt;p&gt;You can&amp;rsquo;t wrap &lt;code&gt;on.exit()&lt;/code&gt; in your own helpers, because there&amp;rsquo;s no way to reach back up into the correct parent frame and schedule cleanup there. But with &lt;code&gt;defer()&lt;/code&gt;, we can! Here is such a custom helper, called &lt;code&gt;local_digits()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;local_digits &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(sig_digits, env &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;parent.frame&lt;/span&gt;()) {
  op &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sig_digits)
  &lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(op), env &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; env)
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We can use it to keep any manipulation of &lt;code&gt;digits&lt;/code&gt; local to a test (or function).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.718282&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;test_that&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;withr lets us write custom helpers for local state manipulation&amp;#34;&lt;/span&gt;, {
  &lt;span style=&#34;color:#00f&#34;&gt;local_digits&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;20&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.7182818284590450908&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.718282&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;test_that&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;we can even make multiple calls to local_digits()&amp;#34;&lt;/span&gt;, {
  &lt;span style=&#34;color:#00f&#34;&gt;local_digits&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
  &lt;span style=&#34;color:#00f&#34;&gt;local_digits&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
  &lt;span style=&#34;color:#00f&#34;&gt;local_digits&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;)
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
})
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.72&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.7183&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;exp&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.718282&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The ability to write &lt;code&gt;on.exit()&lt;/code&gt;-like functions, customized to your own needs, is very empowering. However, you may not even need to. There are certain state changes that come up over and over again, for all of us. These are pre-implemented in withr&amp;rsquo;s &lt;code&gt;local_*()&lt;/code&gt; family of functions. A few examples:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Do / undo this&lt;/th&gt;
&lt;th&gt;withr function&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Create a file&lt;/td&gt;
&lt;td&gt;&lt;code&gt;local_file()&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Set an R option&lt;/td&gt;
&lt;td&gt;&lt;code&gt;local_options()&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Set an environment variable&lt;/td&gt;
&lt;td&gt;&lt;code&gt;local_envvar()&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Change working directory&lt;/td&gt;
&lt;td&gt;&lt;code&gt;local_dir()&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&amp;ldquo;Local&amp;rdquo; here refers to the fact that the state change persists only for the lifetime of an associated environment, usually the execution environment of a function or test.&lt;/p&gt;
&lt;p&gt;We can use &lt;code&gt;withr::local_options()&lt;/code&gt; to write yet another version of &lt;code&gt;neat()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;neater &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(x, sig_digits) {
  &lt;span style=&#34;color:#00f&#34;&gt;local_options&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sig_digits))
  &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(x)
}

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.141593&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;neater&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.14&lt;/span&gt;

&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;pi&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.141593&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Each &lt;code&gt;local_*()&lt;/code&gt; function has a companion &lt;code&gt;with_()&lt;/code&gt; function, which is a nod to 
&lt;a href=&#34;https://rdrr.io/r/base/with.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;with()&lt;/code&gt;&lt;/a&gt;. We won&amp;rsquo;t use the &lt;code&gt;with_*()&lt;/code&gt; functions here, but you can learn more about them at 
&lt;a href=&#34;http://withr.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;withr.r-lib.org&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;test-fixtures&#34;&gt;Test fixtures&lt;/h2&gt;
&lt;p&gt;Testing is often demonstrated with cute little tests and functions where all the inputs and expected results can be inlined. But in real packages, things aren&amp;rsquo;t always so simple. The main functions in your package probably aren&amp;rsquo;t &amp;ldquo;1 number in, 1 number out&amp;rdquo;. They might require more exotic objects or very specific circumstances. Changing state might be the entire purpose of a function! Now what?&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Obligatory caveat: If you find it hard to write tests, this may be the universe telling you that your package has some design problems. Maybe you&amp;rsquo;ve somehow ended up with a small number of monster functions, with oodles of arguments and complex conditional logic, that can do everything from scramble eggs to change a lightbulb. The best move in this case may be to break things up into smaller and simpler functions. And those will be easier to test. End caveat.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Tricky test situations can&amp;rsquo;t always be eliminated by better package design. Let&amp;rsquo;s assume you&amp;rsquo;ve got a reasonable design and you&amp;rsquo;re still facing some test dilemmas. Unless you find a way to make writing tests as pleasant as possible, you won&amp;rsquo;t write nearly enough of them.&lt;/p&gt;
&lt;p&gt;One technique I&amp;rsquo;ve found useful is what I&amp;rsquo;ll call a &lt;strong&gt;self-cleaning test fixture&lt;/strong&gt;.&lt;/p&gt;
&lt;h3 id=&#34;usethis-and-create_local_package&#34;&gt;usethis and &lt;code&gt;create_local_package()&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;The usethis package (
&lt;a href=&#34;https://usethis.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;usethis.r-lib.org&lt;/a&gt;) provides functions for looking after the files and folders in an R project, especially an R package. These function names should give you a vague sense of what usethis does: &lt;code&gt;create_package()&lt;/code&gt;, &lt;code&gt;use_vignette()&lt;/code&gt;, &lt;code&gt;use_testthat()&lt;/code&gt;, &lt;code&gt;use_github()&lt;/code&gt;. Many of these functions only make sense in the context of an R package. That means in order to test them, we have to be working inside an R package. And they can&amp;rsquo;t all target some persistent Frankenpackage.&lt;/p&gt;
&lt;p&gt;We need a way to quickly spin up a minimal package, in the session temp directory. Test some functions against it. Then destroy it. Without a lot of fuss. We need a &lt;strong&gt;local package&lt;/strong&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;create_local_package &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(dir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;file_temp&lt;/span&gt;(), env &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;parent.frame&lt;/span&gt;()) {
  old_project &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;proj_get_&lt;/span&gt;()            &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# --- Record starting state --- &lt;/span&gt;
  
  withr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;({                        &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# --- Defer The Undoing --- &lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;proj_set&lt;/span&gt;(old_project, force &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;) &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# restore active usethis project (-C)&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;setwd&lt;/span&gt;(old_project)                  &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# restore working directory      (-B)&lt;/span&gt;
    fs&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;dir_delete&lt;/span&gt;(dir)                 &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# delete the temporary package   (-A)&lt;/span&gt;
  }, envir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; env)
                                        &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# --- Do The Doing ---      &lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;create_package&lt;/span&gt;(dir, open &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)     &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# create new folder and package  (A)&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;setwd&lt;/span&gt;(dir)                            &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# change working directory       (B)&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;proj_set&lt;/span&gt;(dir)                         &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# switch to new usethis project  (C)&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;invisible&lt;/span&gt;(dir)
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;That&amp;rsquo;s a simplified version of the test helper&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a href=&#34;#fn:3&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt; we use in over 170 tests in usethis. Here&amp;rsquo;s an example of how &lt;code&gt;create_local_package()&lt;/code&gt; is used in a test:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;test_that&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;use_roxygen_md() adds DESCRIPTION fields&amp;#34;&lt;/span&gt;, {
  &lt;span style=&#34;color:#00f&#34;&gt;skip_if_not_installed&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;roxygen2&amp;#34;&lt;/span&gt;)
  
  pkg &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;create_local_package&lt;/span&gt;() &lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# &amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;------------------------ HERE IT IS!!!!!&lt;/span&gt;
  
  &lt;span style=&#34;color:#00f&#34;&gt;use_roxygen_md&lt;/span&gt;()
  
  &lt;span style=&#34;color:#00f&#34;&gt;expect_identical&lt;/span&gt;(
    desc&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;desc_get&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Roxygen&amp;#34;&lt;/span&gt;, pkg),
    &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(Roxygen &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;list(markdown = TRUE)&amp;#34;&lt;/span&gt;)
  )
  &lt;span style=&#34;color:#00f&#34;&gt;expect_true&lt;/span&gt;(desc&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;desc_has_fields&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;RoxygenNote&amp;#34;&lt;/span&gt;, pkg))
  &lt;span style=&#34;color:#00f&#34;&gt;expect_true&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;uses_roxygen_md&lt;/span&gt;())
})
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This test checks that &lt;code&gt;usethis::use_roxygen_md()&lt;/code&gt; does the setup necessary to use roxygen2 in a package and, specifically, to write documentation with markdown syntax. All 3 expectations consult the DESCRIPTION file, directly or indirectly. So it&amp;rsquo;s very convenient that &lt;code&gt;create_local_package()&lt;/code&gt; creates a minimal package, with a valid DESCRIPTION file, for us to test against. And when the test is done &amp;ndash; poof! &amp;ndash; the package is gone.&lt;/p&gt;
&lt;p&gt;The setup and teardown done by &lt;code&gt;create_local_package()&lt;/code&gt; would be aggravating and repetitive to inline in each individual test. The tests would be dominated by this code, making them less readable. If we need to tweak something, we&amp;rsquo;d have to do it in hundreds of places. This sort of friction has a real chilling effect on one&amp;rsquo;s enthusiasm for writing and maintaining tests.&lt;/p&gt;
&lt;p&gt;A few more observations about the self-cleaning test fixture pattern:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Every action has an equal and opposite reaction. Each individual &amp;ldquo;doing&amp;rdquo;
action (A) has a matching, deferred &amp;ldquo;undoing&amp;rdquo; reaction (-A).&lt;/li&gt;
&lt;li&gt;We work in this order (usually and preferably):
&lt;ul&gt;
&lt;li&gt;Record existing state.&lt;/li&gt;
&lt;li&gt;Describe the eventual cleanup.&lt;/li&gt;
&lt;li&gt;Make the desired state change.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;The undoing usually unfolds in the opposite order from the doing (&amp;ldquo;last in,
first out&amp;rdquo;). This is almost always OK and it is often absolutely necessary.
In &lt;code&gt;create_local_package()&lt;/code&gt;:
&lt;ul&gt;
&lt;li&gt;Doing: We must create directory &lt;code&gt;dir&lt;/code&gt; (A) before we can make it the
working directory (B). (A) must come before (B).&lt;/li&gt;
&lt;li&gt;Undoing: We must restore the original working directory (-B) before
we can delete &lt;code&gt;dir&lt;/code&gt; (-A). (-B) must come before (-A). We can&amp;rsquo;t delete
&lt;code&gt;dir&lt;/code&gt; while it&amp;rsquo;s still the working directory!&lt;/li&gt;
&lt;li&gt;Think of it like a stack of plates: the last plate onto the stack has to
be the first one off.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Test fixture&lt;/strong&gt; is a pre-existing term in the software engineering world (and beyond):&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A test fixture is something used to consistently test some item, device, or piece of software.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&amp;ndash; 
&lt;a href=&#34;https://en.wikipedia.org/wiki/Test_fixture&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;When I first heard &amp;ldquo;test fixture&amp;rdquo; (from Gábor Csárdi, I think), a light bulb clicked on in my head. This was something I &lt;em&gt;knew&lt;/em&gt; I needed and had even implemented in various half-baked ways. But I hadn&amp;rsquo;t identified it as A Real Thing, with specific goals and design principles. It&amp;rsquo;s a great example of 
&lt;a href=&#34;https://blogs.scientificamerican.com/observations/unknown-unknowns-the-problem-of-hypocognition/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;hypocognition&lt;/a&gt;. Learning the term &amp;ldquo;test fixture&amp;rdquo; gave me a place to hang this knowledge and allowed me to more quickly identify situations where a test fixture was needed.&lt;/p&gt;
&lt;h3 id=&#34;googlesheets4-and-local_ss&#34;&gt;googlesheets4 and &lt;code&gt;local_ss()&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;The googlesheets4 package (
&lt;a href=&#34;https://googlesheets4.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;googlesheets4.tidyverse.org&lt;/a&gt;) provides an R interface to the Google Sheets API. A typical test needs access to a Google Sheet, constructed to have very specific properties and the test may even need to modify the Sheet&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a href=&#34;#fn:4&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;I need a way to quickly create a Sheet, possibly with very specific initial worksheets, cell data, locale, time zone, etc. Test some functions against it. Then trash it. I need a &lt;em&gt;local spreadsheet&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Here&amp;rsquo;s a simplified version of the helper &lt;code&gt;local_ss()&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;local_ss &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(name, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt;, env &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;parent.frame&lt;/span&gt;()) {
  existing &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;gs4_find&lt;/span&gt;(name)
  &lt;span style=&#34;color:#00f&#34;&gt;if &lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;nrow&lt;/span&gt;(existing) &lt;span style=&#34;color:#666&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;) {
    &lt;span style=&#34;color:#00f&#34;&gt;stop_glue&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;A spreadsheet named {sq(name)} already exists.&amp;#34;&lt;/span&gt;)
  }

  withr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;({
    trash_me &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;gs4_find&lt;/span&gt;(name)
    googledrive&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;drive_trash&lt;/span&gt;(trash_me)
  }, envir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; env)
  
  &lt;span style=&#34;color:#00f&#34;&gt;gs4_create&lt;/span&gt;(name, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt;)
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Even though the Sheets API is very file-ID-oriented, I go out of my way to work here via Sheet name. I bring this up to illustrate another point: you can also use a helper like this to rationalize your development workflow.&lt;/p&gt;
&lt;p&gt;At first, it feels like &lt;code&gt;local_ss()&lt;/code&gt; should create a new Sheet, store its ID, and then schedule it for deletion. But reality is more messy. As I develop a function and its tests, my experimentation can leave behind several instances of a test Sheet (yes, on Drive, you can have several files with the same name!). This leads to a very cluttered and confusing situation in the test account, requiring a periodic &amp;ldquo;search and destroy&amp;rdquo; mission for zombie test Sheets. Now my helper immediately alerts me to this problem and applies constant pressure to keep things neat and tidy.&lt;/p&gt;
&lt;p&gt;If you keep stubbing your toe in a particular way as you work on your package, zoom out and consider if you can design the problem away by adjusting your workflow. The helper that creates a self-cleaning test fixture is great place to apply this sort of leverage.&lt;/p&gt;
&lt;h2 id=&#34;defer-on-the-global-environment&#34;&gt;&lt;code&gt;defer()&lt;/code&gt; on the global environment&lt;/h2&gt;
&lt;p&gt;I conclude with one more story about workflow. We&amp;rsquo;ve talked about two main functions for registering deferred events: base R&amp;rsquo;s &lt;code&gt;on.exit()&lt;/code&gt; and &lt;code&gt;withr::defer()&lt;/code&gt;. Part of what &lt;code&gt;withr::defer()&lt;/code&gt; offers over &lt;code&gt;on.exit()&lt;/code&gt; is the ability to defer events on &lt;em&gt;any&lt;/em&gt; environment. But there was still a big exception: the global environment.&lt;/p&gt;
&lt;p&gt;Until quite recently, here&amp;rsquo;s what happened if you called &lt;code&gt;defer()&lt;/code&gt; in an interactive R session&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a href=&#34;#fn:5&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;withr&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;hi&amp;#34;&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error in withr::defer(print(&amp;#34;hi&amp;#34;)):&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   attempt to defer event on global environment&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;packageVersion&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;withr&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#39;2.1.2&amp;#39;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Frankly, this makes a lot of sense. Deferred events are triggered when an environment goes out of scope. &lt;code&gt;on.exit()&lt;/code&gt; and &lt;code&gt;defer()&lt;/code&gt; are meant to be used in an ephemeral environment, like a function execution environment. Deferring events on the global environment is pretty weird.&lt;/p&gt;
&lt;p&gt;But what about your interactive development of functions and tests? Every time you hit a call to &lt;code&gt;defer()&lt;/code&gt; or &lt;code&gt;local_*()&lt;/code&gt;, that code fails to execute. You&amp;rsquo;re forced to develop your logic at arm&amp;rsquo;s length or implement the intent of the &lt;code&gt;local_*()&lt;/code&gt; calls manually. If you&amp;rsquo;re doing quite a bit of work via &lt;code&gt;local_*()&lt;/code&gt; or &lt;code&gt;on.exit()&lt;/code&gt;, this presents a problem. Basically, it&amp;rsquo;s harder to develop with functions that work one way inside a function, but another way in the global environment (or, worse, don&amp;rsquo;t work at all). &lt;code&gt;substitute()&lt;/code&gt; is another example of this.&lt;/p&gt;
&lt;p&gt;As of withr v2.2.0, you can &lt;code&gt;defer()&lt;/code&gt; events on the global environment, which means that &lt;code&gt;local_*()&lt;/code&gt; functions work too. This is still a pretty weird thing to do, which is why you get a message about how to trigger execution.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(withr)

&lt;span style=&#34;color:#00f&#34;&gt;packageVersion&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;withr&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#39;2.2.0&amp;#39;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;defer&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;hi&amp;#34;&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Setting deferred event(s) on global environment.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   * Execute (and clear) with `deferred_run()`.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   * Clear (without executing) with `deferred_clear()`.&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;deferred_run&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;hi&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Since the global environment isn&amp;rsquo;t perishable, like a test environment is, you have to call &lt;code&gt;deferred_run()&lt;/code&gt; explicitly to execute the deferred events. You can also clear them, without running, with &lt;code&gt;deferred_clear()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;This new capability is especially handy with self-cleaning test fixtures, like &lt;code&gt;create_local_package()&lt;/code&gt; and &lt;code&gt;local_ss()&lt;/code&gt; shown above. Sometimes you have to change global state while developing tests, e.g. change working directory or create test Sheets. But now there&amp;rsquo;s a way to run the associated cleanup on demand.&lt;/p&gt;
&lt;h2 id=&#34;recap&#34;&gt;Recap&lt;/h2&gt;
&lt;p&gt;We&amp;rsquo;ve demonstrated that it&amp;rsquo;s a problem to change state in a function or test. Obviously there are exceptions if, for example, that is the whole point of the function.&lt;/p&gt;
&lt;p&gt;The most common and recommended solution to this is to use &lt;code&gt;on.exit()&lt;/code&gt; to organize the necessary cleanup, i.e. restore the original state. However, &lt;code&gt;on.exit()&lt;/code&gt; has some inherent limitations.&lt;/p&gt;
&lt;p&gt;If this sort of setup/teardown happens frequently in the functions and tests in a package, it makes sense to write a custom helper. This function should follow the conventions of the &lt;code&gt;local_*()&lt;/code&gt; functions in withr and will presumably be built around &lt;code&gt;withr::defer()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;There is some cost to using a custom &lt;code&gt;local_*()&lt;/code&gt; helper, as it is one more thing to maintain and that all contributors must understand. Consider whether the pros outweigh the cons when adding another layer of abstraction.&lt;/p&gt;
&lt;section class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;It&amp;rsquo;s too bad &lt;code&gt;add = TRUE&lt;/code&gt; isn&amp;rsquo;t the default, because you almost always want this. Without it, each call to &lt;code&gt;on.exit()&lt;/code&gt; clobbers the effect of previous calls. &lt;a href=&#34;#fnref:1&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;Note: the &lt;code&gt;after&lt;/code&gt; argument of &lt;code&gt;on.exit()&lt;/code&gt; first appeared in R 3.5.0. &lt;a href=&#34;#fnref:2&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;&lt;code&gt;create_local_package()&lt;/code&gt; is a test helper. The testthat package allows such things to be defined in &lt;code&gt;tests/testthat/helper.R&lt;/code&gt; and then makes them available within package tests. They are also loaded by &lt;code&gt;devtools::load_all()&lt;/code&gt;. &lt;code&gt;tests/testthat/helper.R&lt;/code&gt; is also a great place to define custom expectations. &lt;a href=&#34;#fnref:3&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;You might ask about mocking here, but I usually don&amp;rsquo;t embrace that heavily and, in any case, that is a topic for another day. &lt;a href=&#34;#fnref:4&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;For all practical purposes, you get the same result with &lt;code&gt;on.exit()&lt;/code&gt;. It&amp;rsquo;s just a silent no-op. &lt;a href=&#34;#fnref:5&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</description>
    </item>
    
    <item>
      <title>Announcing tidymodels.org</title>
      <link>/blog/2020/04/tidymodels-org/</link>
      <pubDate>Tue, 21 Apr 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/04/tidymodels-org/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re thrilled to announce the launch of 
&lt;a href=&#34;https://www.tidymodels.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;tidymodels.org&lt;/code&gt;&lt;/a&gt;! The site is a central location for learning and using the tidymodels packages. It has a new take on a familiar look:&lt;/p&gt;
&lt;img src=&#34;tidymodels-front-page.png&#34; alt=&#34;tidymodels.org landing page screenshot&#34;&gt;
&lt;p&gt;This site has a different organization than its tidyverse sibling.&lt;/p&gt;
&lt;p&gt;The 
&lt;a href=&#34;https://www.tidymodels.org/start/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Get Started&lt;/strong&gt;&lt;/a&gt; page has a series of five articles that are aimed at readers who have little to no experience with the tidymodels packages. Each article focuses on a specific task (e.g. building a model), and the last article walks through a case study that brings all of the other elements together.&lt;/p&gt;
&lt;p&gt;We have consolidated many of the vignettes and articles that showcase code examples on the 
&lt;a href=&#34;https://www.tidymodels.org/learn/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Learn&lt;/strong&gt;&lt;/a&gt; page. Previously, these were scattered across package documentation websites. We received feedback that keeping these articles in the individual sites made it too difficult to find the right material, especially when an article used multiple packages. Articles describing specific technical details of an individual package will remain in that specific pkgdown site.&lt;/p&gt;
&lt;p&gt;The content on the 
&lt;a href=&#34;https://www.tidymodels.org/learn/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Learn&lt;/strong&gt;&lt;/a&gt; page is organized into groups:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Perform statistical analysis&lt;/li&gt;
&lt;li&gt;Create robust models&lt;/li&gt;
&lt;li&gt;Tune, compare, and work with your models&lt;/li&gt;
&lt;li&gt;Develop custom modeling tools&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We&amp;rsquo;ll continue to add new articles to the &lt;strong&gt;Learn&lt;/strong&gt; section as we develop new functions in tidymodels, or for specific topics that need more documentation (based on GitHub issues, as well as posts to Stack Overflow or RStudio Community).&lt;/p&gt;
&lt;p&gt;The 
&lt;a href=&#34;https://www.tidymodels.org/help/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Help&lt;/strong&gt;&lt;/a&gt; section is an excellent place to consult when you are stuck on a problem or when you want to know more. There are additional links to books using the tidymodels packages as well as tools to search across 
&lt;a href=&#34;https://www.tidymodels.org/find/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;all tidymodels functions&lt;/a&gt;, 
&lt;a href=&#34;https://www.tidymodels.org/find/parsnip/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;parsnip&lt;/code&gt; model functions&lt;/a&gt;, and 
&lt;a href=&#34;https://www.tidymodels.org/find/recipes/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;recipe&lt;/code&gt; steps&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Blog posts related to tidymodels will still be posted here at the 
&lt;a href=&#34;https://www.tidyverse.org/blog/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidyverse blog&lt;/a&gt;. Both &lt;code&gt;tidyverse.org&lt;/code&gt; and &lt;code&gt;tidymodels.org&lt;/code&gt; are Hugo websites built using R with the 
&lt;a href=&#34;https://bookdown.org/yihui/blogdown/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blogdown&lt;/a&gt; package. We learned a lot while launching this new website and plan to share about this process in detail soon.&lt;/p&gt;
&lt;p&gt;The launch of 
&lt;a href=&#34;https://www.tidymodels.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;tidymodels.org&lt;/code&gt;&lt;/a&gt; would not have been possible without Alison Hill, whose incredible expertise and tenacity made it happen. Additionally, Alison and Julia Silge did an amazing job making the existing content more readable and better organized. 
&lt;a href=&#34;https://desiree.rbind.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Desirée De Leon&lt;/a&gt; worked her magic to make the site look even better. We also used Allison Horst&amp;rsquo;s 
&lt;a href=&#34;https://github.com/allisonhorst/stats-illustrations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;superb illustrations&lt;/a&gt; just to show off. The tidyverse group, Joshua Spiewak, Greg Swinehart, and Sarah Lin (RStudio&amp;rsquo;s Information Architect &amp;amp; Digital Librarian) also contributed valued support and help.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;parsnip-flagger.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>usethis 1.6.0</title>
      <link>/blog/2020/04/usethis-1-6-0/</link>
      <pubDate>Sat, 11 Apr 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/04/usethis-1-6-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re happy to announce that 
&lt;a href=&#34;https://usethis.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;usethis&lt;/a&gt; 1.6.0 is now available on CRAN. usethis is a package that facilitates interactive workflows for R project creation and development. It&amp;rsquo;s mostly focussed on easing day-to-day package development, but many of its functions are also useful for non-package projects. Install usethis with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;usethis&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This blog post discusses three sets of improvements to usethis:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;New helpers for using GitHub Actions for continuous integration. If you&amp;rsquo;re an
R package developer who uses GitHub, we recommend moving away from Travis and
AppVeyor in favour of GitHub Actions.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;We&amp;rsquo;ve made a number of small tweaks to &lt;code&gt;create_package()&lt;/code&gt; in order to reduce
inessential friction in the initial startup phase of a package.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;We&amp;rsquo;ve continued to polish our tools for contributing to other people&amp;rsquo;s
packages through GitHub pull requests.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This release also includes a handful of new functions (my favourite is 
&lt;a href=&#34;https://usethis.r-lib.org/reference/rename_files.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;rename_file()&lt;/code&gt;&lt;/a&gt;), many bug fixes, and lots of small improvements. We&amp;rsquo;re slowly grinding down the rough edges, so usethis just works in more cases, and when it fails, it&amp;rsquo;s more likely to give you error messages that help you quickly figure out the problem. As usual, you can find detailed notes about all these changes in the 
&lt;a href=&#34;https://usethis.r-lib.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;change log&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;github-actions&#34;&gt;GitHub Actions&lt;/h2&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/features/actions&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;GitHub Actions&lt;/a&gt; is a continuous integration service that allows you to automatically run code whenever you push to GitHub. If you&amp;rsquo;re developing a package this allows you to automate tasks like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Run &lt;code&gt;R CMD check&lt;/code&gt; on multiple platforms (Linux, Windows, and Mac) and
different versions of R (e.g., devel, release, oldrel).&lt;/li&gt;
&lt;li&gt;Record the code coverage of your unit tests.&lt;/li&gt;
&lt;li&gt;Re-build your 
&lt;a href=&#34;https://pkgdown.r-lib.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pkgdown&lt;/a&gt; website.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Outside of a package, you can also use GitHub Actions to rebuild 
&lt;a href=&#34;https://github.com/r-lib/actions/blob/master/examples/blogdown.yaml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blogdown&lt;/a&gt; and 
&lt;a href=&#34;https://github.com/r-lib/actions/blob/master/examples/bookdown.yaml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bookdown&lt;/a&gt; sites, or regularly re-knit &lt;code&gt;.Rmd&lt;/code&gt; files.&lt;/p&gt;
&lt;p&gt;Each GitHub Actions &lt;em&gt;workflow&lt;/em&gt; is described in a yaml file stored in the &lt;code&gt;.github/workflows&lt;/code&gt; directory of your repository. usethis v1.6.0 introduces new helper functions to create these files:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/r-lib/actions/blob/master/examples/check-standard.yaml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;use_github_action(&amp;quot;check-standard&amp;quot;)&lt;/code&gt;&lt;/a&gt; runs &lt;code&gt;R CMD check&lt;/code&gt; on the latest
R release on Linux, Windows, and macOS, and R-devel on macOS. This ensures
that your package works on all major operating systems, and alerts you to
any potential problems in the next version of R.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/r-lib/actions/blob/master/examples/test-coverage.yaml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;use_github_action(&amp;quot;test-coverage&amp;quot;)&lt;/code&gt;&lt;/a&gt;
uses 
&lt;a href=&#34;http://covr.r-lib.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;covr&lt;/a&gt; to measure the test coverage of your package
and publishes it to &lt;a href=&#34;http://codecov.io/&#34;&gt;http://codecov.io/&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/r-lib/actions/blob/master/examples/pkgdown.yaml&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;use_github_action(&amp;quot;pkgdown&amp;quot;)&lt;/code&gt;&lt;/a&gt;
uses 
&lt;a href=&#34;https://pkgdown.r-lib.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pkgdown&lt;/a&gt; to build your package website and
publishes it to the &lt;code&gt;gh-pages&lt;/code&gt; branch.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You can see examples of other workflows at &lt;a href=&#34;https://github.com/r-lib/actions/tree/master/examples&#34;&gt;https://github.com/r-lib/actions/tree/master/examples&lt;/a&gt;. The files in this directory are templates that you can easily copy into your package by running &lt;code&gt;use_github_action(&amp;quot;name&amp;quot;)&lt;/code&gt;. We encourage you to look at the &lt;code&gt;.yaml&lt;/code&gt; files that these functions create and customise them to meet your needs.&lt;/p&gt;
&lt;p&gt;Compared to Travis-CI, GitHub Actions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Provides more resources, i.e. 6 hour build times and 20 concurrent builds
instead of 50 minute build times and ~5 concurrent builds.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Has more complete support for building on Windows and macOS and a more
natural way of using Docker containers.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Doesn&amp;rsquo;t require any extra authentication because all code is run on GitHub.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Is considerably easier to customise to provide workflows that we haven&amp;rsquo;t
made easy.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You can learn more by reading 
&lt;a href=&#34;https://ropenscilabs.github.io/actions_sandbox/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Github Actions with R&lt;/em&gt;&lt;/a&gt;, by Chris Brown, Murray Cadzow, Paula A Martinez, Rhydwyn McGuire, David Neuzerling, David Wilkinson, and Saras Windecker, or watching Jim Hester&amp;rsquo;s 
&lt;a href=&#34;https://resources.rstudio.com/rstudio-conf-2020/azure-pipelines-and-github-actions-jim-hester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rstudio::conf(2020) presentation&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;creating-packages&#34;&gt;Creating packages&lt;/h2&gt;
&lt;p&gt;Based on our experience teaching package development, we&amp;rsquo;ve made a few changes to how &lt;code&gt;create_package()&lt;/code&gt; sets up a new package. The biggest difference is that we now assume that you&amp;rsquo;re going to use roxygen2&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a href=&#34;#fn:1&#34; class=&#34;footnote-ref&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt; (although you can opt out with &lt;code&gt;roxygen = FALSE&lt;/code&gt;), reducing some inconsistencies in development behaviours before and after your first run of &lt;code&gt;devtools::document()&lt;/code&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;We automatically add a &lt;code&gt;RoxygenNote&lt;/code&gt; field to the &lt;code&gt;DESCRIPTION&lt;/code&gt;. This is a
subtle change that ensures &lt;code&gt;devtools::check()&lt;/code&gt; re-documents your package even
when you haven&amp;rsquo;t yet run &lt;code&gt;devtools::document()&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The default &lt;code&gt;NAMESPACE&lt;/code&gt; no longer exports anything. This means that you
must always use &lt;code&gt;@export&lt;/code&gt; if you want functions to be available to the
end-user.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We made a couple of small changes to ease other minor frustrations:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;use_rstudio()&lt;/code&gt; now sets the &lt;code&gt;LineEndingConversion&lt;/code&gt; to &lt;code&gt;Posix&lt;/code&gt; so that
packages edited with RStudio always use LF line endings, regardless of
platform. This reduces spurious changes when multiple people collaborate
on the same package.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The &lt;code&gt;usethis.description&lt;/code&gt; option now lets you set &lt;code&gt;Authors@R = person()&lt;/code&gt;
directly. That is, you can make an actual call to &lt;code&gt;person()&lt;/code&gt; as opposed
to writing a &lt;em&gt;string&lt;/em&gt; that, when evaluated as code, returns a &lt;code&gt;person()&lt;/code&gt;.
This makes it less aggravating to detect and correct any mistakes. For
example, in my &lt;code&gt;.Rprofile&lt;/code&gt; I used to have:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(usethis.description &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(
  `Authors@R` &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;person(&amp;#34;Hadley&amp;#34;, &amp;#34;Wickham&amp;#34;, , &amp;#34;hadley@rstudio.com&amp;#34;, role = c(&amp;#34;aut&amp;#34;, &amp;#34;cre&amp;#34;))&amp;#39;&lt;/span&gt;
))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And now I have:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(usethis.description &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(
  `Authors@R` &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; utils&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;person&lt;/span&gt;(
    &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Hadley&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;Wickham&amp;#34;&lt;/span&gt;, , &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;hadley@rstudio.com&amp;#34;&lt;/span&gt;,
    role &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;aut&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;cre&amp;#34;&lt;/span&gt;)
  )
))
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;As you can see from the syntax highlighting, it&amp;rsquo;s now much easier to see if
you&amp;rsquo;ve got all the quotes and commas in the right place. When you do this in
&lt;code&gt;.Rprofile&lt;/code&gt;, note that you &lt;strong&gt;must&lt;/strong&gt; call it as &lt;code&gt;utils::person()&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;contributing-to-packages-via-github-pull-requests&#34;&gt;Contributing to packages via GitHub pull requests&lt;/h2&gt;
&lt;p&gt;Based on our experiences at 
&lt;a href=&#34;https://www.tidyverse.org/blog/2019/11/tidyverse-dev-day-2020/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidyverse developer day&lt;/a&gt;, we&amp;rsquo;ve tweaked the behaviour of usethis to ensure that new files have the same line ending as the rest of the project. We&amp;rsquo;ve also continued to polish our family of pull request helpers to work in more real-world situations. And, thanks to 
&lt;a href=&#34;http://www2.stat.duke.edu/~mc301/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Mine Cetinkaya-Rundel&lt;/a&gt;, we now have an article that 
&lt;a href=&#34;https://usethis.r-lib.org/articles/articles/pr-functions.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;explains the overall workflow&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;thank-you&#34;&gt;Thank you!&lt;/h2&gt;
&lt;p&gt;A big thanks to all 103 contributors who helped make this release happen via their contributions on GitHub. 
&lt;a href=&#34;https://github.com/aaronpeikert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@aaronpeikert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/adelmofilho&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@adelmofilho&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Ahobert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Ahobert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/alandipert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@alandipert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/andrie&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@andrie&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/angela-li&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@angela-li&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/antoine-sachet&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@antoine-sachet&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/aosmith16&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@aosmith16&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/atusy&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@atusy&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/avalcarcel9&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@avalcarcel9&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/barryrowlingson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@barryrowlingson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/boshek&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@boshek&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cderv&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cderv&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Cervangirard&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Cervangirard&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/chsafouane&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@chsafouane&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/coatless&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@coatless&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ColinFay&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ColinFay&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/CorradoLanera&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@CorradoLanera&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/csgillespie&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@csgillespie&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cstepper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cstepper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/davechilders&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@davechilders&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/davidchall&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@davidchall&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DavisVaughan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DavisVaughan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dchiu911&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dchiu911&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dragosmg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dragosmg&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/edgararuiz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@edgararuiz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/erindb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@erindb&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/espinielli&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@espinielli&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fabian-s&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fabian-s&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fermumen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fermumen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/florianm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@florianm&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fmichonneau&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fmichonneau&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/friep&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@friep&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gaborcsardi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gaborcsardi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/GegznaV&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@GegznaV&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/GregorDeCillia&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@GregorDeCillia&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/igordot&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@igordot&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ijlyttle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ijlyttle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/IndrajeetPatil&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@IndrajeetPatil&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/irenetlv&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@irenetlv&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/isteves&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@isteves&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jdblischak&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jdblischak&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jennybc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jennybc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimmyg3g&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimmyg3g&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jmgirard&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jmgirard&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/JohnCoene&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@JohnCoene&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jplecavalier&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jplecavalier&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jpritikin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jpritikin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jrosen48&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jrosen48&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jules32&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jules32&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jzadra&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jzadra&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/karawoo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@karawoo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kevinushey&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kevinushey&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kiernann&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kiernann&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/krlmlr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@krlmlr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lettucehead&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lettucehead&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/leungi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@leungi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lionel-&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lionel-&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/llrs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@llrs&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lorenzwalthert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lorenzwalthert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maelle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maelle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MagdyLaban&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MagdyLaban&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/malcolmbarrett&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@malcolmbarrett&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Maschette&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Maschette&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/matthijsvanderloos&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@matthijsvanderloos&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maurolepore&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maurolepore&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maxheld83&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maxheld83&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MichaelChirico&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MichaelChirico&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikmart&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mikmart&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MilesMcBain&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MilesMcBain&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mine-cetinkaya-rundel&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mine-cetinkaya-rundel&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mitchelloharawild&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mitchelloharawild&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/muschellij2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@muschellij2&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nandriychuk&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nandriychuk&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/njtierney&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@njtierney&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/noamross&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@noamross&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/okhoma&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@okhoma&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/overdodactyl&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@overdodactyl&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pachamaltese&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pachamaltese&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pat-s&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pat-s&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/perezp44&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@perezp44&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/petrbouchal&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@petrbouchal&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/phargarten2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@phargarten2&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pieterjanvc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pieterjanvc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ramiromagno&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ramiromagno&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/riccardoporreca&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@riccardoporreca&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rich-iannone&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rich-iannone&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Robinlovelace&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Robinlovelace&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/romainfrancois&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@romainfrancois&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rossellhayes&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rossellhayes&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rundel&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rundel&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ryapric&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ryapric&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/slyrus&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@slyrus&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/smingerson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@smingerson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/smwindecker&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@smwindecker&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/strboul&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@strboul&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/timtrice&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@timtrice&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/TylerGrantSmith&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@TylerGrantSmith&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/VincentGuyader&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@VincentGuyader&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/wch&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wch&lt;/a&gt;.&lt;/p&gt;
&lt;section class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34; role=&#34;doc-endnote&#34;&gt;
&lt;p&gt;This seems like a reasonable assumption given that ~80% of new CRAN packages use roxygen2. &lt;a href=&#34;#fnref:1&#34; class=&#34;footnote-backref&#34; role=&#34;doc-backlink&#34;&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/section&gt;
</description>
    </item>
    
    <item>
      <title>tibble 3.0.0</title>
      <link>/blog/2020/04/tibble-3-0-0/</link>
      <pubDate>Thu, 09 Apr 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/04/tibble-3-0-0/</guid>
      <description>&lt;p&gt;Version 3.0.0 of the tibble package is on CRAN now. Tibbles are a modern reimagining of the data frame, keeping what time has shown to be effective, and throwing out what is not, with nicer default output too! Grab the latest version with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;tibble&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Tibble now fully embraces vctrs, using it under the hood for its subsetting and subset assignment (&amp;ldquo;subassignment&amp;rdquo;) operations.
Accessing and updating rows and columns is now based on a rock-solid framework and works consistently for all types of columns, including list, data frame, and
matrix columns.
We believe that the changes will ultimately lead to better and safer code.&lt;/p&gt;
&lt;p&gt;This major release required quite some preparation, including a 
&lt;a href=&#34;https://tibble.tidyverse.org/articles/invariants.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;new vignette&lt;/a&gt; that describes the behavior of subsetting and subset assignment operations and the reasoning behind it.
For a complete overview please see the 
&lt;a href=&#34;https://tibble.tidyverse.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;release notes&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;In a nutshell: if an object is a vector, it can be part of a tibble.
My new 
&lt;a href=&#34;https://github.com/krlmlr/awesome-vctrs#readme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Awesome vectors&lt;/a&gt; list aims at giving an overview of available implementations of vector types in R.
If you&amp;rsquo;re using a specialized class, or even implemented one, please file an issue in that repository or contribute an example.
For problems with tibble, use the 
&lt;a href=&#34;https://github.com/tidyverse/tibble/issues&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;issue tracker&lt;/a&gt; to report bugs or suggest ideas, your contributions are always welcome.&lt;/p&gt;
&lt;p&gt;The rest of the post is about the technical details of a tibble, and therefore mostly suited for interested R programmers:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;What can be part of a tibble?&lt;/li&gt;
&lt;li&gt;Size and length&lt;/li&gt;
&lt;li&gt;Sturdy recycling&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;what-can-be-part-of-a-tibble&#34;&gt;What can be part of a tibble?&lt;/h2&gt;
&lt;p&gt;Tibbles and data frames are collections of columns, where each column is a vector of the same size.
Neat.&lt;/p&gt;
&lt;p&gt;What is a vector?
What is its size?&lt;/p&gt;
&lt;p&gt;The new 
&lt;a href=&#34;https://vctrs.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vctrs package&lt;/a&gt; is dedicated to answering these surprisingly tricky questions.
Because this blog post describes many functions of this package, we load it.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(vctrs)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The 
&lt;a href=&#34;https://vctrs.r-lib.org/reference/vec_assert.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;vec_is()&lt;/code&gt;&lt;/a&gt; function decides if an object is a vector.
This is important, because some objects are inherently scalar and cannot be added as a column to a data frame.&lt;/p&gt;
&lt;p&gt;Obviously, integers, characters, and other atomic objects (logical, numeric, complex, and raw) are vectors.
Environments, functions, and other &amp;ldquo;special&amp;rdquo; types of objects are clearly non-vectors.
Most objects that consist of an atomic type with a &lt;code&gt;&amp;quot;class&amp;quot;&lt;/code&gt; attribute are also vectors: examples are &lt;code&gt;POSIXct&lt;/code&gt; and 
&lt;a href=&#34;https://hms.tidyverse.org/reference/hms.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;hms::hms()&lt;/code&gt;&lt;/a&gt;.
Lists are harder because some lists are vectors and some are not.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;vec_is()&lt;/code&gt; function implements a heuristic that works automatically in most cases and adds a few special cases from base R.
By relying on &lt;code&gt;vec_is()&lt;/code&gt;, the &lt;code&gt;tibble()&lt;/code&gt; function and others can give an early error if used with an inherent scalar:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tibble)
model &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;lm&lt;/span&gt;(y &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; x, &lt;span style=&#34;color:#00f&#34;&gt;data.frame&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, y &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;), model &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(model)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: All columns in a tibble must be vectors.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x Column `model` is a `lm` object.&lt;/span&gt;
time &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;Sys.time&lt;/span&gt;()
&lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(time)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   time               &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dttm&amp;gt;             &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2020-04-09 20:38:28&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The new 
&lt;a href=&#34;https://tibble.tidyverse.org/reference/tibble.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;tibble_row()&lt;/code&gt;&lt;/a&gt; function reverses this: inherent scalars are wrapped in lists:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;tibble_row&lt;/span&gt;(model)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   model &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;list&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 &amp;lt;lm&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;tibble_row&lt;/span&gt;(time)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   time               &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dttm&amp;gt;             &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2020-04-09 20:38:28&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;tibble_row&lt;/span&gt;(time &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(time, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: All vectors must be size one, use `list()` to wrap.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x Column `time` is of size 2.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you have implemented a vector class, double-check that 
&lt;a href=&#34;https://vctrs.r-lib.org/reference/vec_assert.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;vec_is()&lt;/code&gt;&lt;/a&gt; returns &lt;code&gt;TRUE&lt;/code&gt; for your objects.
Please also add it to my 
&lt;a href=&#34;https://github.com/krlmlr/awesome-vctrs#readme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Awesome vectors&lt;/a&gt; list, or file an issue.&lt;/p&gt;
&lt;h2 id=&#34;size-and-length&#34;&gt;Size and length&lt;/h2&gt;
&lt;p&gt;Data frames and matrices are also recognized vectors, and can be part of a tibble:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;df &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;data.frame&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
m &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;matrix&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;, ncol &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;vec_is&lt;/span&gt;(df)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] TRUE&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;vec_is&lt;/span&gt;(m)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] TRUE&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(packed &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; df)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   packed$a    $b&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;      &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1        1     2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2        2     3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3        3     4&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(m)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   m[,1]  [,2]  [,3]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1     3     5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     2     4     6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The &amp;ldquo;elements&amp;rdquo; of a data frame or matrix are its rows.
All subsetting and subassignment operations now use 
&lt;a href=&#34;https://vctrs.r-lib.org/reference/vec_slice.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;vec_slice()&lt;/code&gt;&lt;/a&gt; under the hood.
Contrary to &lt;code&gt;[&lt;/code&gt;, slicing will work along the rows for matrices and data frames.&lt;/p&gt;
&lt;p&gt;For these and a few types, length and size are different: the length refers to the size of the internal data format, whereas the size is the number of elements.
The 
&lt;a href=&#34;https://vctrs.r-lib.org/reference/vec_size.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;vec_size()&lt;/code&gt;&lt;/a&gt; function, modeled after &lt;code&gt;NROW()&lt;/code&gt;, returns the latter:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;vec_size&lt;/span&gt;(df)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;length&lt;/span&gt;(df)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;vec_size&lt;/span&gt;(m)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;length&lt;/span&gt;(m)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 6&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For your own code, it is almost always safer to use &lt;code&gt;vec_size()&lt;/code&gt; instead of &lt;code&gt;length()&lt;/code&gt;.
Use &lt;code&gt;ncol()&lt;/code&gt; to count the columns in a data frame.&lt;/p&gt;
&lt;h2 id=&#34;sturdy-recycling&#34;&gt;Sturdy recycling&lt;/h2&gt;
&lt;p&gt;We always recycled only vectors of size one in &lt;code&gt;tibble()&lt;/code&gt; and &lt;code&gt;as_tibble()&lt;/code&gt;.
This now also applies to subassignment.
We believe that most of the time this is an unintended error.
Please use an explicit &lt;code&gt;rep()&lt;/code&gt; or &lt;code&gt;rep_len()&lt;/code&gt; if you really need to create a column that consists of multiple repetitions of a vector.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
x&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;a &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: Assigned data `1:2` must be compatible with existing data.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x Existing data has 4 rows.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x Assigned data has 2 rows.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ℹ Only vectors of size 1 are recycled.&lt;/span&gt;
x&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;a &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;rep&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
x
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       a&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Related errors may also appear when applying a pattern that works with regular data frames:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;data.frame&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
x[1, ] &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
x
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   a b&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 3 4&lt;/span&gt;

x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
x[1, ] &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: Assigned data `c(a = 3, b = 4)` must be compatible with row subscript `1`.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x 1 row must be assigned.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; x Assigned data has 2 rows.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; ℹ Row updates require a list value. Do you need `list()` or `as.list()`?&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This is because all vectors on the right-hand side are treated as columnar data.
Convert to a list to treat the input as row data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;x[1, ] &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
x
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       a     b&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     3     4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The ambiguity between a row vector and a column vector also affects the &lt;code&gt;as_tibble()&lt;/code&gt; function.
For this reason, it is now superseded for atomic and list inputs.
In new code, use the new 
&lt;a href=&#34;https://tibble.tidyverse.org/reference/as_tibble.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;as_tibble_row()&lt;/code&gt; and &lt;code&gt;as_tibble_col()&lt;/code&gt;&lt;/a&gt; functions to clarify intent.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;as_tibble_row&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       a     b&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     3     4&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;as_tibble_col&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(a &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, b &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   value&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;acknowledgments&#34;&gt;Acknowledgments&lt;/h2&gt;
&lt;p&gt;Due to the nature of the changes, about 60 CRAN packages were failing with our release candidate.
Many thanks to the maintainers of downstream packages who were very helpful in making this upgrade a smooth experience.&lt;/p&gt;
&lt;p&gt;Thanks to the following contributors who sent issues, pull requests, and comments since tibble 2.1.3:&lt;/p&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/adamdsmith&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@adamdsmith&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/alankjackson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@alankjackson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/anabbott&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@anabbott&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/billdenney&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@billdenney&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/borisleto&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@borisleto&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Breza&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Breza&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Cervangirard&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Cervangirard&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/courtiol&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@courtiol&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dan-reznik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dan-reznik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/daviddalpiaz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@daviddalpiaz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DavisVaughan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DavisVaughan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/elinw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@elinw&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/EmilHvitfeldt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@EmilHvitfeldt&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/eran3006&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@eran3006&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/frederikziebell&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@frederikziebell&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gavinsimpson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gavinsimpson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gdequeiroz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gdequeiroz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/guiastrennec&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@guiastrennec&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/HashRocketSyntax&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@HashRocketSyntax&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hope-data-science&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hope-data-science&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jennybc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jennybc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jmgirard&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jmgirard&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kevinwolz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kevinwolz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kieranjmartin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kieranjmartin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lionel-&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lionel-&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/LudvigOlsen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@LudvigOlsen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mabafaba&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mabafaba&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/matteodefelice&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@matteodefelice&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MatthieuStigler&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@MatthieuStigler&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/md0u80c9&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@md0u80c9&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/michaelquinn32&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@michaelquinn32&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mitchelloharawild&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mitchelloharawild&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/moodymudskipper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@moodymudskipper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/msberends&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@msberends&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pavopax&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pavopax&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rbjanis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rbjanis&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/romainfrancois&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@romainfrancois&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rvg02010&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rvg02010&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/sfirke&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@sfirke&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Shians&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Shians&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ShixiangWang&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ShixiangWang&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stephensrmmartin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stephensrmmartin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stufield&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stufield&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Tazinho&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Tazinho&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/TimTeaFan&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@TimTeaFan&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tyluRp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@tyluRp&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wgrundlingh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wgrundlingh&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/xvrdm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@xvrdm&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yannabraham&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yannabraham&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ycroissant&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ycroissant&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yogat3ch&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yogat3ch&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/yutannihilation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yutannihilation&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Your contributions are very valuable and important to us!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>pkgdown 1.5.0</title>
      <link>/blog/2020/03/pkgdown-1-5-0/</link>
      <pubDate>Wed, 25 Mar 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/03/pkgdown-1-5-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re well chuffed to announce that 
&lt;a href=&#34;https://pkgdown.r-lib.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pkgdown&lt;/a&gt; 1.5.0 is now available on CRAN. pkgdown is designed to make it quick and easy to build a website for your package. Install it with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pkgdown&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The most important changes are highlighted below and you can see a full list of changes in the 
&lt;a href=&#34;https://pkgdown.r-lib.org/news/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;changelog&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;articles&#34;&gt;Articles&lt;/h2&gt;
&lt;p&gt;For packages with many vignettes/articles, we&amp;rsquo;ve provided much greater control over the 
&lt;a href=&#34;https://pkgdown.r-lib.org/articles&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;articles index&lt;/a&gt; and navbar. There are two major new features in this release:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The articles index page now displays article &lt;code&gt;description&lt;/code&gt;s, taken from
YAML metadata in the header of each article. This lets you provide
more context for each article.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The articles navbar is now controlled by the &lt;code&gt;articles&lt;/code&gt; section in
&lt;code&gt;_pkgdown.yml&lt;/code&gt;. The ordering of the sections, and articles within
them, control the order of the articles in the navbar, and you can
use the new &lt;code&gt;navbar&lt;/code&gt; field to control whether or not each section
appears in the navbar.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Learn more about both of these features in 
&lt;a href=&#34;https://pkgdown.r-lib.org/reference/build_articles.html#index-and-navbar&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;?&lt;code&gt;build_articles&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Thanks to 
&lt;a href=&#34;https://github.com/gadenbuie&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Garrick Aden-Buie&lt;/a&gt;, you also get much richer control over Open Graph and Twitter metadata for individual articles. See new 
&lt;a href=&#34;https://pkgdown.r-lib.org/articles/metadata.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;vignette(&amp;quot;metadata&amp;quot;)&lt;/code&gt;&lt;/a&gt; for details.&lt;/p&gt;
&lt;h2 id=&#34;reference-index&#34;&gt;Reference index&lt;/h2&gt;
&lt;p&gt;For packages with many documentation topics, you can add an additional layer of hierarchy to the reference index, using the new &lt;code&gt;subtitle&lt;/code&gt; field. To give you some sense for what that might look like, here&amp;rsquo;s an example for a partial (and imaginary) dplyr reference index:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;references&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;title&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;Data&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;manipulation&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;subtitle&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;One&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;table&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;contents&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;- arrange&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;- filter&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;- mutate&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;subtitle&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;two&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;table&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;contents&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;- ends_with(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;_join&amp;#34;&lt;/span&gt;)&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;title&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;Datasets&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;&lt;/span&gt;- &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;contents&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; 
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;- has_keyword(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;datasets&amp;#34;&lt;/span&gt;)&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;tables-of-contents&#34;&gt;Tables of contents&lt;/h2&gt;
&lt;p&gt;Sidebar tables of contents now use 
&lt;a href=&#34;https://afeld.github.io/bootstrap-toc/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bootstrap-toc&lt;/a&gt;, which considerably improves navigation for long articles and reference pages.&lt;/p&gt;
&lt;h2 id=&#34;other-source-repositories&#34;&gt;Other source repositories&lt;/h2&gt;
&lt;p&gt;You can now control the links to source files (in reference pages and articles) and issues and users (in the NEWS) with new the &lt;code&gt;repo$url&lt;/code&gt; config parameter. This makes it easier to use pkgdown with GitHub enterprise, packages in subdirectories, and other source hosts (like bitbucket).&lt;/p&gt;
&lt;p&gt;The YAML looks something like this:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-yaml&#34; data-lang=&#34;yaml&#34;&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;repo&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;  &lt;/span&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;url&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;    &lt;/span&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;home&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;https://github.com/r-lib/pkgdown/&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;    &lt;/span&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;source&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;https://github.com/r-lib/pkgdown/blob/master/&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;    &lt;/span&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;issue&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;https://github.com/r-lib/pkgdown/issues/&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;span style=&#34;color:#bbb&#34;&gt;    &lt;/span&gt;&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;user&lt;/span&gt;:&lt;span style=&#34;color:#bbb&#34;&gt; &lt;/span&gt;https://github.com/&lt;span style=&#34;color:#bbb&#34;&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The individual components (e.g. path, issue number, username) are pasted on the end of these urls so they should have trailing &lt;code&gt;/&lt;/code&gt;s.&lt;/p&gt;
&lt;p&gt;pkgdown now detects GitLab urls automatically (since they use the same structure as GitHub), so you don&amp;rsquo;t need to set these links if you package is hosted on GitLab, and you&amp;rsquo;ve included a link to your source repo in the &lt;code&gt;URL&lt;/code&gt; or &lt;code&gt;BugReports&lt;/code&gt; &lt;code&gt;DESCRIPTION&lt;/code&gt; fields.&lt;/p&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;A big thank you goes to 
&lt;a href=&#34;https://github.com/jayhesselberth&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jayhesselberth&lt;/a&gt; (the co-maintainer of pkgdown), and to to the 61 other people who helped make this release possible:

&lt;a href=&#34;https://github.com/AshesITR&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@AshesITR&lt;/a&gt;,  
&lt;a href=&#34;https://github.com/baptiste&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@baptiste&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Bisaloo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Bisaloo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/carloscinelli&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@carloscinelli&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cboettig&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cboettig&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/coatless&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@coatless&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/coolbutuseless&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@coolbutuseless&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DanielEWeeks&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DanielEWeeks&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/davidchall&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@davidchall&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DavorJ&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DavorJ&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dimagor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dimagor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/erikcs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@erikcs&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ferroao&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ferroao&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/floriandeboissieu&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@floriandeboissieu&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/flying-sheep&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@flying-sheep&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fmichonneau&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fmichonneau&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/fmmattioni&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@fmmattioni&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gaborcsardi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gaborcsardi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/genomaths&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@genomaths&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gustavdelius&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gustavdelius&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hbaniecki&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hbaniecki&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ijlyttle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ijlyttle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/IndrajeetPatil&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@IndrajeetPatil&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jangorecki&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jangorecki&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jayhesselberth&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jayhesselberth&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jeffwong-nflx&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jeffwong-nflx&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jennybc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jennybc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jeroen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jeroen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/JoshuaSturm&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@JoshuaSturm&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jranke&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jranke&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kevinushey&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kevinushey&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kevinwang09&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kevinwang09&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/krlmlr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@krlmlr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lbusett&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lbusett&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lcolladotor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lcolladotor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lgatto&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lgatto&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lindeloev&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lindeloev&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lionel-&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lionel-&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lorenzwalthert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@lorenzwalthert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/m-l-1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@m-l-1&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maelle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maelle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mattmalin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mattmalin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/meghapsimatrix&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@meghapsimatrix&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikldk&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mikldk&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mllg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mllg&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ms609&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ms609&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nealrichardson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nealrichardson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nschiett&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nschiett&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nteetor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nteetor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pat-s&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pat-s&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/peterdesmet&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@peterdesmet&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rupertoverall&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@rupertoverall&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/schloerke&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@schloerke&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/slowkow&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@slowkow&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/t-kalinowski&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@t-kalinowski&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wendtke&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wendtke&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ycphs&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ycphs&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yiluheihei&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yiluheihei&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/yonicd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yonicd&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>roxygen2 7.1.0</title>
      <link>/blog/2020/03/roxygen2-7-1-0/</link>
      <pubDate>Wed, 11 Mar 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/03/roxygen2-7-1-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re chuffed to announce the release of 
&lt;a href=&#34;https://roxygen2.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;roxygen2 7.1.0&lt;/a&gt;.
roxygen2 allows you to write specially formatted R comments that generate R documentation files (&lt;code&gt;man/*.Rd&lt;/code&gt;) and the &lt;code&gt;NAMESPACE&lt;/code&gt; file.
roxygen2 is used by over 8,800 CRAN packages.&lt;/p&gt;
&lt;p&gt;Install the latest version of roxygen2 with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;roxygen2&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This is a minor release, with many small improvements and bug fixes, and some new features.
This blog post discusses the most important changes. See the 
&lt;a href=&#34;https://roxygen2.r-lib.org/news/index.html#roxygen2-7-1-0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;full changelog&lt;/a&gt; for all changes.&lt;/p&gt;
&lt;h2 id=&#34;roxygen2-meets-knitr&#34;&gt;roxygen2 meets knitr&lt;/h2&gt;
&lt;p&gt;We have been gradually improving roxygen2&amp;rsquo;s markdown support, and this release has an exciting new markdown feature as well.
Starting from roxygen2 7.1.0 you can write inline R code and markdown code fences directly in roxygen2 comments!
roxygen2 runs the R code and inserts its output into the documentation.&lt;/p&gt;
&lt;h3 id=&#34;inline-r-code&#34;&gt;Inline R code&lt;/h3&gt;
&lt;p&gt;For running R code inline, use the usual markdown backticks, and prefix the code with &lt;code&gt;r&lt;/code&gt; and a space character.
roxygen2 interprets the rest of the text within backticks as R code, evaluates it, and replaces the backtick expression with its value.
After all such substitutions, the text of the whole tag is interpreted as markdown, as usual.&lt;/p&gt;
&lt;p&gt;For example, the following code inserts the date and the R version of the roxygen2 run.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; roxygen2 created this manual page on `r Sys.Date()` using R version&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; `r getRversion()`.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;code-blocks-fences&#34;&gt;Code blocks (fences)&lt;/h3&gt;
&lt;p&gt;Markdown code blocks can be dynamic as well, if you use &lt;code&gt;```{r}&lt;/code&gt; to start them, just like in knitr documents.
roxygen evaluates the code, and (by default) both the code and the printed value of the expression will be inserted into the manual page.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```{r}&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; # This block of code will be evaluated&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; summary(iris)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Code blocks support knitr chunk options, e.g. to keep the output of several expressions together, you can specify &lt;code&gt;results= &amp;quot;hold&amp;quot;&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```{r results = &amp;#34;hold&amp;#34;}&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; names(mtcars)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; nrow(mtcars)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;By default plots create &lt;code&gt;.png&lt;/code&gt; files in the &lt;code&gt;man/figures&lt;/code&gt; directory. The file names are created from the chunk names:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```{r r-cookbook-barplot}&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; # https://r-graphics.org/recipe-distribution-basic-hist&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; library(ggplot2)&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ggplot(faithful, aes(x = waiting)) +&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39;   geom_histogram()&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;#39; ```&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Both the inline R code and the markdown code fences are evaluated when you run &lt;code&gt;devtools::document()&lt;/code&gt; (or &lt;code&gt;roxygenize()&lt;/code&gt;).
You probably want to avoid lengthy computations, or set up caching to keep your package development workflow snappy.&lt;/p&gt;
&lt;p&gt;Please see &lt;code&gt;vignette(&amp;quot;rd-formatting&amp;quot;)&lt;/code&gt; for more about dynamic documentation, including potential caveats.&lt;/p&gt;
&lt;h2 id=&#34;line-ending-conversion&#34;&gt;Line ending conversion&lt;/h2&gt;
&lt;p&gt;This version of roxygen2 does a much better job at keeping the line ending characters consistent within each file, across operating systems.
In particular, if a generated file has only Windows (CR LF) line endings, roxygen2 keeps the file that way.
If a file has mixed Windows and Unix (LF) line endings, roxygen2 converts all line endings to Unix (LF).
For new files roxygen2 uses LF file endings.
If you want to convert a file from CR LF line endings to LF, simply remove it and let roxygen2 re-create it.&lt;/p&gt;
&lt;p&gt;The new convention works better with git on Windows.
Windows git, depending on configuration, might or might not convert between CR LF and LF line endings when checking out and when committing code.
No matter which git option you use, roxygen2 now does not introduce spurious line ending changes.&lt;/p&gt;
&lt;p&gt;Thanks to 
&lt;a href=&#34;https://github.com/jonthegeek&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jonthegeek&lt;/a&gt; and 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt; for working on this feature.&lt;/p&gt;
&lt;h2 id=&#34;some-other-improvements&#34;&gt;Some other improvements&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Hyperlinks to R6 methods are also added in the PDF manual.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;@description NULL&lt;/code&gt; and &lt;code&gt;@details NULL&lt;/code&gt; no longer fail; instead, these tags
are ignored, except for &lt;code&gt;@description NULL&lt;/code&gt; in package level documentation,
where it can be used to suppress the auto-generated &amp;lsquo;Description&amp;rsquo; section.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Multiple &lt;code&gt;@format&lt;/code&gt; tags are now combined.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;@evalNamespace()&lt;/code&gt; works again.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;A big thanks to all 40 contributors who helped make this release possible! 
&lt;a href=&#34;https://github.com/alandipert&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@alandipert&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/allenzhuaz&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@allenzhuaz&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/BenEngbers&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@BenEngbers&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bgctw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@bgctw&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/billdenney&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@billdenney&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Bisaloo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Bisaloo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cboettig&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@cboettig&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dmurdoch&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dmurdoch&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dragosmg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dragosmg&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/eddelbuettel&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@eddelbuettel&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gaborcsardi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gaborcsardi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/genomaths&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@genomaths&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/goldingn&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@goldingn&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/HenrikBengtsson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@HenrikBengtsson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Hong-Revo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Hong-Revo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hughjonesd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hughjonesd&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/iferres&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@iferres&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/IndrajeetPatil&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@IndrajeetPatil&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jameslamb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jameslamb&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kingaa&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kingaa&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kortschak&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kortschak&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/krlmlr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@krlmlr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maelle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@maelle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/michaelquinn32&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@michaelquinn32&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikemahoney218&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mikemahoney218&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mstr3336&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mstr3336&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/muschellij2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@muschellij2&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/nteetor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@nteetor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ntguardian&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ntguardian&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pat-s&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pat-s&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/RaphaelS1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@RaphaelS1&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/russHyde&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@russHyde&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/s-fleck&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@s-fleck&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stefanfritsch&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stefanfritsch&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/strboul&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@strboul&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/TomKellyGenetics&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@TomKellyGenetics&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/VPetukhov&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@VPetukhov&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/zachary-foster&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@zachary-foster&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>More 2020 tidy dev days!</title>
      <link>/blog/2020/02/tidy-dev-days-2020/</link>
      <pubDate>Fri, 28 Feb 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/02/tidy-dev-days-2020/</guid>
      <description>&lt;h2 id=&#34;post-rstudioconf2020-dev-day&#34;&gt;Post-rstudio::conf(2020) dev day&lt;/h2&gt;
&lt;p&gt;First and foremost: thank you to everyone who participated in Tidyverse developer day following rstudio::conf(2020)! We merged 
&lt;a href=&#34;https://twitter.com/juliasilge/status/1223408797378416640?s=20&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;60 pull requests&lt;/a&gt; on the day of, have merged 
&lt;a href=&#34;https://github.com/search?q=is%3Apr&amp;#43;label%3A%22tidy-dev-day&amp;#43;%3Anerd_face%3A%22&amp;#43;created%3A%3E2020-01-30&amp;#43;is%3Amerged&amp;amp;type=Issues&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;over 80&lt;/a&gt; PRs to date, with roughly 
&lt;a href=&#34;https://github.com/search?q=is%3Apr&amp;#43;label%3A%22tidy-dev-day&amp;#43;%3Anerd_face%3A%22&amp;#43;created%3A%3E2020-01-30&amp;#43;is%3Aunmerged&amp;#43;is%3Aopen&amp;amp;type=Issues&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;15&lt;/a&gt; more in the pipeline for review.&lt;/p&gt;
&lt;p&gt;Having participants ring the gong themselves after their PRs were accepted was definitely a change for the better! Witness Gabriel Odom in action 
&lt;a href=&#34;https://twitter.com/RevDocGabriel/status/1223445860421230592?s=20&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;save-the-date-pre-user-2020-dev-day&#34;&gt;Save the date: Pre-useR! 2020 dev day&lt;/h2&gt;
&lt;p&gt;We will be holding our next Tidyverse developer day on &lt;strong&gt;July 6th&lt;/strong&gt; in St. Louis prior to 
&lt;a href=&#34;https://user2020.r-project.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;useR! 2020&lt;/a&gt;.
We haven&amp;rsquo;t yet finalized all the details, but it will be in the conference hotel.
Tickets will go on sale once the logistics are firmly in place.&lt;/p&gt;
&lt;p&gt;Please note that there will be the first ever R 
&lt;a href=&#34;https://github.com/useR-stl/geospatial-dev-day&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Geospatial Dev Day&lt;/a&gt; at the same time, run by the Geospatial-Track team, and led by 
&lt;a href=&#34;https://angela-li.github.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Angela Li&lt;/a&gt;! If you work with geospatial data in R, we highly recommend that you check it out (especially if you&amp;rsquo;ve already been to a Tidyverse developer day).&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>themis 0.1.0</title>
      <link>/blog/2020/02/themis-0-1-0/</link>
      <pubDate>Tue, 11 Feb 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/02/themis-0-1-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re chuffed to announce the release of 
&lt;a href=&#34;https://github.com/tidymodels/themis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;themis&lt;/a&gt; on CRAN. 
&lt;a href=&#34;https://tidymodels.github.io/themis/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;themis&lt;/a&gt; implements a collection of new steps for the 
&lt;a href=&#34;https://github.com/tidymodels/recipes&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;recipes&lt;/a&gt; package to deal with unbalanced data. themis is still in early development so any and all feedback is highly appreciated.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(modeldata)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(recipes)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(themis)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;In a classification context, a dataset is said to be unbalanced if there is an unequal number of observations in each class. Many models perform best when the number of observations is equal and, thus, tend to struggle with unbalanced data.&lt;/p&gt;
&lt;p&gt;The steps in this package can be divided into two camps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Ones that remove observations from the majority class(es), and&lt;/li&gt;
&lt;li&gt;Ones that add observations to the minority class(es).&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You can do more than one action, and, thus, are able to mix and match by for example first removing observations from the majority class followed by adding observations to the minority class to achieve the balance you want.&lt;/p&gt;
&lt;h2 id=&#34;hybrid-sampling-steps&#34;&gt;Hybrid-sampling steps&lt;/h2&gt;
&lt;p&gt;Hybrid-sampling involves adding observations to the minority class. This can be done in multiple ways, one way is to sample existing data points like &lt;code&gt;step_upsample()&lt;/code&gt; does, another way is to synthetically generate new points based on existing points, &lt;code&gt;step_smote()&lt;/code&gt; and related steps uses k nearest neighbor information to generate new points. Currently &lt;code&gt;step_upsample()&lt;/code&gt;, &lt;code&gt;step_smote()&lt;/code&gt;, &lt;code&gt;step_bsmote()&lt;/code&gt;, &lt;code&gt;step_adasyn()&lt;/code&gt;, and &lt;code&gt;step_rose()&lt;/code&gt; are available. All steps have references in their respective help pages. They have slightly different requirements according to the data they can handle; most need all numeric with no missing data, but those requirements can and should be handled by previous steps.&lt;/p&gt;
&lt;p&gt;In the following example, let&amp;rsquo;s look at the &lt;code&gt;okc&lt;/code&gt; dataset. and we can see that the imbalance is 1-to-6.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;data&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;okc&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;(okc&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;Class)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  stem other &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9539 50316&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;We will use &lt;code&gt;age&lt;/code&gt;, &lt;code&gt;diet&lt;/code&gt; and &lt;code&gt;height&lt;/code&gt; in modeling to predict &lt;code&gt;Class&lt;/code&gt;. Since &lt;code&gt;diet&lt;/code&gt; is a factor, we first need to dummify it before we normalize and perform mean imputation to handle all the missing data.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(Class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; age &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; diet &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; height, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; okc) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_unknown&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_meanimpute&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_smote&lt;/span&gt;(Class) 

rec &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;pull&lt;/span&gt;(Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; .&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  stem other &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 50316 50316&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And we see that the resulting dataset has a perfectly even distribution. All the hybrid-sampling steps share the parameter &lt;code&gt;over_ratio&lt;/code&gt;, which specifies the desired ratio between the biggest class and the smallest class. It defaults to 1 for an even distribution but can be set to something like &lt;code&gt;0.5&lt;/code&gt; to have the minority class become half the size of the majority class.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(Class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; age &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; diet &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; height, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; okc) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_unknown&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_meanimpute&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_smote&lt;/span&gt;(Class, over_ratio &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0.5&lt;/span&gt;) 

rec &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;pull&lt;/span&gt;(Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; .&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  stem other &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 25158 50316&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;under-sampling-steps&#34;&gt;Under-sampling steps&lt;/h2&gt;
&lt;p&gt;Under-sampling is removing observations from the majority class. Currently &lt;code&gt;step_downsample()&lt;/code&gt;, &lt;code&gt;step_nearmiss()&lt;/code&gt; and &lt;code&gt;step_tomek()&lt;/code&gt; are available. These steps should have the same user experience as the previous steps as they have a similar shared parameter &lt;code&gt;under_ratio&lt;/code&gt; which is the ratio between the smallest and the biggest class. Simply using &lt;code&gt;step_downsample()&lt;/code&gt; randomly removes samples in the majority classes to get them to be the same size as the smallest class.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;rec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;recipe&lt;/span&gt;(Class &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; age &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; diet &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; height, data &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; okc) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_unknown&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_dummy&lt;/span&gt;(diet) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_normalize&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_meanimpute&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;all_predictors&lt;/span&gt;()) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;step_downsample&lt;/span&gt;(Class) 

rec &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;prep&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;juice&lt;/span&gt;() &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;pull&lt;/span&gt;(Class) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;table&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; .&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  stem other &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9539  9539&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description>
    </item>
    
    <item>
      <title>slider 0.1.0</title>
      <link>/blog/2020/02/slider-0-1-0/</link>
      <pubDate>Mon, 10 Feb 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/02/slider-0-1-0/</guid>
      <description>&lt;p&gt;I&amp;rsquo;m thrilled to announce that the first version of 
&lt;a href=&#34;https://davisvaughan.github.io/slider/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;slider&lt;/a&gt; is now available on CRAN!&lt;/p&gt;
&lt;p&gt;slider provides a family of general purpose sliding window functions, which can be used to compute moving averages, cumulatives sums, rolling regressions, and any other sliding operation.&lt;/p&gt;
&lt;p&gt;This package is a combination of ideas from a variety of sources, including:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://purrr.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;purrr&lt;/a&gt; for the overall package API&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://www.postgresql.org/docs/9.1/sql-expressions.html#SYNTAX-WINDOW-FUNCTIONS&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;SQL&amp;rsquo;s window functions&lt;/a&gt; for the argument API&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/tidyverts/tsibble&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Earo Wang&amp;rsquo;s &lt;code&gt;tsibble::slide()&lt;/code&gt;&lt;/a&gt; for the function names&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://rdatatable.gitlab.io/data.table/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;data.table&amp;rsquo;s non-equi joins&lt;/a&gt; for inspiration on how &lt;code&gt;slide_index()&lt;/code&gt; should work&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Install slider with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;slider&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This blog post summarizes the three key functions in slider: &lt;code&gt;slide()&lt;/code&gt;, &lt;code&gt;slide_index()&lt;/code&gt;, and &lt;code&gt;slide_period()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(slider)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(tibble)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(purrr)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(lubridate, warn.conflicts &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(dplyr, warn.conflicts &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;FALSE&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;sliding&#34;&gt;Sliding&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;purrr::map()&lt;/code&gt; allows you to iterate over a vector one element at a time and apply a function to each element. &lt;code&gt;slide()&lt;/code&gt; takes that concept and generalizes it so that you can iterate over &lt;em&gt;sliding windows&lt;/em&gt; of a vector, applying any function that you want to each window. To start exploring this idea, note that the defaults of &lt;code&gt;slide()&lt;/code&gt; are essentially identical to &lt;code&gt;map()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# A vector of sales data for our business&lt;/span&gt;
sales_vec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(sales_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Things get more interesting when you consider the additional arguments to &lt;code&gt;slide()&lt;/code&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;.before&lt;/code&gt;: How many elements &lt;em&gt;before&lt;/em&gt; the current one should be included in the window?&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;.after&lt;/code&gt;: How many elements &lt;em&gt;after&lt;/em&gt; the current one should be included in the window?&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;.complete&lt;/code&gt;: Should &lt;code&gt;.f&lt;/code&gt; only be evaluated when there is enough data to make a complete window?&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;.step&lt;/code&gt;: The number of elements to shift forward between calls to &lt;code&gt;.f&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;By setting &lt;code&gt;.before = 1&lt;/code&gt; we can construct moving windows along &lt;code&gt;.x&lt;/code&gt;, adding the current element and the one before it into the window.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(sales_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 4 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3 5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Notice how in the first result our slice just has one element. This is an &lt;em&gt;incomplete&lt;/em&gt; window because it isn&amp;rsquo;t possible to look one element before the first element. By default, &lt;code&gt;slide()&lt;/code&gt; computes &lt;code&gt;.f&lt;/code&gt; on incomplete windows, but you can force it to only be computed on complete windows with &lt;code&gt;.complete&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(sales_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, .complete &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; NULL&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2 4&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 4 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3 5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The API of slider is intentionally very similar to purrr. &lt;code&gt;slide()&lt;/code&gt; always returns a list (like &lt;code&gt;map()&lt;/code&gt;), and the size of the result is always the same size as the input. As you might have guessed, there are also suffixed versions available to return more specific output. Say we want to compute a 2-value moving average from our sales. We might use:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide_dbl&lt;/span&gt;(sales_vec, mean, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.0 3.0 3.5 4.0&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;There is also a new suffix, &lt;code&gt;*_vec()&lt;/code&gt;, which attempts to automatically simplify the results using the type rules provided by 
&lt;a href=&#34;https://vctrs.r-lib.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vctrs&lt;/a&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide_vec&lt;/span&gt;(sales_vec, mean, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.0 3.0 3.5 4.0&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;slide_vec&lt;/span&gt;(sales_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(.x) &lt;span style=&#34;color:#666&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] FALSE FALSE  TRUE  TRUE&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Lastly, the one big difference between how &lt;code&gt;slide()&lt;/code&gt; and &lt;code&gt;map()&lt;/code&gt; iterate over vectors is how they treat data frames. To &lt;code&gt;map()&lt;/code&gt;, a data frame is a vector of columns, to &lt;code&gt;slide()&lt;/code&gt; it is a 
&lt;a href=&#34;https://blog.davisvaughan.com/2019/10/16/data-frames-as-vectors-of-rows/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vector of rows&lt;/a&gt;. In a way, this makes &lt;code&gt;slide()&lt;/code&gt; a generic row-wise iterator over data frames.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;index_vec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.Date&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;2019-08-29&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;)
wday_vec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;as.character&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;wday&lt;/span&gt;(index_vec, label &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;))

company &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(
  sales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; sales_vec,
  index &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; index_vec,
  wday &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; wday_vec
)

company
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4 2019-08-30 Fri  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     3 2019-09-03 Tue  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     5 2019-09-04 Wed&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Over columns:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;map&lt;/span&gt;(company, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $sales&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2 4 3 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $index&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-08-29&amp;#34; &amp;#34;2019-08-30&amp;#34; &amp;#34;2019-09-03&amp;#34; &amp;#34;2019-09-04&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $wday&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34; &amp;#34;Fri&amp;#34; &amp;#34;Tue&amp;#34; &amp;#34;Wed&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Over rows:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(company, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     4 2019-08-30 Fri  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     3 2019-09-03 Tue  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     5 2019-09-04 Wed&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You can also still use the additional arguments to construct sliding windows along your data frame.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(company, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4 2019-08-30 Fri  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4 2019-08-30 Fri  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     3 2019-09-03 Tue  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     4 2019-08-30 Fri  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     3 2019-09-03 Tue  &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     5 2019-09-04 Wed&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;index-sliding&#34;&gt;Index sliding&lt;/h2&gt;
&lt;p&gt;Throughout R&amp;rsquo;s history, a few of the features of &lt;code&gt;slide()&lt;/code&gt; have been available in other packages. You could accomplish some of the same things with &lt;code&gt;zoo::rollapply()&lt;/code&gt;, &lt;code&gt;tsibble::slide()&lt;/code&gt;, and even with my original attempt at this, &lt;code&gt;tibbletime::rollify()&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;But none of these methods ever solved a problem that is incredibly common in business-oriented data analysis. What happens when you have a date index attached to when the sales happened, and you want to compute a moving average over &lt;em&gt;two days&lt;/em&gt;? &lt;code&gt;slide()&lt;/code&gt; doesn&amp;rsquo;t solve this problem either, because you might have date gaps in your vector that it doesn&amp;rsquo;t know about. To demonstrate this, let&amp;rsquo;s assume that you are interested in computing those two day windows. To visualize them, we&amp;rsquo;ll print out the week day that would be associated with these naive windows if we used &lt;code&gt;slide()&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;wday_vec
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34; &amp;#34;Fri&amp;#34; &amp;#34;Tue&amp;#34; &amp;#34;Wed&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;slide&lt;/span&gt;(wday_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34; &amp;#34;Fri&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Fri&amp;#34; &amp;#34;Tue&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Tue&amp;#34; &amp;#34;Wed&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Notice the third window! We started on Tuesday, which means the window should only include &lt;code&gt;[Mon, Tue]&lt;/code&gt;, but Friday is also included here. This happens because &lt;code&gt;slide()&lt;/code&gt; looks back a number of &lt;em&gt;values&lt;/em&gt;, and knows nothing about how to compute this &lt;code&gt;[Mon, Tue]&lt;/code&gt; &lt;em&gt;range&lt;/em&gt; to slide between. This differentiation between values and ranges comes from SQL, and is further explained in a bit more detail by 
&lt;a href=&#34;https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Analytic/window_frame_clause.htm?origin_team=T02V9CHFH#ROWSversusRANGE&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Vertica&amp;rsquo;s window function documentation&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;To solve this specific problem of sliding with respect to an index, we&amp;rsquo;ll need a new function, &lt;code&gt;slide_index()&lt;/code&gt;. It&amp;rsquo;s similar to &lt;code&gt;slide()&lt;/code&gt;, and has all of the same suffixed versions, but allows you to pass in a secondary index to slide relative to. Ranges to slide between are computed as &lt;code&gt;.i - .before&lt;/code&gt; and &lt;code&gt;.i + .after&lt;/code&gt;, meaning that you can use more interesting objects for &lt;code&gt;.before&lt;/code&gt;, like &lt;code&gt;lubridate::days()&lt;/code&gt;! It just has to implement &lt;code&gt;-&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; methods when combined with your index.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;wday_vec
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34; &amp;#34;Fri&amp;#34; &amp;#34;Tue&amp;#34; &amp;#34;Wed&amp;#34;&lt;/span&gt;
index_vec
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-08-29&amp;#34; &amp;#34;2019-08-30&amp;#34; &amp;#34;2019-09-03&amp;#34; &amp;#34;2019-09-04&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;slide_index&lt;/span&gt;(.x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; wday_vec, .i &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; index_vec, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;days&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Thu&amp;#34; &amp;#34;Fri&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Tue&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;Tue&amp;#34; &amp;#34;Wed&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This correctly buckets Tuesday in its own group, since there is no data point for the Monday before it. We can compare the difference between a two-value vs a two-day moving average like so:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;company &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(
    two_value &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;slide_dbl&lt;/span&gt;(sales, mean, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;),
    two_day &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;slide_index_dbl&lt;/span&gt;(sales, index, mean, .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;days&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)),
  )
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index      wday  two_value two_day&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;     &amp;lt;chr&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-29 Thu         2         2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4 2019-08-30 Fri         3         3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     3 2019-09-03 Tue         3.5       3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     5 2019-09-04 Wed         4         4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;period-sliding&#34;&gt;Period sliding&lt;/h2&gt;
&lt;p&gt;While &lt;code&gt;slide()&lt;/code&gt; and &lt;code&gt;slide_index()&lt;/code&gt; are great because they are size-stable, sometimes you&amp;rsquo;ll want to take data that has a daily index, break it into monthly chunks, and return results at the monthly level. This implies returning an output that has a different size from your input. To power these ideas, you can use &lt;code&gt;slide_period()&lt;/code&gt;, which takes an index and a period to chunk by, and then iterates over &lt;code&gt;.x&lt;/code&gt; relative to those period chunks.&lt;/p&gt;
&lt;p&gt;Say we want to take &lt;code&gt;big_company&lt;/code&gt; below, break it into monthly chunks, and compute and return just the monthly totals.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;big_index_vec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(
  &lt;span style=&#34;color:#00f&#34;&gt;as.Date&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;2019-08-30&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;,
  &lt;span style=&#34;color:#00f&#34;&gt;as.Date&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;2019-11-30&amp;#34;&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;
)

big_sales_vec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;8&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;9&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)

big_company &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(
  sales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; big_sales_vec,
  index &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; big_index_vec
)

big_company
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 10 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    sales index     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;    &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  1     2 2019-08-30&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  2     4 2019-08-31&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  3     6 2019-09-01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  4     2 2019-09-02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  5     8 2019-09-03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  6    10 2019-11-30&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  7     9 2019-12-01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  8     3 2019-12-02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  9     5 2019-12-03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 10     2 2019-12-04&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;slide_period()&lt;/code&gt; allows you to iterate over your data frame in these monthly chunks, applying whatever function you want to each one.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide_period&lt;/span&gt;(big_company, big_company&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;index, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt;.x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     2 2019-08-30&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     4 2019-08-31&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 3 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     6 2019-09-01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     2 2019-09-02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     8 2019-09-03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1    10 2019-11-30&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[4]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   sales index     &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;date&amp;gt;    &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     9 2019-12-01&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2     3 2019-12-02&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3     5 2019-12-03&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4     2 2019-12-04&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;I find it easiest to wrap up what you want to do into a function, and then apply that to each slice.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;monthly_summary &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(data) {
  &lt;span style=&#34;color:#00f&#34;&gt;summarise&lt;/span&gt;(data, start &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;min&lt;/span&gt;(index), end &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;max&lt;/span&gt;(index), total_sales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(sales))
}

&lt;span style=&#34;color:#00f&#34;&gt;slide_period_dfr&lt;/span&gt;(
  big_company,
  big_company&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;index,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;,
  monthly_summary
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   start      end        total_sales&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;date&amp;gt;     &amp;lt;date&amp;gt;           &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2019-08-30 2019-08-31           6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 2019-09-01 2019-09-03          16&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 2019-11-30 2019-11-30          10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 2019-12-01 2019-12-04          19&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, you might recognize that you can do this with dplyr:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;big_company &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;mutate&lt;/span&gt;(monthly &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;floor_date&lt;/span&gt;(index, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;)) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;group_by&lt;/span&gt;(monthly) &lt;span style=&#34;color:#666&#34;&gt;%&amp;gt;%&lt;/span&gt;
  &lt;span style=&#34;color:#00f&#34;&gt;summarise&lt;/span&gt;(sales &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;sum&lt;/span&gt;(sales))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   monthly    sales&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;date&amp;gt;     &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2019-08-01     6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 2019-09-01    16&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 2019-11-01    10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 2019-12-01    19&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;But what you can&amp;rsquo;t easily do is slide over multiple monthly chunks at once, effectively creating a rolling monthly total, from daily data. With &lt;code&gt;slide_period()&lt;/code&gt;, &lt;code&gt;.before&lt;/code&gt; works at the period level, so you get to control how many monthly periods are included in your sliding window. Notice how the start dates below go back into the previous month (but only if there was data for it).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;slide_period_dfr&lt;/span&gt;(
  big_company,
  big_company&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;index,
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;month&amp;#34;&lt;/span&gt;,
  monthly_summary,
  .before &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 4 x 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   start      end        total_sales&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;date&amp;gt;     &amp;lt;date&amp;gt;           &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 2019-08-30 2019-08-31           6&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 2019-08-30 2019-09-03          22&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 3 2019-11-30 2019-11-30          10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 4 2019-11-30 2019-12-04          29&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;A big thanks to some of the early adopters of slider!&lt;/p&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/AlanFeder&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@AlanFeder&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/AlunHewinson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@AlunHewinson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/echasnovski&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@echasnovski&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mik3y64&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mik3y64&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/perlatex&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@perlatex&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/RobertMyles&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@RobertMyles&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>rstudio::conf 2020</title>
      <link>/events/rstudio-conf-2020/</link>
      <pubDate>Mon, 27 Jan 2020 00:00:00 +0000</pubDate>
      <guid>/events/rstudio-conf-2020/</guid>
      <description>&lt;p&gt;rstudio::conf 2020 covers all things RStudio, including workshops to teach you the tidyverse, and talks to show you the latest and greatest features.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>vroom 1.1.0</title>
      <link>/blog/2020/01/vroom-1-1-0/</link>
      <pubDate>Wed, 15 Jan 2020 00:00:00 +0000</pubDate>
      <guid>/blog/2020/01/vroom-1-1-0/</guid>
      <description>&lt;p&gt;I&amp;rsquo;m excited to announce that 
&lt;a href=&#34;http://vroom.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;vroom 1.1.0&lt;/a&gt; is now
available on CRAN!&lt;/p&gt;
&lt;p&gt;vroom reads rectangular data, such as comma separated (csv), tab separated (tsv) or fixed width files (fwf) into R.
It performs similar roles to functions like 
&lt;a href=&#34;http://readr.r-lib.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;readr::read_csv()&lt;/code&gt;&lt;/a&gt;, 
&lt;a href=&#34;http://r-datatable.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;data.table::fread()&lt;/code&gt;&lt;/a&gt; or &lt;code&gt;read.csv()&lt;/code&gt;.
But for many datasets &lt;code&gt;vroom::vroom()&lt;/code&gt; can read them much, much faster (hence the name).
Get the latest version with:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;vroom&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;And attach the package by running&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(vroom)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Improvements in this release include: a hex logo, support for big integer data, improved delimiter guessing, including delimiters in specifications, and streamlined reading from standard input.&lt;/p&gt;
&lt;p&gt;See the 
&lt;a href=&#34;https://vroom.r-lib.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;change log&lt;/a&gt; for a full list of changes and bug fixes in this version.&lt;/p&gt;
&lt;h2 id=&#34;hex-logo&#34;&gt;Hex logo&lt;/h2&gt;
&lt;p&gt;Thanks to 
&lt;a href=&#34;https://github.com/allisonhorst/stats-illustrations&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Allison Horst&lt;/a&gt; we now have an awesome hex logo for vroom!&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;logo.png&#34; alt=&#34;&#34;&gt;&lt;/p&gt;
&lt;h2 id=&#34;big-integer-support&#34;&gt;Big integer support&lt;/h2&gt;
&lt;p&gt;R&amp;rsquo;s standard integers are stored in 32 bits of binary data, which means that the largest value they can store is 2,147,483,647 (&lt;code&gt;2^31 - 1&lt;/code&gt;).
R implicitly converts integers for most operations with doubles to 64-bit floating point values, which is why you may not have noticed this limitation before.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;options&lt;/span&gt;(digits &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;22&lt;/span&gt;)
x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2147483647L&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  int 2147483647&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1L&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Warning in x + 1L: NAs produced by integer overflow&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  int NA&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  num 2.15e+09&lt;/span&gt;
x &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2147483648&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;However, even 64-bit floating point values can only store consecutive integers up to 9,007,199,254,740,992 (&lt;code&gt;2^53&lt;/code&gt;) without losing precision.
You can observe this because if you try adding 1 to this number you will get the same number.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;y &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;9007199254740992&lt;/span&gt;
z &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; y &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
z
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 9007199254740992&lt;/span&gt;
y &lt;span style=&#34;color:#666&#34;&gt;==&lt;/span&gt; z
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] TRUE&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;To store consecutive integers bigger than this you need to use a 64-bit integer type.
R does not have native support for 64-bit integers, however the 
&lt;a href=&#34;https://cran.r-project.org/package=bit64&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;bit64&lt;/a&gt; package provides support for them.
Because these integers are so large, they rarely occur in real world data, however they can often be obtained from generated data, such as database identifiers.&lt;/p&gt;
&lt;p&gt;vroom 1.1.0 now supports reading these big integers into the &lt;code&gt;integer64&lt;/code&gt; type provided by bit64 with a new &lt;code&gt;col_big_integer()&lt;/code&gt; column type (shortcut &amp;lsquo;I&amp;rsquo;).&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vroom&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;id\n9007199254740993\n&amp;#34;&lt;/span&gt;, col_type &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;I&amp;#34;&lt;/span&gt;, delim &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;,&amp;#34;&lt;/span&gt;)
x
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   id              &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;int64&amp;gt;         &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1 9007199254740993&lt;/span&gt;

x&lt;span style=&#34;color:#666&#34;&gt;$&lt;/span&gt;id &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; integer64&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 9007199254740994&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;improved-delimiter-guessing&#34;&gt;Improved delimiter guessing&lt;/h2&gt;
&lt;p&gt;The code to guess delimiters has been rewritten, which should make it more robust to most inputs.
Previous versions of vroom would fall back to using a newline delimiter if a delimiter could not be guessed.
vroom 1.1.0 instead throws an error.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;vroom&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;x\n1\n&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: Could not guess the delimiter.&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Use `vroom(delim =)` to specify one explicitly.&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;vroom&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;x\n1\n&amp;#34;&lt;/span&gt;, delim &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;,&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Rows: 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Columns: 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Delimiter: &amp;#34;,&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dbl [1]: x&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Use `spec()` to retrieve the guessed column specification&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Pass a specification to the `col_types` argument to quiet this message&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 1 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;       x&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1     1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;delimiters-in-the-specification&#34;&gt;Delimiters in the specification&lt;/h2&gt;
&lt;p&gt;vroom now includes the delimiter in the specification object, which means you no longer have to separately provide the delimiter if you are using an existing specification.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# read a csv file, the delimiter is provided as &amp;#39;,&amp;#39;&lt;/span&gt;
x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vroom&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;vroom_example&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;mtcars.csv&amp;#34;&lt;/span&gt;), delim &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;,&amp;#39;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Rows: 32&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Columns: 12&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Delimiter: &amp;#34;,&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; chr [ 1]: model&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; dbl [11]: mpg, cyl, disp, hp, drat, wt, qsec, vs, am, gear, carb&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Use `spec()` to retrieve the guessed column specification&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Pass a specification to the `col_types` argument to quiet this message&lt;/span&gt;
mtcars_spec &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;spec&lt;/span&gt;(x)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# If the file is read again with the spec, no need to provide the delimiter&lt;/span&gt;
y &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;vroom&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;vroom_example&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;mtcars.csv&amp;#34;&lt;/span&gt;), col_types &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; mtcars_spec)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;reading-from-standard-input&#34;&gt;Reading from standard input&lt;/h2&gt;
&lt;p&gt;vroom makes it straightforward to read from the C standard input, like you would do when calling R from the terminal command line.
Simply use &lt;code&gt;stdin()&lt;/code&gt; as your input. Let&amp;rsquo;s say you want to take the first few lines the mtcars file and find the average horsepower.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;head mtcars.tsv | Rscript -e &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#39;hp &amp;lt;- vroom::vroom(stdin(), col_types = list())$hp; mean(hp)&amp;#39;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 122.7778&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;This release also contains a number of bug fixes and improvements which should make it more robust than previous releases. See the 
&lt;a href=&#34;https://vroom.r-lib.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;change log&lt;/a&gt; for full details.&lt;/p&gt;
&lt;p&gt;A big thanks to all contributors of code, issues and documentation to this release, including many who helped out at the tidyverse developer day in Toulouse, France!&lt;/p&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/2005m&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@2005m&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/atomman&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@atomman&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/blairj09&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@blairj09&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Chris-M-P&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Chris-M-P&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/chsafouane&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@chsafouane&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/CriscelyLP&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@CriscelyLP&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DyfanJones&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DyfanJones&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ecoquant&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ecoquant&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/edzer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@edzer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ericbrownaustin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ericbrownaustin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/estroger34&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@estroger34&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/frm1789&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@frm1789&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/georgevbsantiago&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@georgevbsantiago&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/guiastrennec&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@guiastrennec&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/HenrikBengtsson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@HenrikBengtsson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/henry090&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@henry090&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jaapwalhout&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jaapwalhout&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jonaszierer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jonaszierer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kiernann&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kiernann&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/martindut&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@martindut&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/meta00&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@meta00&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mgirlich&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mgirlich&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mllg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@mllg&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/osiris08&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@osiris08&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Plebejer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Plebejer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/R3myG&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@R3myG&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/randomgambit&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@randomgambit&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/sanromd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@sanromd&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Shians&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@Shians&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/stephen-hayne&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@stephen-hayne&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/vjcitn&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@vjcitn&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wlattner&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@wlattner&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/xiaodaigh&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@xiaodaigh&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>ICSE 2019</title>
      <link>/events/icse-2019/</link>
      <pubDate>Tue, 28 May 2019 00:00:00 +0000</pubDate>
      <guid>/events/icse-2019/</guid>
      <description></description>
    </item>
    
    <item>
      <title>Building Tidy Tools</title>
      <link>/events/tidy-tools-2019/</link>
      <pubDate>Wed, 01 May 2019 00:00:00 +0000</pubDate>
      <guid>/events/tidy-tools-2019/</guid>
      <description>&lt;p&gt;The class is taught by Hadley Wickham, Chief Scientist at RStudio.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>stringr 1.4.0</title>
      <link>/blog/2019/02/stringr-1-4-0/</link>
      <pubDate>Thu, 21 Feb 2019 00:00:00 +0000</pubDate>
      <guid>/blog/2019/02/stringr-1-4-0/</guid>
      <description>&lt;html&gt;
&lt;style&gt;
h2 code {
    font-size: 1em;
&lt;p&gt;}
&lt;/style&gt;&lt;/p&gt;
&lt;/html&gt;
&lt;p&gt;We are happy to announce that 
&lt;a href=&#34;http://stringr.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;stringr&lt;/a&gt; 1.4.0
is now on CRAN. stringr provides a cohesive set of functions designed to make
working with strings as easy as possible. For a full list of changes, please see the 
&lt;a href=&#34;https://stringr.tidyverse.org/news/index.html#stringr-1-4-0&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;release notes&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;You can install the released version from CRAN:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;stringr&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(stringr)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;new-functions&#34;&gt;New functions&lt;/h2&gt;
&lt;p&gt;Thanks to the hard work of 
&lt;a href=&#34;https://github.com/jonthegeek&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;John Harmon&lt;/a&gt; at 
&lt;a href=&#34;https://github.com/tidyverse/dev-day-2019&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tidyverse Developer Day&lt;/a&gt;, stringr has three new functions.&lt;/p&gt;
&lt;p&gt;
&lt;a href=&#34;https://stringr.tidyverse.org/reference/str_starts.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_starts()&lt;/code&gt;&lt;/a&gt; and 
&lt;a href=&#34;https://stringr.tidyverse.org/reference/str_starts.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_ends()&lt;/code&gt;&lt;/a&gt; detect the presence or absence of patterns at the beginning or end of strings.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;fruit &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;apple&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;banana&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pear&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;pineapple&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;str_starts&lt;/span&gt;(fruit, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;p&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] FALSE FALSE  TRUE  TRUE&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str_starts&lt;/span&gt;(fruit, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;p&amp;#34;&lt;/span&gt;, negate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1]  TRUE  TRUE FALSE FALSE&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str_ends&lt;/span&gt;(fruit, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;e&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1]  TRUE FALSE FALSE  TRUE&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str_ends&lt;/span&gt;(fruit, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;e&amp;#34;&lt;/span&gt;, negate &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] FALSE  TRUE  TRUE FALSE&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The new 
&lt;a href=&#34;https://stringr.tidyverse.org/reference/case.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_to_sentence()&lt;/code&gt;&lt;/a&gt; function capitalizes strings with sentence case, like so:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;str_to_sentence&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;the quick brown dog&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;The quick brown dog&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;support-for-negate&#34;&gt;Support for &lt;code&gt;negate&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;
&lt;a href=&#34;https://stringr.tidyverse.org/reference/str_subset.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_subset()&lt;/code&gt;&lt;/a&gt;, 
&lt;a href=&#34;https://stringr.tidyverse.org/reference/str_detect.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_detect()&lt;/code&gt;&lt;/a&gt;, and 
&lt;a href=&#34;https://stringr.tidyverse.org/reference/str_subset.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;str_which()&lt;/code&gt;&lt;/a&gt; now have the &lt;code&gt;negate&lt;/code&gt; argument, which is used to find the elements that do &lt;em&gt;not&lt;/em&gt; match a pattern (as seen above in the &lt;code&gt;str_starts()&lt;/code&gt; and &lt;code&gt;str_ends()&lt;/code&gt; examples).&lt;/p&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;Thank you to everyone who contributed to this release: 
&lt;a href=&#34;https://github.com/AmeliaMN&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@AmeliaMN&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/beckymaust&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@beckymaust&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/BenjaminLouis&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@BenjaminLouis&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/blablablerg&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@blablablerg&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bschneidr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@bschneidr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bwiernik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@bwiernik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ctmann&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@ctmann&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/damianooldoni&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@damianooldoni&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dan-reznik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@dan-reznik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/denrou&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@denrou&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/diegogarcilazo&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@diegogarcilazo&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/DieselAnalytics&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@DieselAnalytics&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/elisakreiss&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@elisakreiss&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/giovannikraushaar&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@giovannikraushaar&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hammer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@hammer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jennybc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jennybc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jimhester&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jimhester&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jonocarroll&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jonocarroll&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jonthegeek&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jonthegeek&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jrnold&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jrnold&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/juanrocha&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@juanrocha&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kmace&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@kmace&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/krlmlr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@krlmlr&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/osorensen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@osorensen&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/paleolimbot&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@paleolimbot&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pdelboca&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pdelboca&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/pgrandinetti&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@pgrandinetti&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/PirateGrunt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@PirateGrunt&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/samhinshaw&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@samhinshaw&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/sastoudt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@sastoudt&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/seanpor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@seanpor&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yj-danielyang&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yj-danielyang&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/yutannihilation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@yutannihilation&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>haven 2.1.0</title>
      <link>/blog/2019/02/haven-2-1-0/</link>
      <pubDate>Tue, 19 Feb 2019 00:00:00 +0000</pubDate>
      <guid>/blog/2019/02/haven-2-1-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re delighted to announce that 
&lt;a href=&#34;https://haven.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;haven 2.1.0&lt;/a&gt; is now on CRAN. haven enables R to read and write various data formats used by other statistical packages by wrapping the 
&lt;a href=&#34;https://github.com/WizardMac/ReadStat&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ReadStat&lt;/a&gt; C library written by 
&lt;a href=&#34;https://www.evanmiller.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Evan Miller&lt;/a&gt;. For a full account of updates in this release, see the 
&lt;a href=&#34;https://haven.tidyverse.org/news/index.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Changelog&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;improved-labelling&#34;&gt;Improved labelling&lt;/h2&gt;
&lt;p&gt;Both 
&lt;a href=&#34;https://haven.tidyverse.org/reference/labelled.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;labelled()&lt;/code&gt;&lt;/a&gt; and 
&lt;a href=&#34;https://haven.tidyverse.org/reference/labelled_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;labelled_spss()&lt;/code&gt;&lt;/a&gt; now allow &lt;code&gt;NULL&lt;/code&gt; labels. This makes both classes more flexible, allowing you to use them for their other attributes.&lt;code&gt;labelled()&lt;/code&gt; also now tests that value labels are unique.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;labelled&lt;/code&gt; objects now get pretty printing that shows the labels and &lt;code&gt;NA&lt;/code&gt; values when inside of a &lt;code&gt;tbl_df&lt;/code&gt;. You can turn this behaviour off by using &lt;code&gt;option(haven.show_pillar_labels = FALSE)&lt;/code&gt;.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;tibble&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;tibble&lt;/span&gt;(s &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; haven&lt;span style=&#34;color:#666&#34;&gt;::&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;labelled&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;), labels &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;A&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;B&amp;#34;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; # A tibble: 2 x 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   s        &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   &amp;lt;dbl+lbl&amp;gt;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 1  1       &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; 2 10&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;minor-improvements-and-fixes&#34;&gt;Minor improvements and fixes&lt;/h2&gt;
&lt;p&gt;This release is updated to the latest version of Evan Miller&amp;rsquo;s 
&lt;a href=&#34;https://github.com/WizardMac/ReadStat&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ReadStat&lt;/a&gt;, which includes the following changes:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;read_por()&lt;/code&gt;&lt;/a&gt; can now read files from SPSS 25.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;read_por()&lt;/code&gt;&lt;/a&gt; uses base-10 instead of base-30 for the exponent.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_sas.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;read_sas()&lt;/code&gt;&lt;/a&gt; can read zero-column files.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;read_sav()&lt;/code&gt;&lt;/a&gt; now reads long strings, and has greater memory limit, allowing it to read more labels.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;read_spss()&lt;/code&gt;&lt;/a&gt; reads long variable labels.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;write_sav()&lt;/code&gt;&lt;/a&gt; no longer creates incorrect column names when &amp;gt;10k columns.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://haven.tidyverse.org/reference/read_spss.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;code&gt;write_sav()&lt;/code&gt;&lt;/a&gt; no longer crashes when writing long label names.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;acknowledgements&#34;&gt;Acknowledgements&lt;/h2&gt;
&lt;p&gt;Thank you to Evan Miller, as well as

&lt;a href=&#34;https://github.com/armenic&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@armenic&lt;/a&gt;,  
&lt;a href=&#34;https://github.com/beckerbenj&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@beckerbenj&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/caayala&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@caayala&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gergness&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@gergness&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jeffeaton&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@jeffeaton&lt;/a&gt;,  
&lt;a href=&#34;https://github.com/philstraforelli&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@philstraforelli&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/thays42&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@thays42&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/visseho&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;@visseho&lt;/a&gt; for their contributions to this release.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>purrr 0.3.0</title>
      <link>/blog/2019/02/purrr-0-3-0/</link>
      <pubDate>Wed, 06 Feb 2019 00:00:00 +0000</pubDate>
      <guid>/blog/2019/02/purrr-0-3-0/</guid>
      <description>&lt;p&gt;We&amp;rsquo;re excited to announce the release of 
&lt;a href=&#34;https://purrr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;purrr&lt;/a&gt; 0.3.0! purrr enhances R’s functional programming toolkit by providing a complete and consistent set of tools for working with functions and vectors.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Install the latest version with:&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;install.packages&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;purrr&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Start working with purrr:&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(purrr)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;In this release, &lt;code&gt;pluck()&lt;/code&gt; gets a few new variants:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;pluck&amp;lt;-&lt;/code&gt;, &lt;code&gt;assign_in()&lt;/code&gt; and &lt;code&gt;modify_in()&lt;/code&gt; allow deep modifications of nested structures.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;chuck()&lt;/code&gt; is a stricter variant of &lt;code&gt;pluck()&lt;/code&gt; that consistently fails instead of returning &lt;code&gt;NULL&lt;/code&gt; when the structure does not have the expected shape.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A new set of tools helps limiting the rate at which a function is called:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;slowly()&lt;/code&gt; forces a function to sleep between each invokation.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;insistently()&lt;/code&gt; automatically invokes a function again on error until it succeeds, and sleeps between invokations.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The &lt;code&gt;rate_delay()&lt;/code&gt; and &lt;code&gt;rate_backoff()&lt;/code&gt; helpers control the invokation rate of &lt;code&gt;slowly()&lt;/code&gt; and &lt;code&gt;insistently()&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The reduce and map functions gain a few improvements:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;map_if()&lt;/code&gt; accepts an optional function with the &lt;code&gt;.else&lt;/code&gt; parameter. This function is applied on elements for which the predicate is &lt;code&gt;FALSE&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;map_at()&lt;/code&gt; now accepts &lt;code&gt;vars()&lt;/code&gt; selections. This lets you use selection helpers like &lt;code&gt;dplyr::starts_with()&lt;/code&gt; to determine the elements of a list which should be mapped.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;reduce()&lt;/code&gt; now supports early termination of a computation. Just return a value wrapped in a &lt;code&gt;done()&lt;/code&gt; to signal to &lt;code&gt;reduce()&lt;/code&gt; that you&amp;rsquo;re done.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Besides these new functions and tools, purrr 0.3.0 is mostly a polishing release. We have improved the consistency of behaviour:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;modify()&lt;/code&gt; is now a wrapper around &lt;code&gt;[[&amp;lt;-&lt;/code&gt; instead of &lt;code&gt;[&amp;lt;-&lt;/code&gt;. This makes it compatible with a larger variety of S3 vector classes.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Predicate functions (such that you would pass to &lt;code&gt;map_if()&lt;/code&gt;) now must return a single &lt;code&gt;TRUE&lt;/code&gt; or &lt;code&gt;FALSE&lt;/code&gt;. Missing values and integers are no longer valid predicate outputs.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Finally, we improved the consistency of the interface:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;The direction of iteration/application is now consistently specified with a &lt;code&gt;.dir&lt;/code&gt; argument.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Many missing functions were added to fill the gaps: &lt;code&gt;accumulate2()&lt;/code&gt;, &lt;code&gt;imodify()&lt;/code&gt;, &lt;code&gt;map_depth()&lt;/code&gt;, &amp;hellip;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;code&gt;partial()&lt;/code&gt; has a much improved and more flexible interface.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Find a detailed account of the changes in the 
&lt;a href=&#34;https://github.com/tidyverse/purrr/blob/master/NEWS.md#purrr-030&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;NEWS&lt;/a&gt; file.&lt;/p&gt;
&lt;h2 id=&#34;new-pluck-variants&#34;&gt;New pluck variants&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;pluck()&lt;/code&gt; implements a generalised form of &lt;code&gt;[[&lt;/code&gt; that allow you to index deeply and flexibly into data structures. For instance, &lt;code&gt;pluck(x, &amp;quot;foo&amp;quot;, 2)&lt;/code&gt; is equivalent to &lt;code&gt;x[[&amp;quot;foo&amp;quot;]][[2]]&lt;/code&gt;. You can also supply a default value in case the element does not exist. For instance, &lt;code&gt;pluck(x, &amp;quot;foo&amp;quot;, 2, .default = NA)&lt;/code&gt; is equivalent to &lt;code&gt;x[[&amp;quot;foo&amp;quot;]][[2]]&lt;/code&gt;, returning an &lt;code&gt;NA&lt;/code&gt; if that element doesn&amp;rsquo;t exist. purrr 0.3.0 introduces variants of &lt;code&gt;pluck()&lt;/code&gt; to make it easier to work with deep data structures.&lt;/p&gt;
&lt;h3 id=&#34;pluck-assignment&#34;&gt;Pluck assignment&lt;/h3&gt;
&lt;p&gt;This release introduces the new functions &lt;code&gt;pluck&amp;lt;-&lt;/code&gt;, &lt;code&gt;assign_in()&lt;/code&gt; and &lt;code&gt;modify_in()&lt;/code&gt; as assignment variants of &lt;code&gt;pluck()&lt;/code&gt;. To illustrate deep assignment, let&amp;rsquo;s create a nested data structure:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(foo &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;), bar &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;))
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ foo:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ bar:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This sort of repeated structure is the kind of data where &lt;code&gt;pluck()&lt;/code&gt; shines:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;pluck&lt;/span&gt;(x, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;pluck&lt;/span&gt;(x, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;bar&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You can now use the same syntax to modify the data:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;pluck&lt;/span&gt;(x, &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;) &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ foo:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 100&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ bar:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;pluck&amp;lt;-&lt;/code&gt; also has a functional form that does not modify objects in your environment, but instead returns a modified copy:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;out &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;assign_in&lt;/span&gt;(x, &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;), &lt;span style=&#34;color:#666&#34;&gt;2000&lt;/span&gt;)

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# The object is still the same as before&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(x)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ foo:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 100&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ bar:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 4&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# The modified data is in `out`&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(out)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ foo:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 2000&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ bar:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Finally, &lt;code&gt;modify_in()&lt;/code&gt; is a variant of &lt;code&gt;modify()&lt;/code&gt; that only changes the pluck location with the result of applying a function:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;out &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;modify_in&lt;/span&gt;(x, &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;), as.character)
&lt;span style=&#34;color:#00f&#34;&gt;str&lt;/span&gt;(out)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ foo:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : chr &amp;#34;100&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;  $ bar:List of 2&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 3&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt;   ..$ : num 4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;stricter-pluck&#34;&gt;Stricter pluck()&lt;/h3&gt;
&lt;p&gt;Thanks to Daniel Barnett (@daniel-barnett on Github), &lt;code&gt;pluck()&lt;/code&gt; now has a stricter cousin &lt;code&gt;chuck()&lt;/code&gt;. Whereas &lt;code&gt;pluck()&lt;/code&gt; is very permissive regarding non-existing locations and returns &lt;code&gt;NULL&lt;/code&gt; in these cases, and &lt;code&gt;[[&lt;/code&gt; inconsistently returns &lt;code&gt;NULL&lt;/code&gt;, &lt;code&gt;NA&lt;/code&gt;, or throws an error, &lt;code&gt;chuck()&lt;/code&gt; fails consistently with informative messages (i.e., it &amp;ldquo;chucks&amp;rdquo; an error message):&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;pluck&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;), &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; NULL&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;chuck&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;), &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: Index 1 is attempting to pluck from an unnamed vector using a string name&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;rates&#34;&gt;Rates&lt;/h2&gt;
&lt;p&gt;Thanks to Richie Cotton (@richierocks) and Ian Lyttle (@ijlyttle), purrr gains a function operator to make a function call itself repeatedly when an error occurs.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;counter &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;

f &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt;) {
  &lt;span style=&#34;color:#00f&#34;&gt;if &lt;/span&gt;(counter &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;) {
    counter &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;&amp;lt;-&lt;/span&gt; counter &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;
    &lt;span style=&#34;color:#00f&#34;&gt;stop&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;tilt!&amp;#34;&lt;/span&gt;)
  }
  &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;result&amp;#34;&lt;/span&gt;
}

&lt;span style=&#34;color:#00f&#34;&gt;f&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error in f(): tilt!&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If the function is wrapped with &lt;code&gt;insistently()&lt;/code&gt;, it will try a few times before giving up:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Reset counter&lt;/span&gt;
counter &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;

f2 &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;insistently&lt;/span&gt;(f)
&lt;span style=&#34;color:#00f&#34;&gt;f2&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;result&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Another rate limiting function is &lt;code&gt;slowly()&lt;/code&gt;. While &lt;code&gt;insistently()&lt;/code&gt; loops by itself, &lt;code&gt;slowly()&lt;/code&gt; is designed to be used in your own loops, for instance in a map iteration:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;f &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;function&lt;/span&gt;(&lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt;) &lt;span style=&#34;color:#00f&#34;&gt;print&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;Sys.time&lt;/span&gt;())

&lt;span style=&#34;color:#00f&#34;&gt;walk&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, f)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:03 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:03 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:03 PST&amp;#34;&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;walk&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#00f&#34;&gt;slowly&lt;/span&gt;(f))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:03 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:04 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:05 PST&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;slowly()&lt;/code&gt; uses a constant rate by default while &lt;code&gt;insistently()&lt;/code&gt; uses a backoff rate. The rate limiting can be configured with optional jitter via &lt;code&gt;rate_backoff()&lt;/code&gt; and &lt;code&gt;rate_delay()&lt;/code&gt;, which implement exponential backoff rate and constant rate respectively.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;walk&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;, &lt;span style=&#34;color:#00f&#34;&gt;slowly&lt;/span&gt;(f, &lt;span style=&#34;color:#00f&#34;&gt;rate_backoff&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;, max_times &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;Inf&lt;/span&gt;)))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:05 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:07 PST&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;2019-03-06 12:50:10 PST&amp;#34;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id=&#34;map-and-reduce-improvements&#34;&gt;Map and reduce improvements&lt;/h2&gt;
&lt;h3 id=&#34;map_if-or-else&#34;&gt;&lt;code&gt;map_if()&lt;/code&gt;&amp;hellip; or else?&lt;/h3&gt;
&lt;p&gt;If you like using &lt;code&gt;map_if()&lt;/code&gt;, perhaps you&amp;rsquo;ll find the new &lt;code&gt;.else&lt;/code&gt; argument useful. &lt;code&gt;.else&lt;/code&gt; is a function applied to elements for which the predicate is &lt;code&gt;FALSE&lt;/code&gt;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;map_if&lt;/span&gt;(iris, is.numeric, mean, .else &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; nlevels)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $Sepal.Length&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 5.843333&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $Sepal.Width&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.057333&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $Petal.Length&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3.758&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $Petal.Width&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1.199333&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $Species&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 3&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;new-map_at-features&#34;&gt;New &lt;code&gt;map_at()&lt;/code&gt; features&lt;/h3&gt;
&lt;p&gt;Colin Fay (@ColinFay) has added support for tidyselect expressions to &lt;code&gt;map_at()&lt;/code&gt; and other &lt;code&gt;_at&lt;/code&gt; mappers. This brings the interface of these functions closer to scoped functions from the dplyr package, such as &lt;code&gt;dplyr::mutate_at()&lt;/code&gt;. Note that &lt;code&gt;vars()&lt;/code&gt; is currently not reexported from purrr, so you need to use &lt;code&gt;dplyr::vars()&lt;/code&gt; or &lt;code&gt;ggplot2::vars()&lt;/code&gt; for the time being.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;suppressMessages&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;dplyr&amp;#34;&lt;/span&gt;))

x &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(
  foo &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;,
  bar &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;6&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;,
  baz &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;11&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;15&lt;/span&gt;
)

&lt;span style=&#34;color:#00f&#34;&gt;map_at&lt;/span&gt;(x, &lt;span style=&#34;color:#00f&#34;&gt;vars&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;starts_with&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;b&amp;#34;&lt;/span&gt;)), mean)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $foo&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1 2 3 4 5&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $bar&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 8&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $baz&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 13&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;map_at()&lt;/code&gt; now also supports negative selections:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;map_at&lt;/span&gt;(x, &lt;span style=&#34;color:#666&#34;&gt;-2&lt;/span&gt;, `*`, &lt;span style=&#34;color:#666&#34;&gt;1000&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $foo&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1000 2000 3000 4000 5000&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $bar&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1]  6  7  8  9 10&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $baz&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 11000 12000 13000 14000 15000&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;early-termination-of-reduction&#34;&gt;Early termination of reduction&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;reduce()&lt;/code&gt; is an operation that combines the elements of a vector into a single value by calling a binary function repeatedly with the result so far and the next input of a vector. &lt;code&gt;reduce()&lt;/code&gt; and its variant &lt;code&gt;accumulate()&lt;/code&gt; now support early termination of the reduction. To halt the computation, just return the last value wrapped in a &lt;code&gt;done()&lt;/code&gt; box:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# This computes the total sum of the input vector&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;reduce&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; .x &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; .y)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 5050&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# This stops as soon as the sum is greater than 50&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;reduce&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;100&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;if &lt;/span&gt;(.x &lt;span style=&#34;color:#666&#34;&gt;&amp;gt;&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;50&lt;/span&gt;) &lt;span style=&#34;color:#00f&#34;&gt;done&lt;/span&gt;(.x) else .x &lt;span style=&#34;color:#666&#34;&gt;+&lt;/span&gt; .y)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 55&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This feature takes inspiration from the 
&lt;a href=&#34;https://clojuredocs.org/clojure.core/reduced&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Clojure&lt;/a&gt; language.&lt;/p&gt;
&lt;h2 id=&#34;consistency&#34;&gt;Consistency&lt;/h2&gt;
&lt;p&gt;In this polishing release, a lot of effort went towards consistency of behaviour and of the interface.&lt;/p&gt;
&lt;h3 id=&#34;behaviour&#34;&gt;Behaviour&lt;/h3&gt;
&lt;h4 id=&#34;better-support-for-s3-vectors&#34;&gt;Better support for S3 vectors&lt;/h4&gt;
&lt;p&gt;We are working hard on improving support for S3 vectors in the tidyverse. As of this release, &lt;code&gt;modify()&lt;/code&gt; is now a wrapper around &lt;code&gt;[[&amp;lt;-&lt;/code&gt; instead of &lt;code&gt;[&amp;lt;-&lt;/code&gt;. This should make it directly compatible with a larger set of vector classes. Thanks to the work of Mikko Marttila (@mikmart), &lt;code&gt;pmap()&lt;/code&gt; and &lt;code&gt;pwalk()&lt;/code&gt; also do a better job of preserving S3 classes. Finally, &lt;code&gt;pluck()&lt;/code&gt; now properly calls the &lt;code&gt;[[&lt;/code&gt; methods of S3 objects.&lt;/p&gt;
&lt;p&gt;In the next version of purrr, we plan to use the in-development vctrs package to provide more principled and predictable vector operations. This should help us preserve the class and properties of S3 vectors like factors, dates, or your custom classes.&lt;/p&gt;
&lt;h4 id=&#34;stricter-predicate-checking&#34;&gt;Stricter predicate checking&lt;/h4&gt;
&lt;p&gt;purrr now checks the results of your predicate functions, which must now consistently return &lt;code&gt;TRUE&lt;/code&gt; or &lt;code&gt;FALSE&lt;/code&gt;. We no longer offer support for &lt;code&gt;NA&lt;/code&gt; or for boolish numeric values (R normally interprets 0 as &lt;code&gt;FALSE&lt;/code&gt; and all other values as &lt;code&gt;TRUE&lt;/code&gt;). The purpose of this change is to detect errors earlier with a more relevant error message.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;keep&lt;/span&gt;(&lt;span style=&#34;color:#00f&#34;&gt;c&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;NA&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;), &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; . &lt;span style=&#34;color:#666&#34;&gt;%%&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;==&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; Error: Predicate functions must return a single `TRUE` or `FALSE`, not a missing value&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id=&#34;interface&#34;&gt;Interface&lt;/h3&gt;
&lt;h4 id=&#34;direction-of-application&#34;&gt;Direction of application&lt;/h4&gt;
&lt;p&gt;The direction of application is now specified the same way across purrr functions. &lt;code&gt;reduce()&lt;/code&gt;, &lt;code&gt;compose()&lt;/code&gt; and &lt;code&gt;detect()&lt;/code&gt; now have a &lt;code&gt;.dir&lt;/code&gt; parameter that can take the value &lt;code&gt;&amp;quot;forward&amp;quot;&lt;/code&gt; or &lt;code&gt;&amp;quot;backward&amp;quot;&lt;/code&gt;. This terminology should be less ambiguous than &amp;ldquo;left&amp;rdquo; and &amp;ldquo;right&amp;rdquo;:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#00f&#34;&gt;reduce&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;, `-`, .dir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;backward&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;compose&lt;/span&gt;(foo, bar, .dir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;forward&amp;#34;&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;detect&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;5&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;~&lt;/span&gt; . &lt;span style=&#34;color:#666&#34;&gt;%%&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;==&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;0&lt;/span&gt;, .dir &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;backward&amp;#34;&lt;/span&gt;)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Note that the backward version of &lt;code&gt;reduce()&lt;/code&gt; (called right-reduce in the literature) applies the reduced function in a slightly different way than &lt;code&gt;reduce_right()&lt;/code&gt;. The new algorithm is more consistent with how this operation is usually defined in other languages.&lt;/p&gt;
&lt;p&gt;Following the introduction of the &lt;code&gt;.dir&lt;/code&gt; parameters, the &lt;code&gt;_right&lt;/code&gt; variants such as &lt;code&gt;reduce_right()&lt;/code&gt; have been soft-deprecated, as well as the &lt;code&gt;.right&lt;/code&gt; parameter of &lt;code&gt;detect()&lt;/code&gt; and &lt;code&gt;detect_index()&lt;/code&gt;.&lt;/p&gt;
&lt;h4 id=&#34;partial&#34;&gt;partial()&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;partial()&lt;/code&gt; has been rewritten to be a simple wrapper around &lt;code&gt;call_modify()&lt;/code&gt; and &lt;code&gt;eval_tidy()&lt;/code&gt; from the rlang package. Consequently, the &lt;code&gt;.env&lt;/code&gt;, &lt;code&gt;.lazy&lt;/code&gt; and &lt;code&gt;.first&lt;/code&gt; arguments are soft-deprecated and replaced by a flexible syntax.&lt;/p&gt;
&lt;p&gt;To control the timing of evaluation, unquote the partialised arguments that should be evaluated only once when the function is created. The non-unquoted arguments are evaluated at each invokation of the function:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;my_list &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;partial&lt;/span&gt;(list, lazy &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;rnorm&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;), eager &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;!!&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;rnorm&lt;/span&gt;(&lt;span style=&#34;color:#666&#34;&gt;3&lt;/span&gt;))

&lt;span style=&#34;color:#00f&#34;&gt;my_list&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $lazy&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1]  0.2945451  0.3897943 -1.2080762&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $eager&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] -0.1842525 -1.3713305 -0.5991677&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;my_list&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $lazy&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] -0.3636760 -1.6266727 -0.2564784&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; $eager&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] -0.1842525 -1.3713305 -0.5991677&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You can also control the position of the future arguments by passing an empty &lt;code&gt;... = &lt;/code&gt; parameter. This syntax is powered by &lt;code&gt;rlang::call_modify()&lt;/code&gt; and allows you to add or move dots in a quoted function call. In the case of &lt;code&gt;partial()&lt;/code&gt;, the dots represent the future arguments. We use this syntax in the following snippet to position the future arguments right between two partialised arguments:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;my_list &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#00f&#34;&gt;partial&lt;/span&gt;(list, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;, &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;...&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; , &lt;span style=&#34;color:#666&#34;&gt;2&lt;/span&gt;)

&lt;span style=&#34;color:#00f&#34;&gt;my_list&lt;/span&gt;()
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;my_list&lt;/span&gt;(&lt;span style=&#34;color:#ba2121&#34;&gt;&amp;#34;foo&amp;#34;&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[1]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[2]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] &amp;#34;foo&amp;#34;&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; &lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [[3]]&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id=&#34;exec-replaces-invoke&#34;&gt;&lt;code&gt;exec()&lt;/code&gt; replaces &lt;code&gt;invoke()&lt;/code&gt;&lt;/h4&gt;
&lt;p&gt;We are retiring &lt;code&gt;invoke()&lt;/code&gt; and &lt;code&gt;invoke_map()&lt;/code&gt; in favour of &lt;code&gt;exec()&lt;/code&gt;. Retirement means that we&amp;rsquo;ll keep these functions indefinitely in the package, but we won&amp;rsquo;t add features or recommend using them.&lt;/p&gt;
&lt;p&gt;We are now favouring &lt;code&gt;exec()&lt;/code&gt;, which uses the tidy dots syntax for passing lists of arguments:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-r&#34; data-lang=&#34;r&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# Before:&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;invoke&lt;/span&gt;(mean, &lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(na.rm &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;), x &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 5.5&lt;/span&gt;

&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;# After&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;exec&lt;/span&gt;(mean, &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;10&lt;/span&gt;, &lt;span style=&#34;color:#666&#34;&gt;!!!&lt;/span&gt;&lt;span style=&#34;color:#00f&#34;&gt;list&lt;/span&gt;(na.rm &lt;span style=&#34;color:#666&#34;&gt;=&lt;/span&gt; &lt;span style=&#34;color:#008000;font-weight:bold&#34;&gt;TRUE&lt;/span&gt;))
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 5.5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id=&#34;filling-the-missing-parts&#34;&gt;Filling the missing parts&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;purrr 0.3.0 introduces &lt;code&gt;accumulate2()&lt;/code&gt;, &lt;code&gt;modify2()&lt;/code&gt; and &lt;code&gt;imodify()&lt;/code&gt; variants.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;By popular request, &lt;code&gt;at_depth()&lt;/code&gt; is back as &lt;code&gt;map_depth()&lt;/code&gt;. Unlike &lt;code&gt;modify_depth()&lt;/code&gt; which preserves the class structure of the input tree, this variant only returns trees made of lists of lists (up to the given depth), coercing vectors if needed.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;thanks&#34;&gt;Thanks!&lt;/h2&gt;
&lt;p&gt;Thanks to all the contributors for this release!&lt;/p&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/ArtemSokolov&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠ArtemSokolov&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/batpigandme&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠batpigandme&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/bbrewington&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠bbrewington&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/billdenney&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠billdenney&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cderv&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠cderv&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/cfhammill&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠cfhammill&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ColinFay&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠ColinFay&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/dan-reznik&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠dan-reznik&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/daniel-barnett&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠daniel-barnett&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/danilinares&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠danilinares&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/drtjc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠drtjc&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/egnha&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠egnha&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Eluvias&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠Eluvias&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/flying-sheep&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠flying-sheep&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gergness&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠gergness&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/gvwilson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠gvwilson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hadley&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠hadley&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/hammer&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠hammer&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ijlyttle&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠ijlyttle&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ilarischeinin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠ilarischeinin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/IndrajeetPatil&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠IndrajeetPatil&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/ISPritchin&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠ISPritchin&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jameslairdsmith&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠jameslairdsmith&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jameslamb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠jameslamb&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/jrnold&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠jrnold&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/kcf-jackson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠kcf-jackson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/leungi&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠leungi&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/lionel-&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠lionel-&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/loladze&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠loladze&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/maxheld83&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠maxheld83&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mikmart&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠mikmart&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/MilesMcBain&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠MilesMcBain&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/moodymudskipper&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠moodymudskipper&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/mrstlee&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠mrstlee&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/namelessjon&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠namelessjon&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/r-cheologist&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠r-cheologist&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/randomgambit&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠randomgambit&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/rmflight&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠rmflight&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/roumail&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠roumail&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/Ryo-N7&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠Ryo-N7&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/serina-robinson&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠serina-robinson&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/skaltman&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠skaltman&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/suraggupta&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠suraggupta&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/thays42&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠thays42&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tyluRp&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠tyluRp&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/tzakharko&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠tzakharko&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/VincentGuyader&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠VincentGuyader&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wlandau&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠wlandau&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/wmayner&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠wmayner&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yanxianl&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠yanxianl&lt;/a&gt;, 
&lt;a href=&#34;https://github.com/yutannihilation&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠yutannihilation&lt;/a&gt;, and 
&lt;a href=&#34;https://github.com/yysh12&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;＠yysh12&lt;/a&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>rstudio::conf 2019</title>
      <link>/events/rstudio-conf-2019/</link>
      <pubDate>Tue, 15 Jan 2019 00:00:00 +0000</pubDate>
      <guid>/events/rstudio-conf-2019/</guid>
      <description>&lt;p&gt;We had a great time at the 2019 rstudio::conf! Many attendees arrived early for 2 days of in-depth workshops, followed by two days of conference sessions. This year&amp;rsquo;s conference featured 15 workshops on tidyverse, Shiny, R Markdown, modeling and machine learning, deep learning, big data, and what they forgot to teach you about working with R. Some of the new workshops for this year touched on topics like putting Shiny applications into production at scale and R &amp;amp; Tensorflow. The conference also featured certification workshops on RStudio Professional Administrator and Train-the-trainer for tidyverse and Shiny.&lt;/p&gt;
&lt;p&gt;Below is a list of all workshops we hosted, with links to materials. Even though
the materials alone cannot replace the actual workshop experience, we hope that you&amp;rsquo;ll find them useful. RStudio regularly hosts workshops throughout the year so please subscribe to 
&lt;a href=&#34;https://www.rstudio.com/about/subscription-management/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;training updates&lt;/a&gt;. You can also find out more about each of the workshops
at the 
&lt;a href=&#34;https://github.com/rstudio/rstudio-conf/blob/master/2019/workshops.md&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;conference repository&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Workshop&lt;/th&gt;
&lt;th&gt;Instructor(s)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/AmeliaMN/data-science-in-tidyverse&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Introduction to Data Science in the Tidyverse&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Amelia McNamara, Hadley Wickham&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://rstd.io/tidytools19&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Building Tidy Tools&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Charlotte Wickham, Hadley Wickham&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://rstd.io/wtf-2019-rsc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;What They Forgot to Teach You About R&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Jenny Bryan, Jim Hester&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/dtkaplan/shinymark&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intro to Shiny and RMarkdown&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Danny Kaplan&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://arm.rbind.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Advanced R Markdown&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Alison Hill, Yihui Xie&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/aimeegott/RStudio-Conf-Intermediate-Shiny&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intermediate Shiny&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Aimee Gott, Winston Chang&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/kellobri/spc-app&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Using Shiny in Production&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Kelly O&amp;rsquo;Briant, Sean Lopp&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/topepo/rstudio-conf-2019&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Applied Machine Learning&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Max Kuhn, Alex Hayes, Davis Vaughan&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio/conf_tensorflow_training_day2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Introduction to Deep Learning + Beyond the Basics&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Sigrid Keydana, Kevin Kuo, Rick Scavetta&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio/bigdataclass&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Big Data with R&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Edgar Ruiz, James Blair&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio-education/teaching-workshop-2019-01&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Greg Wilson&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;http://teach-shiny.rbind.io&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Shiny Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Mine Çetinkaya-Rundel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio-education/teach-tidy&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tidyverse Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Garrett Grolemund&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://colorado.rstudio.com/rsc/pro-admin-training/overview/Overview.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RStudio Professional Administrator Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Andrie de Vries&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>rstudio::conf 2018</title>
      <link>/events/rstudio-conf-2018/</link>
      <pubDate>Mon, 15 Jan 2018 00:00:00 +0000</pubDate>
      <guid>/events/rstudio-conf-2018/</guid>
      <description>&lt;p&gt;We had a great time at the 2019 rstudio::conf! Many attendees arrived early for 2 days of in-depth workshops, followed by two days of conference sessions. This year&amp;rsquo;s conference featured 15 workshops on tidyverse, Shiny, R Markdown, modeling and machine learning, deep learning, big data, and what they forgot to teach you about working with R. Some of the new workshops for this year touched on topics like putting Shiny applications into production at scale and R &amp;amp; Tensorflow. The conference also featured certification workshops on RStudio Professional Administrator and Train-the-trainer for tidyverse and Shiny.&lt;/p&gt;
&lt;p&gt;Below is a list of all workshops we hosted, with links to materials. Even though
the materials alone cannot replace the actual workshop experience, we hope that you&amp;rsquo;ll find them useful. RStudio regularly hosts workshops throughout the year so please subscribe to 
&lt;a href=&#34;https://www.rstudio.com/about/subscription-management/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;training updates&lt;/a&gt;. You can also find out more about each of the workshops
at the 
&lt;a href=&#34;https://github.com/rstudio/rstudio-conf/blob/master/2019/workshops.md&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;conference repository&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Workshop&lt;/th&gt;
&lt;th&gt;Instructor(s)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/AmeliaMN/data-science-in-tidyverse&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Introduction to Data Science in the Tidyverse&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Amelia McNamara, Hadley Wickham&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://rstd.io/tidytools19&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Building Tidy Tools&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Charlotte Wickham, Hadley Wickham&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://rstd.io/wtf-2019-rsc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;What They Forgot to Teach You About R&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Jenny Bryan, Jim Hester&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/dtkaplan/shinymark&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intro to Shiny and RMarkdown&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Danny Kaplan&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://arm.rbind.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Advanced R Markdown&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Alison Hill, Yihui Xie&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/aimeegott/RStudio-Conf-Intermediate-Shiny&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Intermediate Shiny&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Aimee Gott, Winston Chang&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/kellobri/spc-app&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Using Shiny in Production&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Kelly O&amp;rsquo;Briant, Sean Lopp&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/topepo/rstudio-conf-2019&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Applied Machine Learning&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Max Kuhn, Alex Hayes, Davis Vaughan&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio/conf_tensorflow_training_day2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Introduction to Deep Learning + Beyond the Basics&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Sigrid Keydana, Kevin Kuo, Rick Scavetta&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio/bigdataclass&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Big Data with R&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Edgar Ruiz, James Blair&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio-education/teaching-workshop-2019-01&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Greg Wilson&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;http://teach-shiny.rbind.io&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Shiny Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Mine Çetinkaya-Rundel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://github.com/rstudio-education/teach-tidy&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tidyverse Train-the-Trainer Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Garrett Grolemund&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;a href=&#34;https://colorado.rstudio.com/rsc/pro-admin-training/overview/Overview.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RStudio Professional Administrator Certification Workshop&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Andrie de Vries&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
</description>
    </item>
    
    <item>
      <title>Education @ RStudio</title>
      <link>/home/band_one/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/home/band_one/</guid>
      <description>&lt;p&gt;Our mission is to equip everyone, regardless of means, to participate in a global economy that rewards data literacy.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Get help!</title>
      <link>/help/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/help/</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;In space, no one can hear you scream.&lt;/p&gt;
&lt;p&gt;&amp;ndash; &lt;cite&gt;Alien (1979)&lt;/cite&gt;&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Luckily the tidyverse is a friendlier place. Ease of adoption and ease of use are fundamental design principles for the packages in the tidyverse. If you are banging your head in frustration, here&amp;rsquo;s how you can help us help you.&lt;/p&gt;
&lt;h2 id=&#34;reprex&#34;&gt;Make a reprex&lt;/h2&gt;
&lt;p&gt;If you need help getting unstuck, the first step is to create a &lt;strong&gt;reprex&lt;/strong&gt;, or reproducible example. The goal of a reprex is to package your problematic code in such a way that other people can run it and feel your pain. Then, hopefully, they can provide a solution and put you out of your misery.&lt;/p&gt;
&lt;p&gt;There are two parts to creating a reprex:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;First, you need to make your code reproducible. This means that you need
to capture everything, i.e., include any &lt;code&gt;library()&lt;/code&gt; calls and create all necessary objects. The easiest way to make sure you&amp;rsquo;ve done this is to use the 
&lt;a href=&#34;/help#reprex-pkg&#34;&gt;reprex package&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Second, you need to make it minimal. Strip away everything that is not directly related to your problem. This usually involves creating a much smaller and simpler R object than the one you&amp;rsquo;re facing in real life or even using built-in data.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;That sounds like a lot of work!  And it can be, but it has a great payoff:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;80% of the time creating an excellent reprex reveals the source of your problem. It&amp;rsquo;s amazing how often the process of writing up a self-contained and minimal example allows you to answer your own question.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The other 20% of time you will have captured the essence of your problem in
a way that is easy for others to play with. This substantially improves
your chances of getting help!&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;reprex-pkg&#34;&gt;The reprex package&lt;/h2&gt;
&lt;p&gt;When creating a reprex by hand, it&amp;rsquo;s easy to accidentally miss something that means your code can&amp;rsquo;t be run on someone else&amp;rsquo;s computer. Avoid this problem by using the 
&lt;a href=&#34;http://reprex.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;reprex package&lt;/a&gt;. It&amp;rsquo;s installed as part of the tidyverse &lt;em&gt;(will be true soon)&lt;/em&gt; or you can install it by itself. Go ahead and load it.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;## pick one:&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;##   install.packages(&amp;#34;tidyverse&amp;#34;) &amp;lt;-- will work soon&lt;/span&gt;
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;##   install.packages(&amp;#34;reprex&amp;#34;)    &amp;lt;-- works today&lt;/span&gt;

&lt;span style=&#34;color:#00f&#34;&gt;library&lt;/span&gt;(reprex)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Write a bit of code and copy it to the clipboard:&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;(y &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
&lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(y)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Enter &lt;code&gt;reprex()&lt;/code&gt; in the R Console. In RStudio, you&amp;rsquo;ll see a preview of your rendered reprex.&lt;/p&gt;
&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-R&#34; data-lang=&#34;R&#34;&gt;(y &lt;span style=&#34;color:#666&#34;&gt;&amp;lt;-&lt;/span&gt; &lt;span style=&#34;color:#666&#34;&gt;1&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;:&lt;/span&gt;&lt;span style=&#34;color:#666&#34;&gt;4&lt;/span&gt;)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 1 2 3 4&lt;/span&gt;
&lt;span style=&#34;color:#00f&#34;&gt;mean&lt;/span&gt;(y)
&lt;span style=&#34;color:#408080;font-style:italic&#34;&gt;#&amp;gt; [1] 2.5&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;It is now ready and waiting on your clipboard, so you can paste it into, say, a GitHub issue. In RStudio, you can access reprex from the 
&lt;a href=&#34;https://rstudio.github.io/rstudioaddins/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;addins menu&lt;/a&gt;, which makes it even easier to point out your code and select the output format.&lt;/p&gt;
&lt;p&gt;&lt;img src=&#34;reprex-addins-menu.png&#34; alt=&#34;reprex addins menu&#34;&gt;&lt;/img&gt;
&lt;img src=&#34;reprex-addin.png&#34; alt=&#34;reprex addin interface&#34;&gt;&lt;/img&gt;&lt;/p&gt;
&lt;p&gt;In either case, you can eventually 
&lt;a href=&#34;http://reprex.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;explore other features&lt;/a&gt;, such as formatting output for Stack Overflow or as a commented R script. reprex even uploads figures so you can easily ask questions about ggplot2.&lt;/p&gt;
&lt;p&gt;If your code is not self-contained, running &lt;code&gt;reprex()&lt;/code&gt; results in an error. It may feel like tough love, but this way you can get your story straight in private. The reprex format also strongly encourages you to find the minimal dataset necessary to show your problem. Creating an effective reprex is a learned skill and the immediate feedback from reprex makes this very concrete.&lt;/p&gt;
&lt;h2 id=&#34;where-to-ask&#34;&gt;Where to ask&lt;/h2&gt;
&lt;p&gt;&lt;img src=&#34;help-is-on-the-way.jpg&#34; alt=&#34;&#34; width=&#34;200&#34; height=&#34;200&#34; align=&#34;right&#34; style=&#34;padding:1em;&#34; /&gt;&lt;/p&gt;
&lt;!-- Thanks to Mark Hansen for the image! https://twitter.com/cocteau/status/893811714420088832 --&gt;
&lt;p&gt;Now that you&amp;rsquo;ve made a reprex that you can easily inflict on others, you need to share it in an appropriate forum. Here are some options:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;http://community.rstudio.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;community.rstudio.com&lt;/strong&gt;&lt;/a&gt;: This is a warm
and welcoming place to ask any questions you might have about the
tidyverse (and you can also ask questions about shiny and RStudio there
too!)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://stackoverflow.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Stack Overflow&lt;/strong&gt;&lt;/a&gt;. You&amp;rsquo;re probably already familiar
with Stack Overflow from googling: it&amp;rsquo;s a frequent source of answers to
coding related questions. Asking a question on Stack Overflow can be
intimidating, but if you&amp;rsquo;ve taken the time to create a reprex, you&amp;rsquo;re much
more likely to get a useful answer. Make sure to 
&lt;a href=&#34;https://stackoverflow.com/help/tagging&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tag your question&lt;/a&gt; with R
and tidyverse so that the right people are more likely to see it.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://twitter.com/search?q=%23rstats&amp;amp;src=typd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;strong&gt;Twitter&lt;/strong&gt;&lt;/a&gt;. It&amp;rsquo;s hard to share your reprex only on twitter, because 140 characters are rarely enough and screenshots don&amp;rsquo;t help others play with your code. But twitter is a great place to share a link to your reprex that&amp;rsquo;s hosted elsewhere. The 
&lt;a href=&#34;https://twitter.com/search?q=%23rstats&amp;amp;src=typd&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;#rstats twitter&lt;/a&gt; community is extremely friendly and active, and is a great crowd to be a part of. Make sure you tag your tweet with #rstats and #tidyverse.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;If you think you&amp;rsquo;ve found a &lt;strong&gt;bug&lt;/strong&gt;, please follow the instructions on

&lt;a href=&#34;/contribute#issues&#34;&gt;contributing to the tidyverse&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>Learn the tidyverse</title>
      <link>/home/band_two/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/home/band_two/</guid>
      <description>&lt;p&gt;See how the tidyverse makes data science faster, easier and more fun with &amp;ldquo;R for Data Science&amp;rdquo;. Read it 
&lt;a href=&#34;http://r4ds.had.co.nz/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;online&lt;/a&gt;, buy 
&lt;a href=&#34;http://amzn.to/2aHLAQ1&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;the book&lt;/a&gt; or try another 
&lt;a href=&#34;/learn/&#34;&gt;resource&lt;/a&gt; from the community.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Need help?</title>
      <link>/home/band_three/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/home/band_three/</guid>
      <description>&lt;p&gt;First learn how to make a 
&lt;a href=&#34;/help/#reprex&#34;&gt;reprex&lt;/a&gt; then 
&lt;a href=&#34;/help/#where-to-ask&#34;&gt;share it&lt;/a&gt; with others.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Tidyverse packages</title>
      <link>/packages/</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>/packages/</guid>
      <description>&lt;h2 id=&#34;installation-and-use&#34;&gt;Installation and use&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Install all the packages in the tidyverse by running &lt;code&gt;install.packages(&amp;quot;tidyverse&amp;quot;)&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run &lt;code&gt;library(tidyverse)&lt;/code&gt; to load the core tidyverse and make it available
in your current R session.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Learn more about the tidyverse package at &lt;a href=&#34;https://tidyverse.tidyverse.org&#34;&gt;https://tidyverse.tidyverse.org&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id=&#34;core-tidyverse&#34;&gt;Core tidyverse&lt;/h2&gt;
&lt;p&gt;The core tidyverse includes the packages that you&amp;rsquo;re likely to use in everyday data analyses. As of tidyverse 1.3.0, the following packages are included in the core tidyverse:&lt;/p&gt;


&lt;div class=&#34;package-section&#34;&gt;
  &lt;div class=&#34;packages&#34;&gt;
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://ggplot2.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;ggplot2&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;ggplot2 is a system for declaratively creating graphics, based on The Grammar of Graphics. You provide the data, tell ggplot2 how to map variables to aesthetics, what graphical primitives to use, and it takes care of the details.&amp;nbsp;&lt;a href=&#34;https://ggplot2.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://dplyr.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;dplyr&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;dplyr provides a grammar of data manipulation, providing a consistent set of verbs that solve the most common data manipulation challenges.&amp;nbsp;&lt;a href=&#34;https://dplyr.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://tidyr.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;tidyr&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;tidyr provides a set of functions that help you get to tidy data. Tidy data is data with a consistent form: in brief, every variable goes in a column, and every column is a variable.&amp;nbsp;&lt;a href=&#34;https://tidyr.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://readr.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;readr&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;readr provides a fast and friendly way to read rectangular data (like csv, tsv, and fwf). It is designed to flexibly parse many types of data found in the wild, while still cleanly failing when data unexpectedly changes. &amp;nbsp;&lt;a href=&#34;https://readr.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://purrr.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;purrr&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;purrr enhances R’s functional programming (FP) toolkit by providing a complete and consistent set of tools for working with functions and vectors. Once you master the basic concepts, purrr allows you to replace many for loops with code that is easier to write and more expressive.&amp;nbsp;&lt;a href=&#34;https://purrr.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://tibble.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;tibble&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;tibble is a modern re-imagining of the data frame, keeping what time has proven to be effective, and throwing out what it has not. Tibbles are data.frames that are lazy and surly: they do less and complain more forcing you to confront problems earlier, typically leading to cleaner, more expressive code.&amp;nbsp;&lt;a href=&#34;https://tibble.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://stringr.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;stringr&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;stringr provides a cohesive set of functions designed to make working with strings as easy as possible. It is built on top of stringi, which uses the ICU C library to provide fast, correct implementations of common string manipulations.&amp;nbsp;&lt;a href=&#34;https://stringr.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  &lt;div class=&#34;package&#34;&gt;
    
    &lt;div class=&#34;package-info&#34;&gt;
      &lt;h3&gt;&lt;a href=&#34;https://forcats.tidyverse.org/&#34; target=&#34;_blank&#34;&gt;forcats&lt;/a&gt;&lt;/h3&gt;
      &lt;p&gt;forcats provides a suite of useful tools that solve common problems with factors. R uses factors to handle categorical variables, variables that have a fixed and known set of possible values.&amp;nbsp;&lt;a href=&#34;https://forcats.tidyverse.org/&#34; aria-hidden=&#34;true&#34;  target=&#34;_blank&#34;&gt;Go to docs...&lt;/a&gt;&lt;/p&gt;
    &lt;/div&gt; 
  &lt;/div&gt; 
  
  
  &lt;/div&gt; 
&lt;/div&gt; 
&lt;p&gt;The tidyverse also includes many other packages with more specialised usage. They are not loaded automatically with &lt;code&gt;library(tidyverse)&lt;/code&gt;, so you&amp;rsquo;ll need to load each one with its own call to &lt;code&gt;library()&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&#34;import&#34;&gt;Import&lt;/h2&gt;
&lt;p&gt;As well as 
&lt;a href=&#34;https://readr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;readr&lt;/a&gt;, for reading flat files, the tidyverse package installs a number of other packages for reading data:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/rstats-db/DBI&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;DBI&lt;/a&gt; for relational databases.
(Maintained by 
&lt;a href=&#34;https://www.cynkra.com&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Kirill Müller&lt;/a&gt;.)
You&amp;rsquo;ll need to pair DBI with a database specific backends like

&lt;a href=&#34;https://rsqlite.r-dbi.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RSQLite&lt;/a&gt;,

&lt;a href=&#34;https://rmariadb.r-dbi.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RMariaDB&lt;/a&gt;,

&lt;a href=&#34;https://rpostgres.r-dbi.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;RPostgres&lt;/a&gt;, or

&lt;a href=&#34;https://github.com/r-dbi/odbc&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;odbc&lt;/a&gt;.
Learn more at &lt;a href=&#34;https://db.rstudio.com&#34;&gt;https://db.rstudio.com&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://haven.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;haven&lt;/a&gt; for SPSS, Stata, and SAS data.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/r-lib/httr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;httr&lt;/a&gt; for web APIs.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://readxl.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;readxl&lt;/a&gt; for &lt;code&gt;.xls&lt;/code&gt; and &lt;code&gt;.xlsx&lt;/code&gt; sheets.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/tidyverse/rvest&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;rvest&lt;/a&gt; for web scraping.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/jeroen/jsonlite#jsonlite&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;jsonlite&lt;/a&gt;
for JSON. (Maintained by 
&lt;a href=&#34;https://github.com/jeroen&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Jeroen Ooms&lt;/a&gt;.)&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/r-lib/xml2&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;xml2&lt;/a&gt; for XML.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;wrangle&#34;&gt;Wrangle&lt;/h2&gt;
&lt;p&gt;In addition to 
&lt;a href=&#34;https://tidyr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidyr&lt;/a&gt;, and 
&lt;a href=&#34;https://dplyr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;dplyr&lt;/a&gt;, there are five packages (including 
&lt;a href=&#34;https://stringr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;stringr&lt;/a&gt; and 
&lt;a href=&#34;https://forcats.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;forcats&lt;/a&gt;) which are designed to work with specific types of data:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;a href=&#34;https://lubridate.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;lubridate&lt;/a&gt; for dates and date-times.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://github.com/tidyverse/hms&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;hms&lt;/a&gt; for time-of-day values.&lt;/li&gt;
&lt;li&gt;
&lt;a href=&#34;https://github.com/tidyverse/blob&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;blob&lt;/a&gt; for storing blob (binary) data.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;program&#34;&gt;Program&lt;/h2&gt;
&lt;p&gt;In addition to 
&lt;a href=&#34;https://purrr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;purrr&lt;/a&gt;, which provides very consistent and natural methods for iterating on R objects, there are two additional tidyverse packages that help with general programming challenges:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://magrittr.tidyverse.org&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;magrittr&lt;/a&gt; provides the pipe, &lt;code&gt;%&amp;gt;%&lt;/code&gt; used
throughout the tidyverse. It also provide a number of more specialised
piping operators (like &lt;code&gt;%$%&lt;/code&gt; and &lt;code&gt;%&amp;lt;&amp;gt;%&lt;/code&gt;) that can be useful in other places.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;
&lt;a href=&#34;https://github.com/tidyverse/glue&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;glue&lt;/a&gt; provides an alternative to
&lt;code&gt;paste()&lt;/code&gt; that makes it easier to combine data and strings.&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&#34;model&#34;&gt;Model&lt;/h2&gt;
&lt;p&gt;Modeling with the tidyverse uses the collection of 
&lt;a href=&#34;https://www.tidymodels.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tidymodels packages&lt;/a&gt;, which largely replace the 
&lt;a href=&#34;https://github.com/tidyverse/modelr&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;modelr&lt;/a&gt; package used in 
&lt;a href=&#34;https://r4ds.had.co.nz/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;R4DS&lt;/a&gt;. These packages provide a comprehensive foundation for creating and using models of all types. Visit the 
&lt;a href=&#34;https://www.tidymodels.org/start/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Getting Started&lt;/em&gt;&lt;/a&gt; guide or, for more detailed examples, go straight to the 
&lt;a href=&#34;https://www.tidymodels.org/learn/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;&lt;em&gt;Learn&lt;/em&gt;&lt;/a&gt; page.&lt;/p&gt;
&lt;h2 id=&#34;get-help&#34;&gt;Get help&lt;/h2&gt;
&lt;p&gt;If you’re asking for R help, reporting a bug, or requesting a new feature, you’re more likely to succeed if you include a good reproducible example, which is precisely what the 
&lt;a href=&#34;https://reprex.tidyverse.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;reprex&lt;/a&gt; package is meant for. You can learn more about reprex, along with other tips on how to help others help you in the 
&lt;a href=&#34;https://www.tidyverse.org/help/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;help section&lt;/a&gt;.&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>