06_tidyr.Rmd

---
title: "06_tidyr"
output: html_document
---

class: middle, center, inverse
layout: false

# 4.4 `tidyr`:<br><br>Tidy Messy Data

---

background-image: url(https://raw.githubusercontent.com/tidyverse/tidyr/master/man/figures/logo.png)
background-position: 97.5% 2.5%
background-size: 7.5%
layout: true

---

## 4.4 `tidyr`: Tidy Messy Data

`tidyr` provides several functions that help you bring your data into the *tidy data* format (e.g., reshaping data, splitting columns, handling missing values or nesting data).

```{r}
penguins
```

???
- Let's again start with our `penguins` data set which already is in *tidy data* format
- in the following i highlight the dimensionality of the data to show you what happens

DIM: 344 x 8

---

## 4.4 `tidyr`: Tidy Messy Data

**Pivotting:** Converts between long and wide format using `pivot_longer()` and `pivot_wider()`.

.panelset[
.panel[.panel-name[pivot_longer()]
```{r}
long_penguins <- penguins %>% 
  pivot_longer(
    cols = c(species, island),
    names_to = "variable", values_to = "value"
  )

long_penguins %>% glimpse
```
]
.panel[.panel-name[pivot_wider()]
```{r}
long_penguins %>% 
  pivot_wider(
    names_from = "variable", values_from = "value"
  ) %>%
  glimpse
```
]
]

???
`pivot_longer()`:
- now for each observation we have two rows, one row per variable that are pivotted -> no tidy format any longer
- DIM: 688 x 8

`pivot_wider()`
- invert `pivot_longer()`
- DIM: 344 x 8

---

## 4.4 `tidyr`: Tidy Messy Data

.right[
```{r, echo=F, out.height='80%', out.width='80%'}
knitr::include_graphics("https://raw.githubusercontent.com/apreshill/teachthat/master/pivot/pivot_longer_smaller.gif")
```
]

.footnote[.pull-left[
*Source: [Allison Hill](https://github.com/apreshill/teachthat/blob/master/pivot/pivot_longer_smaller.gif)*

<i>Note: Find more information about `pivot_*()` in the [pivoting vignette](https://tidyr.tidyverse.org/articles/pivot.html).</i>
]]

---

name: tidyr_nest

## 4.4 `tidyr`: Tidy Messy Data

**Nesting:** Groups similar data such that each group becomes a single row in a data frame.

```{r}
nested_penguins <- 
  penguins %>% 
    nest(nested_data = c(island, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, sex))

nested_penguins
```

???
- note that `nest()` produces a nested data frame with one row per species and year
- note that the `nested_data` column contains `tibbles` with six columns each and a varying amount of observations
- the work with nested data can be particularly helpful if you would like to apply functions to each subset of the data (e.g., fit a model for each year or for each species)

---

name: nested-data

## 4.4 `tidyr`: Tidy Messy Data

**Rectangling:** Disentangles nested data structures (e.g., JSON, HTML) and brings it into *tidy data* format.

.panelset[
.panel[.panel-name[pluck()]
Extract individual objects from a nested data structure via `purrr::pluck()`.
```{r}
nested_penguins %>% purrr::pluck("nested_data", 1)
```
]
.panel[.panel-name[unnest()]
Flatten nested data structures via `tidyr::unnest()`.
```{r}
nested_penguins %>% unnest(cols = c(nested_data)) 
```
]
.panel[.panel-name[hoist()]
Selectively extract individual components from an object in a nested data structure via `tidyr::hoist()`.
```{r}
nested_penguins %>% hoist(nested_data, hoisted_col = "bill_length_mm")
```
]
]

???
Alternatively use `unnest_wider()` or `unnest_longer()` for more control over the rectangling operation.

---

## 4.4 `tidyr`: Tidy Messy Data

**Splitting** and **Combining:** Transforms a single character column into multiple columns and vice versa.

.panelset[
.panel[.panel-name[unite()]
Collapse multiple columns into a single column.
```{r}
penguins %>% unite(col = "species_gender", c(species, sex), sep = "_", remove = T)
```
]
.panel[.panel-name[separate()]
Separate a single column, containing multiple values, into multiple columns.
```{r}
penguins %>% separate(bill_length_mm, sep = 2, into = c("cm", "mm"))
```
]
.panel[.panel-name[separate_rows()]
Separate a single column, containing multiple values, into multiple rows.
```{r}
penguins %>% separate_rows(island, sep = "s", convert = T)
```
]
]

???
can also `separate` based on character match

---

## 4.4 `tidyr`: Tidy Messy Data

**Handling missing values:** Drop or replace explicit or implicit missing values (`NA`).

```{r, echo=F}
incompl_penguins <- tibble(
  species = c(rep("Adelie", 2), rep("Gentoo", 1), rep("Chinstrap", 1)),
  year = c(2007, 2008, 2008, 2007),
  measurement = c(rnorm(3, mean = 50, sd = 15), NA)
)
```

.panelset[
.panel[.panel-name[Base Case]
```{r}
incompl_penguins
```
]
.panel[.panel-name[complete()]
Make implicit missing values explicit.
```{r}
incompl_penguins %>% 
  complete(species, year, fill = list(measurement = NA))
```
.pull-right[.pull-right[.footnote[
]]]
]
.panel[.panel-name[drop_na()]
Make explicit missing values implicit.
```{r}
incompl_penguins %>% 
  drop_na(measurement)
```
]
.panel[.panel-name[fill()]
Replace missing values with the next/previous value.
```{r}
incompl_penguins %>% 
  fill(measurement, .direction = "down")
```
]
.panel[.panel-name[replace_na()]
Replace missing values with a pre-defined value.
```{r}
incompl_penguins %>%
  replace_na(replace = list(measurement = mean(.$measurement, na.rm = T)))
```
]
]

.footnote[
*Note: Find more information and functions on the `tidyr` [cheat sheet](https://raw.githubusercontent.com/rstudio/cheatsheets/master/data-import.pdf).*
]

???
Note: function arguments preceded by a dot in the tidyverse may have one of two reasons:
- the function is still pre-mature, i.e. developers still think about the best way of implementing and naming the function
- the function is regularly applied within another function so that you don't confuse function arguments between the inner and outer function