This repository has been archived by the owner on Aug 7, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathREADME.Rmd
193 lines (143 loc) · 10.1 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
<!-- [](https://codecov.io/github/alexioannides/pipeliner) -->
[](https://travis-ci.org/AlexIoannides/pipeliner) [](https://ci.appveyor.com/project/AlexIoannides/pipeliner)
[](https://cran.r-project.org/package=pipeliner) [](https://github.com/metacran/cranlogs.app)
# Machine Learning Pipelines for R
Building machine learning and statistical models often requires pre- and post-transformation of the input and/or response variables, prior to training (or fitting) the models. For example, a model may require training on the logarithm of the response and input variables. As a consequence, fitting and then generating predictions from these models requires repeated application of transformation and inverse-transformation functions - to go from the domain of the original input variables to the domain of the original output variables (via the model). This is usually quite a laborious and repetitive process that leads to messy code and notebooks.
The `pipeliner` package aims to provide an elegant solution to these issues by implementing a common interface and workflow with which it is possible to:
- define transformation and inverse-transformation functions;
- fit a model on training data; and then,
- generate a prediction (or model-scoring) function that automatically applies the entire pipeline of transformations and inverse-transformations to the inputs and outputs of the inner-model and its predicted values (or scores).
The idea of pipelines is inspired by the machine learning pipelines implemented in [Apache Spark's MLib library][spark_pipes] (which are in-turn inspired by Python's scikit-Learn package). This package is still in its infancy and the latest development version can be downloaded from [this GitHub repository][github_pipeliner] using the `devtools` package (bundled with RStudio),
```{r, eval=FALSE, include=TRUE}
devtools::install_github("alexioannides/pipeliner")
```
## Pipes in the Pipleline
There are currently four types of pipeline section - a section being a function that wraps a user-defined function - that can be assembled into a pipeline:
- `transform_features`: wraps a function that maps input variables (or features) to another space - e.g.,
```{r, eval=FALSE, include=TRUE}
transform_features(function(df) {
data.frame(x1 = log(df$var1))
})
```
- `transform_response`: wraps a function that maps the response variable to another space - e.g.,
```{r, eval=FALSE, include=TRUE}
transform_response(function(df) {
data.frame(y = log(df$response))
})
```
- `estimate_model`: wraps a function that defines how to estimate a model from training data in a data.frame - e.g.,
```{r, eval=FALSE, include=TRUE}
estimate_model(function(df) {
lm(y ~ 1 + x1, df)
})
```
- `inv_transform_features(f)`: wraps a function that is the inverse to `transform_response`, such that we can map from the space of inner-model predictions to the one of output domain predictions - e.g.,
```{r, eval=FALSE, include=TRUE}
inv_transform_response(function(df) {
data.frame(pred_response = exp(df$pred_y))
})
```
As demonstrated above, each one of these functions expects as its argument another unary function of a data.frame (i.e. it has to be a function of a single data.frame). With the **exception** of `estimate_model`, which expects the input function to return an object that has a `predict.object-class-name` method existing in the current environment (e.g. `predict.lm` for linear models built using `lm()`), all the other transform functions also expect their input functions to return data.frames (consisting entirely of columns **not** present in the input data.frame). If any of these rules are violated then appropriately named errors will be thrown to help you locate the issue.
If this sounds complex and convoluted then I encourage you to to skip to the examples below - this framework is **very** simple to use in practice. Simplicity is the key aim here.
## Two Interfaces to Rule Them All
I am a great believer and protagonist for functional programming - especially for data-related tasks like building machine learning models. At the same time the notion of a 'machine learning pipeline' is well represented with a simple object-oriented class hierarchy (which is how it is implemented in [Apache Spark's][spark_pipes]). I couldn't decide which style of interface was best, so I implemented both within `pipeliner` (using the same underlying code) and ensured their output can be used interchangeably. To keep this introduction simple, however, I'm only going to talk about the functional interface - those interested in the (more) object-oriented approach are encouraged to read the manual pages for the `ml_pipeline_builder` 'class'.
### Example Usage with a Functional Flavour
We use the `faithful` dataset shipped with R, together with the `pipeliner` package to estimate a linear regression model for the eruption duration of 'Old Faithful' as a function of the inter-eruption waiting time. The transformations we apply to the input and response variables - before we estimate the model - are simple scaling by the mean and standard deviation (i.e. mapping the variables to z-scores).
The end-to-end process for building the pipeline, estimating the model and generating in-sample predictions (that include all interim variable transformations), is as follows,
```{r}
library(pipeliner)
data <- faithful
lm_pipeline <- pipeline(
data,
transform_features(function(df) {
data.frame(x1 = (df$waiting - mean(df$waiting)) / sd(df$waiting))
}),
transform_response(function(df) {
data.frame(y = (df$eruptions - mean(df$eruptions)) / sd(df$eruptions))
}),
estimate_model(function(df) {
lm(y ~ 1 + x1, df)
}),
inv_transform_response(function(df) {
data.frame(pred_eruptions = df$pred_model * sd(df$eruptions) + mean(df$eruptions))
})
)
in_sample_predictions <- predict(lm_pipeline, data, verbose = TRUE)
head(in_sample_predictions)
```
### Accessing Inner Models & Prediction Functions
We can access the estimated inner models directly and compute summaries, etc - for example,
```{r}
summary(lm_pipeline$inner_model)
```
Pipeline prediction functions can also be accessed directly in a similar way - for example,
```{r}
pred_function <- lm_pipeline$predict
predictions <- pred_function(data, verbose = FALSE)
head(predictions)
```
## Turbo-Charged Pipelines in the Tidyverse
The `pipeliner` approach to building models becomes even more concise when combined with the set of packages in the [tidyverse][the_tidyverse]. For example, the 'Old Faithful' pipeline could be rewritten as,
```{r, warning=FALSE, message=FALSE}
library(tidyverse)
lm_pipeline <- data %>%
pipeline(
transform_features(function(df) {
transmute(df, x1 = (waiting - mean(waiting)) / sd(waiting))
}),
transform_response(function(df) {
transmute(df, y = (eruptions - mean(eruptions)) / sd(eruptions))
}),
estimate_model(function(df) {
lm(y ~ 1 + x1, df)
}),
inv_transform_response(function(df) {
transmute(df, pred_eruptions = pred_model * sd(eruptions) + mean(eruptions))
})
)
head(predict(lm_pipeline, data))
```
Nice, compact and expressive (if I don't say so myself)!
### Compact Cross-validation
If we now introduce the `modelr` package into this workflow and adopt the the list-columns pattern described in Hadley Wickham's [R for Data Science][r4ds], we can also achieve wonderfully compact end-to-end model estimation and cross-validation,
```{r}
library(modelr)
# define a function that estimates a machine learning pipeline on a single fold of the data
pipeline_func <- function(df) {
pipeline(
df,
transform_features(function(df) {
transmute(df, x1 = (waiting - mean(waiting)) / sd(waiting))
}),
transform_response(function(df) {
transmute(df, y = (eruptions - mean(eruptions)) / sd(eruptions))
}),
estimate_model(function(df) {
lm(y ~ 1 + x1, df)
}),
inv_transform_response(function(df) {
transmute(df, pred_eruptions = pred_model * sd(eruptions) + mean(eruptions))
})
)
}
# 5-fold cross-validation using machine learning pipelines
cv_rmse <- crossv_kfold(data, 5) %>%
mutate(model = map(train, ~ pipeline_func(as.data.frame(.x))),
predictions = map2(model, test, ~ predict(.x, as.data.frame(.y))),
residuals = map2(predictions, test, ~ .x - as.data.frame(.y)$eruptions),
rmse = map_dbl(residuals, ~ sqrt(mean(.x ^ 2)))) %>%
summarise(mean_rmse = mean(rmse), sd_rmse = sd(rmse))
cv_rmse
```
# Forthcoming Attractions
I built `pipeliner` largely to fill a hole in my own workflows. Up until now I've used Max Kuhn's excellent [caret package][caret] quite a bit, but for in-the-moment model building (e.g. within a R Notebook) it wasn't simplifying the code *that* much, and the style doesn't quite fit with the tidy and functional world that I now inhabit most of the time. So, I plugged the hole by myself. I intend to live with `pipeliner` for a while to get an idea of where it might go next, but I am always open to suggestions (and bug notifications) - please [leave any ideas here][github_pipeliner_issues].
[spark_pipes]: http://spark.apache.org/docs/latest/ml-pipeline.html "Pipelines in Apache Spark MLib"
[github_pipeliner]: https://github.com/AlexIoannides/pipeliner "Pipeliner on GitHub"
[the_tidyverse]: http://tidyverse.org "Welcome to The Tidyverse!"
[r4ds]: http://r4ds.had.co.nz/many-models.html#list-columns-1 "R 4 Data Science - Many Models & List Columns"
[caret]: http://topepo.github.io/caret/index.html "Caret"
[github_pipeliner_issues]: https://github.com/AlexIoannides/pipeliner/issues "Pipeliner Issues on GitHub"