![](\"img/k-nn_RMSE.jpeg\")
"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "deletable": false,
+ "nbgrader": {
+ "cell_type": "code",
+ "checksum": "67e086cc04ad4264b642781e233c7cd7",
+ "grade": false,
+ "grade_id": "cell-8dc7cc6f35de80cb",
+ "locked": false,
+ "schema_version": 3,
+ "solution": true
+ }
+ },
+ "outputs": [],
+ "source": [
+ "# your code here\n",
+ "fail() # No Answer - remove if you provide an answer\n",
+ "answer0.2"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "deletable": false,
+ "editable": false,
+ "nbgrader": {
+ "cell_type": "code",
+ "checksum": "cd327e469d66107c2a4ca6fc92b97eba",
+ "grade": true,
+ "grade_id": "cell-d90b3e7b890ce86b",
+ "locked": true,
+ "points": 1,
+ "schema_version": 3,
+ "solution": false
+ }
+ },
+ "outputs": [],
+ "source": [
+ "test_0.2()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "deletable": false,
+ "editable": false,
+ "nbgrader": {
+ "cell_type": "markdown",
+ "checksum": "5ae59861171794c4f66b5fc614187d8e",
+ "grade": false,
+ "grade_id": "cell-73d7913554df0781",
+ "locked": true,
+ "schema_version": 3,
+ "solution": false,
+ "task": false
+ }
+ },
+ "source": [
+ "### RMSPE Definition\n",
+ "\n",
+ "**Question 0.3** Multiple Choice: \n",
+ "
\n",
+ "\n",
+ "Source: https://media.giphy.com/media/nUN6InE2CodRm/giphy.gif\n",
+ "\n",
+ "What predicts which athletes will perform better than others? Specifically, we are interested in marathon runners, and looking at how the maximum distance ran per week (in miles) during race training predicts the time it takes a runner to finish the race? For this, we will be looking at the `marathon.csv` file in the `data/` folder."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "deletable": false,
+ "editable": false,
+ "nbgrader": {
+ "cell_type": "markdown",
+ "checksum": "9ea30c1434a67e3876226cd543d43d4c",
+ "grade": false,
+ "grade_id": "cell-16d7d432c49c9e53",
+ "locked": true,
+ "schema_version": 3,
+ "solution": false
+ }
+ },
+ "source": [
+ "**Question 1.0** \n",
+ "
+Source: Artwork by @allison_horst
+
+
+Housekeeping¶
-
+
The upcoming tutorial is a project work session
+
+Reminder about projects:
+-
+
- Part of the group project mark is "teamwork". All members will have the opportunity to evaluate your group members at the end of the year and the evaluations will affect your final project mark +
+
Regression prediction problem¶
What if we want to predict a quantitative value instead of a class label?
+
E.g.: predict the price of a 2000 square foot home (from this reduced dataset)
+
K nearest neighbours regression¶
As in k-nn classification, we find the $k$-nearest neighbours (here $k=5$) in terms of the predictors
+
K nearest neighbours regression¶
Then we average the values for the $k$-nearest neighbours, and use that as the prediction:
+
K nearest neighbours regression¶
If we do that for a range of house sizes, we can draw the curve of predictions:
+
Regression prediction problem¶
We still have to answer these two questions:
+-
+
Is our model any good?
+
+How do we choose
+k?
+
The same general strategy as in classification works here!
+
1. Is our model any good?¶
The blue line depicts our predictions from k-nn regression.
+
1. Is our model any good?¶
The blue line depicts our predictions from k-nn regression. The red lines depict the error in our predictions, i.e., the difference between the $i^\text{th}$ test data response and our prediction $\hat{y}_i$:
+
+1. Is our model any good?¶
We (roughly) add up these errors to evaluate our regression model
+-
+
- Not out of 1, but instead in units of the target variable (bit harder to interpret) +
| + |
Root Mean Squared Prediction Error (RMSPE):
+$$RMSPE = \sqrt{\frac{1}{n}\sum\limits_{i=1}^{n}(y_i - \hat{y_i})^2}$$
+- $n$ is the number of observations
+- $y_i$ is the observed value for the $i^\text{th}$ test observation
+- $\hat{y_i}$ is the predicted value for the $i^\text{th}$ test observation
+RMSE vs RMSPE¶
-
+
- Root Mean Squared Error (RMSE): The same formula except on the data used to train the model
-
+
- this indicates how well our model can fit our data +
+ - Root Mean Squared Prediction Error (RMSPE): predicting on unseen data.
-
+
- this indicates how well our model generalizes to future data +
+
No standard terminology.
+2. How do we choose k?¶
+Roughly the same as before!
+-
+
- Cross validation:
-
+
- Split data into $C$ folds +
- Train +
- Evaluate model +
- Pick k that gives the lowest RMSPE on validation set +
+
-
+
- Train model on whole training dataset (not split into folds) +
-
+
- Evaluate how good the predictions are using the test data +
How does $k$ affect k-nn regression?¶
Discussion: Which of the following is overfitting? Underfitting?¶
Worksheet time - go for it!¶
-
+
- Remember work together with your groups - help one another if you can! +
- If you get stuck, we will be walking around to answer questions +
- Please reserve your proposal questions and ask during office hours today +
What did you learn?¶
-
+
+
+
+