forked from DragonflyStats/Coursera-ML
-
Notifications
You must be signed in to change notification settings - Fork 0
/
MLQuiz11Q2SystemDesign.tex
28 lines (18 loc) · 1.84 KB
/
MLQuiz11Q2SystemDesign.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
\documentclass[]{article}
\begin{document}
\section*{Machine Learning System Design}
Suppose a massive dataset is available for training a learning algorithm. Training on a lot of data is likely to give good performance when two of the following conditions hold true. Which are the two?
\begin{itemize}
\item[(i)] We train a learning algorithm with a small number of parameters (that is thus unlikely to overfit).
\textbf{No} - If the model has a small number of parameters, then it will underfit the large training set and not make good use of all the data.
%------------------------%
\item[(ii)] We train a learning algorithm with a large number of parameters (that is able to learn/represent fairly complex functions).
\textbf{Yes} - You should use a "\textit{\textbf{low bias}}" algorithm with many parameters, as it will be able to make use of the large dataset provided. If the model has too few parameters, it will underfit the large training set.
%------------------------%
\item[(iii)] The features x contain sufficient information to predict y accurately. (For example, one way to verify this is if a human expert on the domain can confidently predict y when given only x).
\textbf{Yes} - It is important that the features contain sufficient information, as otherwise no amount of data can solve a learning problem in which the features do not contain enough information to make an accurate prediction.
%------------------------%
\item[(iv)]When we are willing to include high order polynomial features of x (such as $x^2_1$, $x^2_2$, $x_1x_2$, etc.).
\textbf{No} - As we saw with neural networks, polynomial features can still be insufficient to capture the complexity of the data, especially if the features are very high-dimensional. Instead, you should use a complex model with many parameters to fit to the large training set.
\end{itemize}
\end{document}