where
- = Dependent Variable (Target Variable) @@ -222,19 +221,94 @@
- The goal of the linear regression algorithm is to get the best values for $\beta_0$ and $\beta_1$ to find the best fit line. +
- The goal of the linear regression algorithm is to get the best values for + and to find the best fit line.
- The best fit line is a line that has the least error which means the error between predicted values and actual values should be minimum. -
For a datset with $n$ observation $(x_i, y_i)$, where $i=1,2,3...., n$ the above function can be written as follows
-$y_i=\beta_0 + \beta_1 x_i +\epsilon_i$
-where $y_i$ is the value of the observation of the dependent variable (outcome variable) in the smaple, $x_i$ is the value of $ith$ observation - of the independent variable or feature in the sample, $\epsilon_i$ is the random error (also known as residuals) in predicting the value of $y_i$, - $\beta_0$ and $\beta_i$ are the regression parameters (or regression coefficients or feature weights).
+For a datset with observation , + where the above function can be written as follows
+ + + +where is the value of the observation of the dependent variable (outcome variable) in the smaple, is the value of observation + of the independent variable or feature in the sample, is the random error (also known as residuals) in predicting the value of , + and are the regression parameters (or regression coefficients or feature weights).
The quantity ϵ in the model equation is the “error” -- a random variable, assumed to be symmetrically distributed with
+ +It is to be noted here that there are no assumption made about the distribution of ϵ, yet.
+-
+
- The (the intercept of the true regression line) parameter is average value of Y when x is zero. +
- The (the slope of the true regression line): The expected (average) change in Y associated with a 1-unit increase in the value of x. +
- What is ?: is a measure of how much the values of Y spread out about the mean value (homogeneity of variance assumption). +
+ Cost Function or mean squared error (MSE):
+The MSE, measures the average squared difference between the predicted values () and the actual values of the dependent variable (). It is given by:
+$$MSE = \frac{1}{n} \sum (y_i - \hat{y}_i)^2$$
+Where:
+-
+
- $n$ is the number of data points. +
- is the actual value of the dependent variable for the i-th data point. +
- is the predicted value of the dependent variable for the i-th data point. +
+
+ Minimization of the Cost Function:
+The parameters and are estimated by minimizing the cost function. The formulas for calculating the parameter estimates are derived from the derivative of the cost function with respect to each parameter.
+The parameter estimates are given by:
+-
+
- $$ +
- +
is the estimated -intercept.
+
+ - is the estimated slope. +
- is the covariance between and . +
- is the variance of . +
- is the mean of . +
is the mean of .
+
Where:
+-
+
+
Mathematical Explanation:
The goal of linear regression is to estimate the values of the regression coefficients
This algorithm explains the linear relationship between the dependent(output) variable - and the independent(predictor) variable using a straight line
+ and the independent(predictor) variable using a straight line + + +1.2. Goal¶
-
-
Note:
+-
+
1.3. Calculating the regression parameters¶
In simple linear regression, there is only one independent variable () and one dependent variable (). The parameters (coefficients) in simple linear regression can be calculated using the method of ordinary least squares (OLS). The equations and formulas involved in calculating the parameters are as follows:
+Model Representation:
+The simple linear regression model can be represented as: + $$y = \beta_0 + \beta_1 x + \epsilon$$
+Therefore, we can write:
+.
+ +-
+
The estimated parameters and provide the values of the intercept and slope that best fit the data according to the simple linear regression model.
+ +Prediction:
+Once the parameter estimates are obtained, predictions can be made using the equation:
+$$\hat{y} = \hat{\beta_0} + \hat{\beta_1} x$$
+Where:
+-
+
- is the predicted value of the dependent variable. +
- is the estimated y-intercept. +
- is the estimated slope. +
is the value of the independent variable for which the prediction is being made.
+
+
These equations and formulas allow for the calculation of the parameters in simple linear regression using the method of ordinary least squares (OLS). By minimizing the sum of squared differences between predicted and actual values, the parameters are determined to best fit the data and enable prediction of the dependent variable.
+ diff --git a/supervised.html b/supervised.html index 640d87e..493a056 100644 --- a/supervised.html +++ b/supervised.html @@ -228,13 +228,6 @@Common Supervised Learning Algorithms:
that works best for a particular problem. - -
-