Skip to content

Commit

Permalink
portal launch
Browse files Browse the repository at this point in the history
  • Loading branch information
@arunp77
arunp77 committed Nov 29, 2023
1 parent afafd1c commit 77b0f1e
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 18 deletions.
96 changes: 85 additions & 11 deletions Linear-reg.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -206,11 +206,10 @@ <h2 id="Types-of-Linear-Regression">Types of Linear Regression<a class="anchor-l
</ol>

<h5>Mathematical Explanation:</h5>
<p>There are parameters <code>β<sub>0</sub></code>, <code>β<sub>1</sub></code>, and <code>σ<sup>2</sup></code>, such that for any fixed value of the independent variable $x$, the dependent variable is a random variable related to $x$ through the model equation:</p>
$$y=\beta_0 + \beta_1 x +\epsilon$$


<p>There are parameters <code>β<sub>0</sub></code>, <code>β<sub>1</sub></code>, and <code>σ<sup>2</sup></code>, such that for any fixed value of the independent variable <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math>, the dependent variable is a random variable related to <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math> through the model equation:</p>

$$y=\beta_0 + \beta_1 x +\epsilon$$

<p>where</p>
<ul>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"><mi>y</mi></math> = Dependent Variable (Target Variable)</li>
Expand All @@ -222,19 +221,94 @@ <h5>Mathematical Explanation:</h5>
<p>The goal of linear regression is to estimate the values of the regression coefficients</p>
<img src="assets/img/data-engineering/Multi-lin-reg.png" alt="" style="max-width: 60%; max-height: 60%;">
<p>This algorithm explains the linear relationship between the dependent(output) variable <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>y</mi></math>
and the independent(predictor) variable <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>x</mi></math> using a straight line <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> <mo>=</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo>+</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mi>x</mi> </math></p>
and the independent(predictor) variable <math xmlns="http://www.w3.org/1998/Math/MathML"><mi>x</mi></math> using a straight line
<math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> <mo>=</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo>+</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mi>x</mi> </math></p>


<h4 id="1.2.-Goal">1.2. Goal<a class="anchor-link" href="#1.2.-Goal">&#182;</a></h4>

<ul>
<li>The goal of the linear regression algorithm is to get the best values for $\beta_0$ and $\beta_1$ to find the best fit line. </li>
<li>The goal of the linear regression algorithm is to get the best values for <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> </math>
and <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> </math> to find the best fit line. </li>
<li>The best fit line is a line that has the least error which means the error between predicted values and actual values should be minimum.</li>
<li><p>For a datset with $n$ observation $(x_i, y_i)$, where $i=1,2,3...., n$ the above function can be written as follows</p>
<p>$y_i=\beta_0 + \beta_1 x_i +\epsilon_i$</p>
<p>where $y_i$ is the value of the observation of the dependent variable (outcome variable) in the smaple, $x_i$ is the value of $ith$ observation
of the independent variable or feature in the sample, $\epsilon_i$ is the random error (also known as residuals) in predicting the value of $y_i$,
$\beta_0$ and $\beta_i$ are the regression parameters (or regression coefficients or feature weights).</p>
<li><p>For a datset with <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>n</mi> </math> observation <math xmlns="http://www.w3.org/1998/Math/MathML"> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo stretchy="false">)</mo> </math>,
where <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>i</mi> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mn>2</mn> <mo>,</mo> <mn>3....</mn> <mo>,</mo> <mi>n</mi> </math> the above function can be written as follows</p>

<p><math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>=</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo>+</mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>+</mo> <msub> <mi>&#x03F5;<!-- ϵ --></mi> <mi>i</mi> </msub> </math></p>

<p>where <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>y</mi> <mi>i</mi> </msub> </math> is the value of the observation of the dependent variable (outcome variable) in the smaple, <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>x</mi> <mi>i</mi> </msub> </math> is the value of <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>i</mi> <mi>t</mi> <mi>h</mi> </math> observation
of the independent variable or feature in the sample, <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03F5;<!-- ϵ --></mi> <mi>i</mi> </msub> </math> is the random error (also known as residuals) in predicting the value of <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>y</mi> <mi>i</mi> </msub> </math>,
<math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> </math> and <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>i</mn> </msub> </math> are the regression parameters (or regression coefficients or feature weights).</p>
</li>
</ul>

<p><strong>Note:</strong></p>
<ul>
<li><p>The quantity ϵ in the model equation is the “error” -- a random variable, assumed to be symmetrically distributed with</p>
<p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>E</mi> <mo stretchy="false">(</mo> <mi>&#x03F5;<!-- ϵ --></mi> <mo stretchy="false">)</mo> <mo>=</mo> <mn>0</mn> <mtext>&#xA0;</mtext> <mtext>&#xA0;</mtext> <mrow class="MJX-TeXAtom-ORD"> <mi mathvariant="normal">a</mi> <mi mathvariant="normal">n</mi> <mi mathvariant="normal">d</mi> </mrow> <mtext>&#xA0;</mtext> <mtext>&#xA0;</mtext> <mi>V</mi> <mo stretchy="false">(</mo> <mi>&#x03F5;<!-- ϵ --></mi> <mo stretchy="false">)</mo> <mo>=</mo> <msub> <mi>&#x03C3;<!-- σ --></mi> <mrow class="MJX-TeXAtom-ORD"> <msup> <mi>Y<!-- Y --></mi> <mn>2</mn> </msup> </mrow> </msub> <mo>=</mo> <msup> <mi>&#x03C3;<!-- σ --></mi> <mn>2</mn> </msup> </math></p>
<p>It is to be noted here that there are no assumption made about the distribution of ϵ, yet.</p>
<ul>
<li>The <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> </math> (the intercept of the true regression line) parameter is average value of Y when x is zero.</li>
<li>The <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> </math> (the slope of the true regression line): The expected (average) change in Y associated with a 1-unit increase in the value of x.</li>
<li>What is <math xmlns="http://www.w3.org/1998/Math/MathML"> <msubsup> <mi>&#x03C3;<!-- σ --></mi> <mrow class="MJX-TeXAtom-ORD"> <mi>Y</mi> </mrow> <mn>2</mn> </msubsup> </math>?: is a measure of how much the values of Y spread out about the mean value (homogeneity of variance assumption).</li>
</ul>
</li>
</ul>


<h4 id="1.3.-Calculating-the-regression-parameters">1.3. Calculating the regression parameters<a class="anchor-link" href="#1.3.-Calculating-the-regression-parameters">&#182;</a></h4><p>In simple linear regression, there is only one independent variable (<math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math>) and one dependent variable (<math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> </math>). The parameters (coefficients) in simple linear regression can be calculated using the method of <strong>ordinary least squares (OLS)</strong>. The equations and formulas involved in calculating the parameters are as follows:</p>
<p><strong>Model Representation:</strong></p>
<p>The simple linear regression model can be represented as:
$$y = \beta_0 + \beta_1 x + \epsilon$$</p>
<p>Therefore, we can write:</p>
<p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>&#x03F5;<!-- ϵ --></mi> <mo>=</mo> <mi>y</mi> <mo>&#x2212;<!-- − --></mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo>&#x2212;<!-- − --></mo> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mi>x</mi> </math>.</p>

<ol>
<li><p><strong>Cost Function or mean squared error (MSE):</strong></p>
<p>The MSE, measures the average squared difference between the predicted values (<math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>y</mi> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math>) and the actual values of the dependent variable (<math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> </math>). It is given by:</p>
<p>$$MSE = \frac{1}{n} \sum (y_i - \hat{y}_i)^2$$</p>
<p>Where:</p>
<ul>
<li>$n$ is the number of data points.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>y</mi> <mi>i</mi> </msub> </math> is the actual value of the dependent variable for the i-th data point.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>y</mi> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> <mi>i</mi> </msub> </math> is the predicted value of the dependent variable for the i-th data point.</li>
</ul>
</li>

<li><p><strong>Minimization of the Cost Function:</strong></p>
<p>The parameters <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> </math> and <math xmlns="http://www.w3.org/1998/Math/MathML"> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> </math> are estimated by minimizing the cost function. The formulas for calculating the parameter estimates are derived from the derivative of the cost function with respect to each parameter.</p>
<p>The parameter estimates are given by:</p>
<ul>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> <mo>=</mo> <mfrac> <mrow> <mtext>Cov</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo stretchy="false">)</mo> </mrow> <mrow> <mi>V</mi> <mi>a</mi> <mi>r</mi> <mo stretchy="false">(</mo> <mi>x</mi> <mo stretchy="false">)</mo> </mrow> </mfrac> </math><math xmlns="http://www.w3.org/1998/Math/MathML"> <mo stretchy="false">&#x21D2;<!-- ⇒ --></mo> <menclose notation="box"> <mrow class="MJX-TeXAtom-ORD"> <mstyle displaystyle="true" scriptlevel="0"> <mrow class="MJX-TeXAtom-ORD"> <msub> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>&#x03B2;<!-- β --></mi> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> <mn>1</mn> </msub> <mo>=</mo> <mfrac> <mrow> <mo>&#x2211;<!-- ∑ --></mo> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>&#x2212;<!-- − --></mo> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>x</mi> <mo stretchy="false">&#x00AF;<!-- ¯ --></mo> </mover> </mrow> <mo stretchy="false">)</mo> <mo stretchy="false">(</mo> <msub> <mi>y</mi> <mi>i</mi> </msub> <mo>&#x2212;<!-- − --></mo> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>y</mi> <mo stretchy="false">&#x00AF;<!-- ¯ --></mo> </mover> </mrow> <mo stretchy="false">)</mo> </mrow> <mrow> <mo>&#x2211;<!-- ∑ --></mo> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>&#x2212;<!-- − --></mo> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>x</mi> <mo stretchy="false">&#x00AF;<!-- ¯ --></mo> </mover> </mrow> <msup> <mo stretchy="false">)</mo> <mn>2</mn> </msup> </mrow> </mfrac> </mrow> </mstyle> </mrow> </menclose> </math>$$</li>
<li><p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> <mo>=</mo> <mtext>y</mtext> <mo>&#x2212;<!-- − --></mo> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> <mo>&#x00D7;<!-- × --></mo> <mtext>mean</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo stretchy="false">)</mo> </math></p></li>
<p>Where:</p>
<ul>
<li><p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> is the estimated <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> </math>-intercept.</p>
</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> is the estimated slope.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mtext>Cov</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo stretchy="false">)</mo> </math> is the covariance between <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math> and <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> </math>.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mtext>Var</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo stretchy="false">)</mo> </math> is the variance of <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math>.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mtext>mean</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo stretchy="false">)</mo> </math> is the mean of <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math>.</li>
<li><p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mtext>mean</mtext> <mo stretchy="false">(</mo> <mi>x</mi> <mo stretchy="false">)</mo> </math> is the mean of <math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>y</mi> </math>.</p>
</ul>
</li>
</ul>
<p>The estimated parameters <math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> and <math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> provide the values of the intercept and slope that best fit the data according to the simple linear regression model.</p>
</li>
<li><p><strong>Prediction:</strong></p>
<p>Once the parameter estimates are obtained, predictions can be made using the equation:</p>
<p>$$\hat{y} = \hat{\beta_0} + \hat{\beta_1} x$$</p>
<p>Where:</p>
<ul>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <mi>y</mi> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> is the predicted value of the dependent variable.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>0</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> is the estimated y-intercept.</li>
<li><math xmlns="http://www.w3.org/1998/Math/MathML"> <mrow class="MJX-TeXAtom-ORD"> <mover> <msub> <mi>&#x03B2;<!-- β --></mi> <mn>1</mn> </msub> <mo stretchy="false">&#x005E;<!-- ^ --></mo> </mover> </mrow> </math> is the estimated slope.</li>
<li><p><math xmlns="http://www.w3.org/1998/Math/MathML"> <mi>x</mi> </math> is the value of the independent variable for which the prediction is being made.</p>
</li>
</ul>
</li>
<p>These equations and formulas allow for the calculation of the parameters in simple linear regression using the method of <strong>ordinary least squares (OLS)</strong>. By minimizing the sum of squared differences between predicted and actual values, the parameters are determined to best fit the data and enable prediction of the dependent variable.</p>
</ol>



Expand Down
7 changes: 0 additions & 7 deletions supervised.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -228,13 +228,6 @@ <h4 id="common-sla">Common Supervised Learning Algorithms:</h4>
that works best for a particular problem.</p>
</section>


<figure style="text-align: center;">
<img src="assets/img/remote-sensing/swath.png" alt="" style="max-width: 50%; max-height: 50%;">
<figcaption style="text-align: center;">Swaths (<strong>Image credit:</strong>
<a href="https://natural-resources.canada.ca/maps-tools-and-publications/satellite-imagery-and-air-photos/tutorial-fundamentals-remote-sensing/satellites-and-sensors/satellite-characteristics-orbits-and-swaths/9283" target="_blank">
© Remote Sensing Tutorials (Natural Resources Canada).</a>)</figcaption>
</figure>

<!-------Reference ------->
<section id="reference">
Expand Down

0 comments on commit 77b0f1e

Please sign in to comment.