Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CON-2377 linear-regression #2397

Merged
merged 1 commit into from
Jan 24, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions subjects/ai/linear-regression/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,15 +119,15 @@ X, y, coef = make_regression(n_samples=100,

![alt text][q1]

[q1]: ./w2_day1_ex2_q1.png "Scatter plot"
[q1]: ./w2_day1_ex2_q1.png 'Scatter plot'

2. Fit a LinearRegression from Scikit-learn on the generated data and give the equation of the fitted line. The expected output is: `y = coef * x + intercept`

3. Add the fitted line to the plot. The plot should look like this:

![alt text][q3]

[q3]: ./w2_day1_ex2_q3.png "Scatter plot + fitted line"
[q3]: ./w2_day1_ex2_q3.png 'Scatter plot + fitted line'

4. Predict on X.

Expand Down Expand Up @@ -222,7 +222,7 @@ _Warning: The shape of X is not the same as the shape of y. You may need (for so

![alt text][ex5q1]

[ex5q1]: ./w2_day1_ex5_q1.png "Scatter plot "
[ex5q1]: ./w2_day1_ex5_q1.png 'Scatter plot '

As a reminder, fitting a Linear Regression on this data means finding (a, b) that fits well the data points.

Expand Down Expand Up @@ -304,22 +304,22 @@ The expected output is:

![alt text][ex5q5]

[ex5q5]: ./w2_day1_ex5_q5.png "MSE "
[ex5q5]: ./w2_day1_ex5_q5.png 'MSE '

6. From the `losses` list, find the optimal value of a and b and plot the line in the scatter point of question 1.

In this example we computed 160 000 times the MSE. It is frequent to deal with 50 features, which requires 51 parameters to fit the Linear Regression. If we try this approach with 50 features we would need to compute **5.07e+132** MSE. Even if we reduce the scope and try only 5 values per coefficients we would have to compute the MSE **4.4409e+35** times. This approach is not scalable and that is why is not used to find optimal coefficients for Linear Regression.

### Gradient Descent

In a nutshell, Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we use gradient descent to update the parameters (a and b) of our model. Parameters refer to the coefficients used in Linear Regression. Before to start implementing the questions, take the time to read the article. https://jairiidriss.medium.com/gradient-descent-algorithm-from-scratch-using-python-2b36c1548917. It explains the gradient descent and how to implement it. The "tricky" part is the computation of the derivative of the mse. You can admit the formulas of the derivatives to implement the gradient descent (`d_theta_0` and `d_theta_1` in the article).
In a nutshell, Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient. In machine learning, we use gradient descent to update the parameters (a and b) of our model. Parameters refer to the coefficients used in Linear Regression. Before to start implementing the questions, take the time to read [this article](https://medium.com/@yennhi95zz/4-a-beginners-guide-to-gradient-descent-in-machine-learning-773ba7cd3dfe). It explains the gradient descent and how to implement it. The "tricky" part is the computation of the derivative of the mse. You can admit the formulas of the derivatives to implement the gradient descent (`d_theta_0` and `d_theta_1` in the article).

7. Implement the gradient descent to find optimal a and b with `learning rate = 0.1` and `nbr_iterations=100`.

8. Save the a and b through the iterations in a two-dimensional numpy array. Add them to the plot of the previous part and observe a and b that converge towards the minimum. The plot should look like this:

![alt text][ex5q8]

[ex5q8]: ./w2_day1_ex5_q8.png "MSE + Gradient descent"
[ex5q8]: ./w2_day1_ex5_q8.png 'MSE + Gradient descent'

9. Use Linear Regression from Scikit-learn. Compare the results.
Loading