机器学习（周志华）

Machine Learning (Author: Prof. Zhou Zhihua)

This repository contains my personal notes and codes for the book of Machine Learning written by Prof. Zhou Zhihua.

Supervised Learning

1. Linear Model from Stratch
   • Logistic Regresssion from Stratch
   • Linear Discrimination Analysis (LDA) from Stratch
2. Decision Tree from Stratch
   • Iterative Dichotomiser 3 (ID3) Decision Tree
   • CART Decision Tree with Pre- and Post- Pruning
   • Decision Tree Implementation with Non-Recursive Approach
3. Neural Networks from Stratch
   • Standard and Accumulated Back-Propagation
4. Support Vector Machine from Stratch
5. Bayes Classifier from Stratch
   • Naïve Bayes Classifier using Laplacian Correction
   • Averaged One-Dependent Estimator (AODE) of Semi-Naïve Bayes Classifier
6. Ensemble Learning from Stratch
   • AdaBoost
   • Bagging of Decision Stamps
7. Dimension Deduction from Stratch
   • K Nearest Neighbors (KNN)
8. Feature Selection from Stratch
   • Relief Algorithm

Unsupervised Learning

9. Clustering from Stratch
   • K-Means Algorithm
   • Automatic K-Means Algorithm

Example Codes in Jupyter Notebook for Book Exercise

Chapter 3. Linear Model

• Problem 3.3 - Implementation of Logistic Regression
• Problem 3.5 - Implementation of Linear Discrimination Analysis (LDA)

Chapter 4. Decision Tree

• Problem 4.3 - Implementation of Iterative Dichotomiser 3 (ID3) Decision Tree
• Problem 4.4 - Implementation of CART Decision Tree with Pruning
• Problem 4.7 - Decision Tree Implementation with Non-Recursive Approach

Chapter 5. Neural Networks

• Problem 5.5 - Implementation of Standard and Accumulated Back-Propagation

Chapter 6. Support Vector Machine

• Problem 6.2 - Implementation of Support Vector Machine using Kernels

Chapter 7. Bayes Classifier

• Problem 7.3 - Implementation of Naïve Bayes Classifier using Laplacian Correction
• Problem 7.6 - Implementation of Averaged One-Dependent Estimator (AODE) of Semi-Naïve Bayes Classifier

Chapter 8. Ensemble Learning

• Problem 8.3- Implementation of AdaBoost
• Problem 8.5 - Implementation of Bagging for Decision Stamps

Chapter 9. Clustering

• Problem 9.4 - Implementation of K-Means Algorithm
• Problem 9.10 - Implementation of K-Mean with Automatically K Selection

Chapter 10. Dimension Deduction

• Problem 10.1 - Implementation of K Nearest Neighbors Algorithm (KNN)

Chapter 11. Feature Selection

• Problem 11.1 - Implementation of Relief Algorithm for Feature-Selection in Binary Classification Problem