ML (Machine Learning) code in Python
- This repo is for my personal study of some ML (machine learning) conceps and to study sklearn and tensorflow (keras) libraries
- Jupyter Notebooks have some markdown cells which contain explanations in LaTeX which are sometimes rendered incorrectly by GitHub. So, it is sometimes better to study locally
- Example notebooks start with: /examples/check_py_env.ipynb
- Following examples are linked to the beginning cell of the former notebook
- All code and all examples are prone to all kinds of errors
- Any corrections, suggestions, improvements, etc. are welcome
- /examples/check_py_env.ipynb
- Learn about Python, system, GPU on the local computer
- Learn about the specific library versions
- /examples/time_series/generate_data.ipynb
- Write complex equations in markdown cells
- Multi-dimensional numpy array operations
- Random number generation, saving, and various other numpy functions
- sns.pairplot in seaborn
- /examples/time_series/cases
- 1: None compare with 2: Within and 12: Inter
- 2: Within compare with 1: None and 12: Inter
- 3: Dense Shuffled compare with 2: LSTM Shuffled, 7: LSTM Not Shuffled, and 17: Dense Not Shuffled
- 4: Small Batch compare with 2: Medium and 5: Large
- 5: Large Batch compare with 4: Small and 2: Medium
- 6: Large Dataset compare with 2: Small
- 7: LSTM Not Shuffled compare with 2: LSTM Shuffled, 3: Dense Shuffled, and 17: Dense Not Shuffled
- 8: Coupled All Sensors compare with 2: Uncoupled All, 9: Uncoupled Less, and 11: Coupled Less
- 9: Uncoupled Less Sensors compare with 2: Uncoupled All, 8: Coupled All, and 11: Coupled Less
- 10: Single Sensor Used compare with 2: All and 9: 2 Sensors
- 11: Coupled Less Sensors compare with 2: Uncoupled All, 8: Coupled All, and 9: Uncoupled Less
- 12: Inter compare with 1: None and 2: Within
- 13: Not Shuffle compare with 12: Shuffle
- 14: Stateful LSTM compare with 13: Not
- 15: States Manually Reset compare with 14: Not
- 16: Large Dataset compare with 15: Small
- 17: Dense Not Shuffled compare with 2: LSTM Shuffled, 3: Dense Shuffled, and 7: LSTM Not Shuffled
- /examples/time_series/functions.py
- Splitting dataset into training, validation, and test sets using from sklearn.model_selection import train_test_split
- Regression performance metrics using from sklearn.metrics import mean_squared_error, mean_absolute_error
- Sequential models in Keras using from keras.models import Sequential
- Saving and loading Keras models using from keras.models import load_model
- Following ANN layers: from keras.layers import Input, Flatten, LSTM, Dropout, Dense
- Early stopping the training using from keras.callbacks import EarlyStopping
- Customized callback creation and usage from keras.callbacks import Callback
- sns.regplot in seaborn
- sns.histplot in seaborn
- Drawing learning curves using pandas and matplotlib
- /examples/time_series/misc.ipynb
- Python from enum import Enum usage
- Classification performance metrics using from sklearn.metrics import classification_report, confusion_matrix
- Multi-class classification using from keras.utils import np_utils
- Custom loss function and custom metric definition using from keras import backend as ker
- Following ANN layers: from keras.layers import Conv1D, MaxPooling1D, concatenate, BatchNormalization, Bidirectional
- Functional API in Keras used with from keras.models import Model
- Deleting Python variables with del variable
- percentiles = np.percentile(y_train_temp_1, [25, 50, 75])
- Regression - Binary classification - Multi-class classification differences
- Getting input-output information about an intermediate layer
- /examples/toy_datasets.ipynb
- seaborn toy datasets
- scikit-learn toy datasets
- A multi-class classification dataset from sklearn.datasets import load_iris and seaborn
- Usage of help function in Python
- /examples/datasets_misc.ipynb
- Creating a simulated dataset easily with from sklearn.datasets import make_blobs
- Scaling input data using from sklearn.preprocessing import MinMaxScaler, StandardScaler
- Shuffle a dataset using from sklearn.utils import shuffle
- /examples/shallow/random_forests.ipynb
- Usage of a shallow model with from sklearn.ensemble import RandomForestClassifier
- Calling iris.info(), iris.head(), iris.tail() methods in pandas
- /examples/shallow/pca_and_svm.ipynb
- Dimensionality reduction using from sklearn.decomposition import PCA
- SVM models with from sklearn.svm import SVC
- /examples/shallow/grid_search_and_knn.ipynb
- KNN classifier with from sklearn.neighbors import KNeighborsClassifier
- Model parameter optimization with from sklearn.model_selection import GridSearchCV
- A binary classification dataset from sklearn.datasets import load_breast_cancer
- /examples/tensorboard.ipynb
- A regression dataset from sklearn.datasets import load_diabetes
- Current date and time with from datetime import datetime
- Obtain network insight with from keras.callbacks import TensorBoard
- /examples/keras_applications/resnet50.ipynb
- Download and use ResNet50 model with from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions
- Preprocess any image to feed to ResNet50 with from tensorflow.keras.preprocessing.image import load_img, img_to_array
- /examples/keras_applications/intermediate_layers.ipynb
- Download and use MobileNetV2 model with from tensorflow.keras.applications.mobilenet_v2 import MobileNetV2, preprocess_input, decode_predictions
- Reach intermediate layer input-output information of a pre-trained model
- /main/examples/keras_applications/transfer_learning.ipynb
- Download and use InceptionV3 model with from keras.applications.inception_v3 import InceptionV3, preprocess_input
- Augment an image dataset with from tensorflow.keras.preprocessing.image import ImageDataGenerator
- Clone a model with from keras.models import clone_model
- Display an image in the middle of a cell with from IPython.display import Image, display
- Splitting dataset into training and test sets using from sklearn.model_selection import train_test_split
- Following ANN layer: from keras.layers import GlobalAveragePooling2D
- Control the optimization algorithm with from keras.optimizers import SGD
- Transfer learning by stacking additional layers on top of the existing pre-trained model
- Fine tuning the tip of the existing model by playing with layer.trainable
- /examples/nlp/intro.ipynb
- Sequence to sequence model from scratch
- Following ANN layers: from keras.layers import Embedding, GRU
- NLP loss function from keras.losses import sparse_categorical_crossentropy
- String processing to simplfy the learning task
- Training-testing split using dataset = tf.data.Dataset.from_tensor_slices(encoded)
- Have seperate test model to have a batch size of 1
- /examples/autoencoders/intro.ipynb
- Intro to MNIST dataset
- What autoencoders are useful for: denoising and dimensionality reduction for visualization
- What autoencoders are NOT useful for: unsupervised learning of useful representations without the need for labels
- /examples/autoencoders/sparse.ipynb
- Use of regularization with from keras import regularizers
- Effect of L1 regularizer
- /examples/autoencoders/convnet.ipynb
- Following ANN layers: Conv2D, MaxPooling2D, UpSampling2D
- /examples/autoencoders/denoising.ipynb
- Denoising an image using autoencoders
- /examples/autoencoders/vae.ipynb
- Variational autoencoders
- Following ANN layer: Lambda
- add_loss function of a from keras.models import Model
- Dimensionality reduction for data visualization (in 2D)
- /examples/gan/intro.ipynb
- Following ANN layers: Reshape, Conv2DTranspose, LeakyReLU
- Training with a for loop instead of keras.model.fit
- Two phase training by varying trainable
- Generating fake digit images with GAN
- /examples/autoencoders/lstm.ipynb
- Following ANN layers: RepeatVector, TimeDistributed
- Sequence encoding
- Sequence prediction
- /examples/time_series/autoencoder.ipynb
- RootMeanSquaredError
- GRU autoencoder feature extractor combined with a binary classification problem (underperformed compared to classical method)
- /examples/nlp/bidirectional.ipynb
- Wrong minima (loss drops significantly but the output seems to be thrash)
- /examples/nlp/attention.ipynb
- Wrong minima (loss drops significantly but the output seems to be thrash)
- Study and demonstrate working on AWS basics
- Solve the problem in /examples/time_series/cases/16.ipynb. Why the network cannot learn the inter-window relations? The reason might be related to the formation of batches. Try to use dataset = tf.data.Dataset.from_tensor_slices(input)
- Solve the problem in /examples/nlp/bidirectional.ipynb. Why the bidirectional network performs worse than /examples/nlp/intro.ipynb?
- Solve the problem in /examples/nlp/attention.ipynb. Why the attention network performs worse than /examples/nlp/intro.ipynb?
Written by serhatsoyer