After using logistic regression for binomial classification on news data [Part 1, Part 2], I wanted to explore the possibility of logistic regression in case of multiclass classification. Hence, I decided to use Iris Flower Data Set available in Kaggle which has three distinct classes for output variable. Kaggle is a great place for predictive modeling or data mining enthusiasts since one can get access to various kinds of data within its data realm; there are small to medium to even large sets of data, which one can use either for practice or to involve in the competitions out there or for the purpose of learning. I am using the flower data set for the purpose of gaining and sharing knowledge.
Although logistic regression is originally developed for binomial classification where the output variable is dichotomous in nature, it can be used in scenario where the output variable is trichotomous or polychotomous (with more than two discrete values) as well. There are schemes such as multinomial and one –versus-rest that can be used to generalize logistic regression to make it suitable for multi-class classification.
This git post is related to the one-versus-rest scheme of using logistic regression for multi-class classification.
In the one-versus-rest (ovr) strategy of using logistic regression for multi-class classification, a binary logistic regression model is fitted for each class/output label. This means, if we have output variable that can have three possible values A, B and C, we will have three logistic regressions fitted: (i) A vs [B, C], (ii) B vs [A, C], and (iii) C vs [A, B]. So, the number of logistic regression models fitted will be equal to the number of classes. When testing a test data x, its membership score from each fitted logistic regression is obtained and the class with highest score will be considered the destination class for the test data, since the score is actually the probability of falling into that class given the values of x.
Understanding multi-class classification will be easier if we try out an example implementation. Hence, this git post.
In order to better understand the implementation and visualizations, follow the article: here.