Multiclass Classification - One-vs-Rest / One-vs-One
Although many classification problems can be defined using two classes (they are inherently multi-class classifiers), some are defined with more than two classes which requires adaptations of machine learning algorithm....
Generating random variables
Transformation of Random Variables Let’s consider how to take the transformation of a random variable $X$ with cumulative distribution function $F_{X}(x)$. Let $Y=t(X)$, that is, $Y$ is the transformation of...
How to plot ROC curve and compute AUC by hand
Assume we have a probabilistic, binary classifier such as logistic regression. Before presenting the ROC curve (Receiver Operating Characteristic curve), the concept of confusion matrix must be understood. When we...
an Unbiased Estimator and its proof
Unbiasness is one of the properties of an estimator in Statistics. If the following holds, where $\hat{\theta}$ is the estimate of the true population parameter $\theta$: \[E(\hat{\theta}) = \theta\] then...
Logistic Regression / Odds / Odds Ratio / Risk
An important distinction between linear and logistic regression is that the regression coefficients in logistic regression are not directly meaningful. In linear regression, a coefficient $\theta_{j} = 1$ means that...