Regularized Linear Models
Ridge regression and the Lasso are two forms of regularized regression (i.e., to constraint the model) which are typically achieved by constraining the weights of the model. These methods are...
Maximum Likelihood Estimation and Maximum A Posteriori Estimation
Maximum Likelihood Estimation (MLE) Consider $n$ i.i.d. random variables $x_{1}, x_{2}, …, x_{n}$. $x_{i}$ is a sample from density estimation $P(x_{i} \mid \theta)$. Then, we can write the likelihood function...
Generative and Discriminative models
In this article we discuss basic differences between Generative and Discriminative models. Actually, so far, we have already seen couple of those models, which Logistic Regression is one of the...
Decision Trees
Decision tree is a hierarchical data structure that represents data through a divide and conquer strategy. They have a natural “if … then … else …” construction. It is a...
Ensemble Learning in extensive details with examples in Scikit-Learn
Bootstrap Method This statistical technique consists in generating samples (called bootstrap samples) from an initial dataset of size N by randomly drawing with replacement (meaning we can select the same...