Akaike Information Criterion

Used to compare which model is better. AIC rewards goodness of fit (as assessed by the likelihood function), but it also includes a penalty that is an increasing function of the number of estimated parameters. The penalty discourages overfitting, which is desired because increasing the number of parameters in the model almost always improves the goodness of the fit.

Definition

Let $k$ be the number of estimated parameters in the model. Let $\hat{L}$ be the maximum value of the Likelihood Function. Then, the Akaike information criterion (AIC) value of the model is the given by the formula

$$\text{AIC}=2k-2\ln (\hat{L})$$

How it is Used

Given a set of candidate models for the data, the preferred model is the one with the minimum AIC value. This formulated by

$$\begin{array}{rl}\text{find }& f\\ \text{s.t. }& \underset{k,l}{\min }2k-2\ln (\hat{L})\end{array}$$

This implies that the AIC is used model selection performance metric of regression models.