|
|
Multivariate Normal DistributionIn probability theory and statistics, a multivariate normal distribution, also sometimes called a multivariate Gaussian distribution (in honor of Carl Friedrich Gauss, who was not the first to write about the normal distribution) is a specific probability density function. General case A random vector follows a multivariate normal distribution, also sometimes called a multivariate Gaussian distribution, if it satisfies the following equivalent conditions: - there is a vector and a symmetric, positive semi-definite matrix such that the characteristic function of is
-
-
\phi_X(u) = \exp \left( i \mu^T u - \frac{1}{2} u^T \Gamma u \right) . The following is not quite equivalent to the conditions above, since it fails to allow for a singular matrix as the variance: -
f_X(x_1, \cdots, x_N) = \frac {1} {(2\pi)^{N/2} \left|\Sigma\right|^{1/2}} \exp \left( -\frac{1}{2} ( x - \mu)^T \Sigma^{-1} (x - \mu) \right) where is the determinant of . Note how the equation above reduces to that of the univariate normal distribution if is a scalar (i.e., a real number). The vector in these conditions is the expected value of and the matrix is the covariance matrix of the components . It is important to realize that the covariance matrix must be allowed to be singular. That case arises frequently in statistics; for example, in the distribution of the vector of residuals in ordinary linear regression problems. Note also that the are in general not independent; they can be seen as the result of applying the linear transformation to a collection of independent Gaussian variables . Bivariate case In the 2-dimensional nonsingular case, the probability density function is -
f(x,y) = \frac{1}{2 \pi \sigma_x \sigma_y \sqrt{1-\rho^2}} \exp \left( -\frac{1}{2 (1-\rho^2)} \left( \frac{x^2}{\sigma_x^2} + \frac{y^2}{\sigma_y^2} - \frac{2 \rho x y}{ (\sigma_x \sigma_y)} \right) \right) where is the correlation between and . Linear transformation If is a linear transformation of where is an matrix then has a multivariate normal distribution with expected value and variance (i.e., ~ . Corollary: any subset of the has a marginal distribution that is also multivariate normal. To see this consider the following example: to extract the subset , use -
B = \begin{bmatrix} 1 & 0 & 0 & 0 & 0 & \ldots & 0 \\ 0 & 1 & 0 & 0 & 0 & \ldots & 0 \\ 0 & 0 & 0 & 1 & 0 & \ldots & 0 \end{bmatrix} which extracts the desired elements directly. Conditional distributions Then if and are partitioned as follows -
\mu = \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix} \quad with sizes -
\Sigma = \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} \quad with sizes then the distribution of conditional on is multivariate normal ~ where -
\bar{\mu} = \mu_1 + \Sigma_{12} \Sigma_{22}^{-1} \left( a - \mu_2 \right) and covariance matrix -
\overline{\Sigma} = \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21}. This matrix is the Schur complement of in . Note that knowing the value of to be alters the variance; perhaps more surprisingly, the mean is shifted by ; compare this with the situation of not knowing the value of , in which case would have distribution . The matrix is known as the matrix of regression coefficients. Estimation of parameters The derivation of the maximum-likelihood estimator of the covariance matrix of a multivariate normal distribution is perhaps surprisingly subtle and elegant. See estimation of covariance matrices.
|
 |