Information theory is, primarily, concerned with quantifying the uncertainty of sets of related random variables. This uncertainty is usually described by a probability density distribution, the Gaussian distribution being one of the most familiar and widely used, due to its simple parametric form. In this talk, I review “classical” multivariate Gaussian probability density distributions, then extend the concepts to non-Gaussian distributions using mixture models. There is a broad array of practical applications for such probability density distributions, generally falling into one of three functional categories: clustering analysis; discriminant analysis (i.e., feature classification); and regression analysis. I focus on the latter by showing how multivariate Gaussian mixture models can be used to fill in “missing” ground magnetometer observations in magnetic local time coordinates using relatively sparse available observations.