Maximum Entropy Method

The maximum entropy method is a principle for estimating, make statistical inferences over partial knowledge. It is the least biased estimate possible on the given information, a technique used to estimate input probabilities where little or no information is available.
Specifying probabilities in lack of information is an old world problem. Laplace’s ”Principle of Insufficient Reason” was an attempt to supply a criterion of choice, which says that there is no reason to assign other but equal probabilities when no further information is assumed. However, except in a situation where there is an element of symmetry that clearly renders equal probabilities between events, this assumption is just as arbitrary as any other.
The Principle of Maximum Entropy was originally motivated by Statistical Mechanics, trying to relate macroscopic measured properties of physical systems to microscopic models of the matter. This is a typical situation where there is only little information available and a physical system will be approached with no further assumptions. This principle was pioneered by Edwin T. Janes (1922 - 1998), a professor at Washington University in St. Louis, who first published ”Information Theory and Statistical Mechanics”[1].

[1] Edwin T. Jaynes, Information Theory and Statistical Mechanics, Physical Review, vol. 106, no. 4, pp. 620-630, May 15, 1957. http://bayes.wustl.edu/etj/articles/theory.1.pdf