Gibbs algorithm
In statistical mechanics, the Gibbs algorithm, introduced by J. Willard Gibbs in 1902, is a criterion for choosing a probability distribution for the statistical ensemble of microstates of a thermodynamic system by minimizing the average log probability ⟨ ln p i ⟩ = ∑ i p i ln p i {\displaystyle \langle \ln p_{i}\rangle =\sum _{i}p_{i}\ln p_{i}\,} subject to the probability distribution pi satisfying a set of constraints (usually expectation values) corresponding to the known macroscopic quantities. in 1948, Claude Shannon interpreted the negative of this quantity, which he called information entropy, as a measure of the uncertainty in a probability distribution.