|
|
| Line 1: |
Line 1: |
| In [[mathematics]], the '''entropy power inequality''' is a result in [[information theory]] that relates to so-called "entropy power" of [[random variable]]s. It shows that the entropy power of suitably [[well-behaved]] random variables is a [[superadditive]] [[function (mathematics)|function]]. The entropy power inequality was proved in 1948 by [[Claude Shannon]] in his seminal paper "[[A Mathematical Theory of Communication]]". Shannon also provided a sufficient condition for equality to hold; Stam (1959) showed that the condition is in fact necessary.
| | Wilber Berryhill is what his wife enjoys to contact him and he completely enjoys this name. I've always cherished residing in Mississippi. My day job is an invoicing officer but I've currently utilized for another one. To climb is something she would by no means give up.<br><br>Have a look at my web-site phone psychic readings [[http://jplusfn.gaplus.kr/xe/qna/78647 jplusfn.gaplus.kr]] |
| | |
| ==Statement of the inequality==
| |
| | |
| For a random variable ''X'' : Ω → '''R'''<sup>''n''</sup> with [[probability density function]] ''f'' : '''R'''<sup>''n''</sup> → '''R''', the [[differential entropy]] of ''X'', denoted ''h''(''X''), is defined to be
| |
| | |
| :<math>h(X) = - \int_{\mathbb{R}^{n}} f(x) \log f(x) \, d x</math>
| |
| | |
| and the entropy power of ''X'', denoted ''N''(''X''), is defined to be
| |
| | |
| :<math> N(X) = \frac{1}{2\pi e} e^{ \frac{2}{n} h(X) }.</math>
| |
| | |
| In particular, ''N''(''X'') = |''K''| <sup>1/''n''</sup> when ''X'' is normal distributed with covariance matrix ''K''.
| |
| | |
| Let ''X'' and ''Y'' be [[independent random variables]] with probability density functions in the [[Lp space|''L''<sup>''p''</sup> space]] ''L''<sup>''p''</sup>('''R'''<sup>''n''</sup>) for some ''p'' > 1. Then
| |
| | |
| :<math>N(X + Y) \geq N(X) + N(Y). \,</math>
| |
| | |
| Moreover, equality holds [[if and only if]] ''X'' and ''Y'' are [[multivariate normal]] random variables with proportional [[covariance matrix|covariance matrices]].
| |
| | |
| ==See also==
| |
| *[[Information entropy]]
| |
| *[[Information theory]]
| |
| *[[Limiting density of discrete points]]
| |
| *[[Self-information]]
| |
| *[[Kullback–Leibler divergence]]
| |
| *[[Entropy estimation]]
| |
| | |
| ==References==
| |
| | |
| * {{cite journal
| |
| | last = Dembo
| |
| | first = Amir
| |
| | coauthors = Cover, Thomas M. and Thomas, Joy A.
| |
| | title = Information-theoretic inequalities
| |
| | journal = IEEE Trans. Inform. Theory
| |
| | volume = 37
| |
| | year = 1991
| |
| | issue = 6
| |
| | pages = 1501–1518
| |
| | doi = 10.1109/18.104312
| |
| | mr = 1134291
| |
| }}
| |
| * {{cite journal
| |
| | last=Gardner
| |
| | first=Richard J.
| |
| | title=The Brunn–Minkowski inequality
| |
| | journal=[[Bulletin of the American Mathematical Society|Bull. Amer. Math. Soc.]] (N.S.)
| |
| | volume=39
| |
| | issue=3
| |
| | year=2002
| |
| | pages=355–405 (electronic)
| |
| | doi=10.1090/S0273-0979-02-00941-2
| |
| }}
| |
| * {{cite journal
| |
| | last = Shannon
| |
| | first = Claude E.
| |
| | authorlink = Claude Shannon
| |
| | title = A mathematical theory of communication
| |
| | journal = [[Bell System Technical Journal|Bell System Tech. J.]]
| |
| | volume = 27
| |
| | year = 1948
| |
| | pages = 379–423, 623–656
| |
| }}
| |
| * {{cite journal
| |
| | last = Stam
| |
| | first = A. J.
| |
| | title = Some inequalities satisfied by the quantities of information of Fisher and Shannon
| |
| | journal = Information and Control
| |
| | volume = 2
| |
| | year = 1959
| |
| | pages = 101–112
| |
| | doi = 10.1016/S0019-9958(59)90348-1
| |
| | issue = 2
| |
| }}
| |
| | |
| [[Category:Information theory]]
| |
| [[Category:Probabilistic inequalities]]
| |
| [[Category:Statistical inequalities]]
| |
Wilber Berryhill is what his wife enjoys to contact him and he completely enjoys this name. I've always cherished residing in Mississippi. My day job is an invoicing officer but I've currently utilized for another one. To climb is something she would by no means give up.
Have a look at my web-site phone psychic readings [jplusfn.gaplus.kr]