Asymptotic equipartition property: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
en>Mild Bill Hiccup
m lower case
en>Myasuda
Line 1: Line 1:
{{about||Euler's chain rule relating partial derivatives of three independent variables|Triple product rule|the counting principle in combinatorics|Rule of product}}
Hello! My name is Ramonita. I smile that I could join to the entire globe. I live in Brazil, in the south region. I dream to check out the different nations, to obtain familiarized with intriguing people.<br><br>my web page: [http://www.allexampass.com/070-410.html Microsoft 070-410 exam]
{{Calculus |Differential}}
 
In [[calculus]], the '''product rule''' is a formula used to find the [[derivative]]s of products of two or more [[Functions (mathematics)|functions]]. It may be stated thus:
 
:<math>(f\cdot g)'=f'\cdot g+f\cdot g' \,\! </math>
 
or in the [[Leibniz notation]] thus:
 
:<math>\dfrac{d}{dx}(u\cdot v)=u\cdot \dfrac{dv}{dx}+v\cdot \dfrac{du}{dx}</math>.
 
In the notation of differentials this can be written as follows:
:<math> d(uv)=u\,dv+v\,du</math>.
 
The derivative of the product of three functions is:
 
:<math>\dfrac{d}{dx}(u\cdot v \cdot w)=\dfrac{du}{dx} \cdot v \cdot w + u \cdot \dfrac{dv}{dx} \cdot w + u\cdot v\cdot \dfrac{dw}{dx}</math>.
 
== Discovery ==
Discovery of this rule is credited to [[Gottfried Leibniz]], who demonstrated it using [[differential (calculus)|differentials]].<ref>{{cite journal |author=Michelle Cirillo
|journal=The Mathematics Teacher |volume=101 |issue=1 |pages=23&ndash;27 |date=August 2007 |title=Humanizing Calculus
|url=http://www.nctm.org/uploadedFiles/Articles_and_Journals/Mathematics_Teacher/Humanizing%20Calculus.pdf
}}</ref>  (However, Child (2008) argues that it is due to [[Isaac Barrow]]). Here is Leibniz's argument: Let ''u''(''x'') and ''v''(''x'') be two [[differentiable function]]s of ''x''. Then the differential of ''uv'' is
 
: <math>
\begin{align}
d(u\cdot v) & {} = (u + du)\cdot (v + dv) - u\cdot v \\
& {} = u\cdot dv + v\cdot du + du\cdot dv.
\end{align}
</math>
 
Since the term ''du''·''dv'' is "negligible" (compared to ''du'' and ''dv''), Leibniz concluded that
 
:<math>d(u\cdot v) = v\cdot du + u\cdot dv \,\! </math>
 
and this is indeed the differential form of the product rule. If we divide through by the differential ''dx'', we obtain
 
:<math>\frac{d}{dx} (u\cdot v) = v \cdot \frac{du}{dx} + u \cdot  \frac{dv}{dx} \,\!  </math>
 
which can also be written in [[Derivative#Lagrange.27s_notation|Lagrange's notation]] as
 
:<math>(u\cdot v)' = v\cdot  u' + u\cdot  v'. \,\! </math>
 
== Examples ==
*Suppose we want to differentiate ''&fnof;''(''x'') = ''x''<sup>2</sup> [[sine|sin]](''x''). By using the product rule, one gets the derivative ''&fnof;''<nowiki>&nbsp;'</nowiki>(''x'') = 2''x'' sin(''x'') + ''x''<sup>2</sup>cos(''x'') (since the derivative of ''x''<sup>2</sup> is 2''x'' and the derivative of sin(''x'') is cos(''x'')).
*One special case of the product rule is the '''constant multiple rule''' which states: if ''c'' is a [[real number]] and ''&fnof;''(''x'') is a differentiable function, then ''c&fnof;''(''x'') is also differentiable, and its derivative is (''c''&nbsp;&times;&nbsp;''&fnof;'')<nowiki>'</nowiki>(''x'') = ''c'' &times; ''&fnof;''<nowiki>&nbsp;'</nowiki>(''x''). This follows from the product rule since the derivative of any constant is zero. This, combined with the sum rule for derivatives, shows that differentiation is [[linear transformation|linear]].
*The rule for [[integration by parts]] is derived from the product rule, as is (a weak version of) the [[quotient rule]]. (It is a "weak" version in that it does not prove that the quotient is differentiable, but only says what its derivative is ''if'' it is differentiable.)
 
== Proofs ==
{{unreferenced section|date=July 2013}}
A rigorous proof of the product rule can be given using the [[Derivative#Definition_via_difference_quotients|definition of the derivative]] as a [[limit (mathematics)|limit]], and the basic [[Limit_of_a_function#Properties|properties of limits]].
 
Let ''h(x) = f(x) g(x)'', and suppose that ''f'' and ''g'' are each differentiable at ''x<sub>0</sub>''. (Note that ''x<sub>0</sub>'' will remain fixed throughout the proof). We want to prove that ''h'' is differentiable at ''x<sub>0</sub>'' and that its derivative ''h'(x<sub>0</sub>)'' is given by ''f'(x<sub>0</sub>) g(x<sub>0</sub>) + f(x<sub>0</sub>) g'(x<sub>0</sub>)''.
 
Let ''Δh = h(x<sub>0</sub>+Δx) - h(x<sub>0</sub>)''; note that although ''x<sub>0</sub>'' is fixed, ''Δh'' depends on the value of ''Δx'', which is thought of as being "small."
 
The function ''h'' is differentiable at ''x<sub>0</sub>'' if the limit
 
:<math>\lim_{\Delta x\to 0}{ \Delta h \over \Delta x}</math>
 
exists; when it does, ''h'(x<sub>0</sub>)'' is defined to be the value of the limit.
 
As with ''Δh'', let ''Δf = f(x<sub>0</sub>+Δx) - f(x<sub>0</sub>)'' and ''Δg = g(x<sub>0</sub>+Δx) - g(x<sub>0</sub>)'' which, like ''Δh'', also depends on ''Δx''. Then ''f(x<sub>0</sub>+Δx) = f(x<sub>0</sub>) + Δf'' and ''g(x<sub>0</sub>+Δx) = g(x<sub>0</sub>) + Δg''.
 
It follows that ''h(x<sub>0</sub>+Δx) = f(x<sub>0</sub>+Δx) g(x<sub>0</sub>+Δx) = (f(x<sub>0</sub>) + Δf) (g(x<sub>0</sub>)+Δg)''; applying the distributive law, we see that
 
{{NumBlk|:|<math>h(x_0+\Delta x) = f(x_0+\Delta x) g(x_0+\Delta x) = f(x_0) g(x_0) + \Delta f g(x_0) + f(x_0) \Delta g + \Delta f \Delta g</math>|{{EquationRef|*}}}}
 
While it is not necessary for the proof, it can be helpful to understand this product geometrically as the area of the rectangle in this diagram:
 
[[File:Illustration-for-leibniz-product-rule.svg|''h(x+Δx) as the area of a rectangle'']]
 
To get the value, of ''Δh'', subtract ''h(x<sub>0</sub>)=f(x<sub>0</sub>) g(x<sub>0</sub>)'' from equation {{EquationNote|*|(*)}}. This removes the area of the white rectangle, leaving three rectangles:
 
:<math>\Delta h = \Delta f g(x_0) + f(x_0) \Delta g + \Delta f \Delta g</math>
 
To find ''h'(x<sub>0</sub>)'', we need to find the limit as ''Δx'' goes to 0 of
 
{{NumBlk|:|<math>\frac{\Delta h}{\Delta x} = \frac{\Delta f g(x_0) + f(x_0) \Delta g + \Delta f \Delta g}{\Delta x} = \frac{\Delta f}{\Delta x}g(x_0) +f(x_0)  \frac{\Delta g}{\Delta x} + \frac{\Delta f \Delta g}{\Delta x}</math>|{{EquationRef|**}}}}
 
The first two terms of the right-hand side of this equation correspond to the areas of the blue rectangles; the third corresponds to the area of the gray rectangle. Using the basic [[Limit_of_a_function#Properties|properties of limits]] and the definition of the derivative, we can tackle this term-by term. First,
 
:<math>\lim_{\Delta x\to 0}\left ( \frac{\Delta f}{\Delta x}g(x_0) \right ) = f'(x_0)g(x_0)</math>.
 
Similarly,
 
:<math>\lim_{\Delta x\to 0} \left ( f(x_0)  \frac{\Delta g}{\Delta x} \right ) = f(x_0)g'(x_0)</math>.
 
The third term, corresponding to the small gray rectangle, winds up being negligible (i.e. going to 0 in the limit) because ''Δf Δg'' "vanishes to second order." Rigorously,
 
:<math>\lim_{\Delta x\to 0} \frac{\Delta f\Delta g}{\Delta x} = \lim_{\Delta x\to 0} \left ( \frac{\Delta f}{\Delta x}\frac{\Delta g}{\Delta x}\Delta x  \right ) = \lim_{\Delta x\to 0}{\frac{\Delta f}{\Delta x}} \cdot \lim_{\Delta x\to 0}{\frac{\Delta g}{\Delta x}} \cdot \lim_{\Delta x\to 0}{\Delta x}= f'(x_0) g'(x_0) \cdot 0 = 0</math>
 
We have shown that the limit of each of the three terms on the right-hand side of equation {{EquationNote|**|(**)}} exists, hence
:<math>\lim_{\Delta x\to 0} \frac{\Delta h}{\Delta x}</math>
exists and is equal to the sum of the three limits. Thus, the product ''h(x)'' is differentiable at ''x<sub>0</sub>'' and its derivative is given by
:<math>\begin{align}
h'(x_0) & = \lim_{\Delta x\to 0} \frac{\Delta h}{\Delta x}\\
&  =  \lim_{\Delta x\to 0}  \left ( \frac{\Delta f}{\Delta x}g(x_0) \right ) + \lim_{\Delta x\to 0}  \left ( f(x_0)  \frac{\Delta g}{\Delta x}\right )  +  \lim_{\Delta x\to 0}  \left ( \frac{\Delta f \Delta g}{\Delta x} \right ) \\
& = f'(x_0)g(x_0) + f(x_0)g'(x_0) + 0 \\
& = f'(x_0)g(x_0) + f(x_0)g'(x_0) \\
\end{align}</math>
as was to be shown.
 
=== Brief proof ===
By definition, if <math> f, g: \mathbb{R} \rightarrow \mathbb{R} </math> are differentiable at <math> x </math> then we can write
:<math> f(x+h) = f(x) + f'(x)h + \psi_1(h) \qquad \qquad g(x+h) = g(x) + g'(x)h + \psi_2(h) </math>
such that <math> \lim_{h \to 0} \frac{\psi_1(h)}{h} = \lim_{h \to 0} \frac{\psi_2(h)}{h} = 0 </math>, [[Big O notation#Little-o notation|also written]] <math>\psi_1, \psi_2 \sim o(h)</math>. Then:
 
:<math> \begin{align} fg(x+h) - fg(x) =  (f(x) + f'(x)h +\psi_1(h))(g(x) + g'(x)h + \psi_2(h)) - fg(x)= f'(x)g(x)h + f(x)g'(x)h + O(h) \\[12pt] \end{align} </math>
Taking the limit for small <math> h </math> gives the result.
 
=== Logarithms and quarter squares ===
Let ''f'' = ''uv'' and suppose ''u'' and ''v'' are positive functions of ''x''. Then
 
:<math>\ln f  =\ln (u\cdot v)=\ln u + \ln v.\, </math>
 
Differentiating both sides:
 
:<math> {1 \over f} {df \over dx} = {1 \over u} {du \over dx} + {1 \over v} {dv \over dx}\, </math>
 
and so, multiplying the left side by ''f'', and the right side by ''uv'' (note: ''f'' = ''uv''),
 
:<math>{df \over dx} = v {du \over dx} + u {dv \over dx}.\, </math>
 
The proof appears in [http://planetmath.org/encyclopedia/LogarithmicProofOfProductRule.html]. Note that since ''u'', ''v'' need to be continuous, the assumption on positivity does not diminish the generality.
 
This proof relies on the [[chain rule]] and on the properties of the [[natural logarithm]] function, both of which are deeper than the product rule (however, information about the derivative of a logarithm that is sufficient to carry out a variant of the proof can be inferred by considering the derivative at ''x'' = ''1'' of the logarithm to any base of ''cx'', where ''c'' is a constant, then generalising ''c''). From one point of view, that is a disadvantage of this proof. On the other hand, the simplicity of the algebra in this proof perhaps makes it easier to understand than a proof using the definition of differentiation directly.
 
There is an analogous but arguably even easier proof (i.e., some people may find it easier as it can be used before being able to differentiate logarithms), using [[Multiplication_algorithm#Quarter_square_multiplication|quarter square multiplication]], which similarly relies on the [[chain rule]] and on the properties of the quarter square function (shown here as ''q'', i.e, with <math>q(x)={x^2 \over 4}</math>):
 
:<math>f=q(u+v)-q(u-v), </math>
 
Differentiating both sides:
 
:<math>f'=q'(u+v)(u'+v') - q'(u-v)(u'-v')</math>
 
:<math>=\left({1 \over 2}(u+v)(u'+v')\right) - \left({1 \over 2}(u-v)(u'-v')\right)</math>
 
:<math>={1 \over 2}(uu' + vu' + uv' + vv') - {1 \over 2}(uu' - vu' - uv' + vv')</math>
 
:<math>=vu'+uv'</math>
 
:<math>=uv'+u'v. \, </math>
 
This does not present issues of whether the values are positive or negative, and the function's properties are much simpler to demonstrate (indeed, it can be differentiated without using first principles by considering the derivative at ''x'' = ''0'' of ''cx'', where ''c'' is a constant, then generalising ''c'').
 
Note also, these proofs are only valid for numbers or similar, whereas proofs from first principles are also valid for matrices and such like.
 
=== Chain rule ===
The product rule can be considered a special case of the [[chain rule]] for several variables.
 
: <math> {d (ab) \over dx} = \frac{\partial(ab)}{\partial a}\frac{da}{dx}+\frac{\partial (ab)}{\partial b}\frac{db}{dx} = b \frac{da}{dx} + a \frac{db}{dx}. \, </math>
 
=== Non-standard analysis ===
Let ''u'' and ''v'' be continuous functions in ''x'', and let d''x'', d''u'' and d''v'' be [[infinitesimal]]s within the framework of [[non-standard analysis]], specifically the [[hyperreal number]]s. Using st to denote the [[standard part function]] that associates to a [[wiktionary:Finite number|finite]] hyperreal number the real infinitely close to it, this gives
 
:{|
|-
|<math>\frac{d(uv)}{dx}\,</math>
|<math>=\operatorname{st}\left(\frac{(u + \mathrm du)(v + \mathrm dv) - uv}{\mathrm dx}\right)</math>
|-
|
|<math>=\operatorname{st}\left(\frac{uv + u \cdot \mathrm dv + v \cdot \mathrm du + \mathrm dv \cdot \mathrm du -uv}{\mathrm dx}\right)</math>
|-
|
|<math>=\operatorname{st}\left(\frac{u \cdot \mathrm dv + (v + \mathrm dv) \cdot \mathrm du}{\mathrm dx}\right)</math>
|-
|
|<math>={u}\frac{dv}{dx} + {v}\frac{du}{dx} </math>.
|}
This was essentially [[Leibniz]]'s proof exploiting the [[transcendental law of homogeneity]] (in place of the standard part above).
 
===Smooth infinitesimal analysis===
In the context of Lawvere's approach to infinitesimals, let dx be a nilsquare infinitesimal. Then du = u' dx and dv = v' dx, so that
 
: <math>
\begin{align}
d(uv) & {} = (u + du)(v + dv)  -uv \\
& {} = uv + u\cdot dv + v\cdot du + du\cdot dv - uv \\
& {} = u\cdot dv + v\cdot du + du\cdot dv \\
& {} = u\cdot dv + v\cdot du\,\!
\end{align}
</math>
 
since
 
:<math>du \cdot dv = u' v' (dx)^2 = 0\,\!</math>
 
== Generalizations ==
{{unreferenced section|date=July 2013}}
=== A product of more than two factors ===
The product rule can be generalized to products of more than two factors. For example, for three factors we have
 
:<math>\frac{d(uvw)}{dx} = \frac{du}{dx}vw + u\frac{dv}{dx}w + uv\frac{dw}{dx}\,\! </math>.
 
For a collection of functions <math>f_1, \dots, f_k</math>, we have
 
:<math>\frac{d}{dx} \left [ \prod_{i=1}^k f_i(x) \right ]
= \sum_{i=1}^k \left(\frac{d}{dx} f_i(x) \prod_{j\ne i} f_j(x) \right)
= \left(  \prod_{i=1}^k f_i(x) \right) \left( \sum_{i=1}^k \frac{f'_i(x)}{f_i(x)} \right).</math>
 
=== Higher derivatives ===
It can also be generalized to the [[Leibniz rule (generalized product rule)|Leibniz rule]] for the ''n''th derivative of a product of two factors:
:<math>(uv)^{(n)}(x) = \sum_{k=0}^n {n \choose k} \cdot u^{(n-k)}(x)\cdot  v^{(k)}(x).</math>
 
See also [[binomial coefficient]] and the formally quite similar [[binomial theorem]]. See also [[General Leibniz rule]].
 
=== Higher partial derivatives ===
For [[partial derivative]]s, we have
 
:<math>{\partial^n \over \partial x_1\,\cdots\,\partial x_n} (uv)
= \sum_S {\partial^{|S|} u \over \prod_{i\in S} \partial x_i} \cdot {\partial^{n-|S|} v \over \prod_{i\not\in S} \partial x_i}</math>
 
where the index ''S'' runs through the whole list of 2<sup>''n''</sup> subsets of {1,&nbsp;...,&nbsp;''n''}. For example, when ''n''&nbsp;=&nbsp;3, then
 
:<math>\begin{align} &{}\quad {\partial^3 \over \partial x_1\,\partial x_2\,\partial x_3} (uv)  \\  \\
&{}= u \cdot{\partial^3 v \over \partial x_1\,\partial x_2\,\partial x_3} + {\partial u \over \partial x_1}\cdot{\partial^2 v \over \partial x_2\,\partial x_3} +  {\partial u \over \partial x_2}\cdot{\partial^2 v \over \partial x_1\,\partial x_3} + {\partial u \over \partial x_3}\cdot{\partial^2 v \over \partial x_1\,\partial x_2} \\  \\
&{}\qquad + {\partial^2 u \over \partial x_1\,\partial x_2}\cdot{\partial v \over \partial x_3}
+ {\partial^2 u \over \partial x_1\,\partial x_3}\cdot{\partial v \over \partial x_2}
+ {\partial^2 u \over \partial x_2\,\partial x_3}\cdot{\partial v \over \partial x_1}
+ {\partial^3 u \over \partial x_1\,\partial x_2\,\partial x_3}\cdot v. \end{align}</math>
 
=== Banach space ===
Suppose ''X'', ''Y'', and ''Z'' are [[Banach space]]s (which includes [[Euclidean space]]) and ''B'' : ''X'' &times; ''Y'' → ''Z'' is a [[continuous function (topology)|continuous]] [[bilinear operator]]. Then ''B'' is differentiable, and its derivative at the point (''x'',''y'') in ''X'' &times; ''Y'' is the [[linear map]] ''D''<sub>(''x'',''y'')</sub>''B'' : ''X'' &times; ''Y'' → ''Z'' given by
:<math> (D_\left( x,y \right)\,B)\left( u,v \right) = B\left( u,y \right) + B\left( x,v \right)\qquad\forall (u,v)\in X \times Y. </math>
 
=== Derivations in abstract algebra ===
In [[abstract algebra]], the product rule is used to ''define'' what is called a [[derivation (abstract algebra)|derivation]], not vice versa.
 
=== Vector functions ===
The product rule extends to [[scalar multiplication]], [[dot product]]s, and [[cross product]]s of vector functions.
 
For scalar multiplication:
<math>(f \cdot \vec g)' = f\;'\cdot \vec g + f \cdot \vec g\;' \,</math>
 
For dot products:
<math>(\vec f \cdot \vec g)' = \vec f\;'\cdot \vec g + \vec f \cdot \vec g\;' \,</math>
 
For cross products:
<math>(\vec f \times \vec g)' = \vec f\;' \times \vec g + \vec f \times \vec g\;' \,</math>
 
(Beware: since cross products are not commutative, it is not correct to write <math>(f\times g)'=f'\times g+g'\times f. \,</math> But cross products are anticommutative, so it can be written as <math>(f\times g)'=f'\times g-g'\times f. \,</math>
 
=== Scalar fields ===
For scalar fields the concept of [[gradient]] is the analog of the derivative:
 
<math>\nabla (f \cdot g) = \nabla f \cdot g + f \cdot \nabla g \,</math>
 
== Applications ==
Among the applications of the product rule is a proof that
 
:<math> {d \over dx} x^n = nx^{n-1}\,\! </math>
 
when ''n'' is a positive integer (this rule is true even if ''n'' is not positive or is not an integer, but the proof of that must rely on other methods). The proof is by [[mathematical induction]] on the exponent ''n''. If ''n''&nbsp;=&nbsp;0 then ''x''<sup>''n''</sup> is constant and ''nx''<sup>''n''&nbsp;&minus;&nbsp;1</sup> = 0. The rule holds in that case because the derivative of a constant function is&nbsp;0. If the rule holds for any particular exponent ''n'', then for the next value, ''n''&nbsp;+&nbsp;1, we have
 
:<math>\begin{align}
{d \over dx}x^{n+1} &{}= {d \over dx}\left( x^n\cdot x\right) \\[12pt]
&{}= x{d \over dx} x^n + x^n{d \over dx}x \qquad\mbox{(the product rule is used here)} \\[12pt]
&{}= x\left(nx^{n-1}\right) + x^n\cdot 1\qquad\mbox{(the induction hypothesis is used here)} \\[12pt]
&{}= (n + 1)x^n.
\end{align} </math>
 
Therefore if the proposition is true of ''n'', it is true also of&nbsp;''n''&nbsp;+&nbsp;1.
 
=== Definition of tangent space ===
Product rule is also used in [[Tangent_space#Definition_via_derivations|definition]] of abstract [[tangent space]] of some abstract geometric figure ([[smooth manifold]]). This definition we can use if we cannot or wish to not use surrounding ambient space where our chosen geometric figure lives (since there might be no such surrounding space). It uses the fact that it is possible to define derivatives of real-valued functions on that geometric figure at a point p solely with the product rule and that the set of all such derivations in fact forms a [[vector space]] that is the desired tangent space.
 
==See also==
* [[Derivation (abstract algebra)]]
* [[Differential (calculus)]]
* [[General Leibniz rule]]
* [[Quotient rule]]
* [[Reciprocal rule]]
 
==References==
{{reflist}}
{{refbegin}}
*Child, J. M. (2008) "The early mathematical manuscripts of Leibniz", Gottfried Wilhelm Leibniz, translated by J. M. Child; page 29, footnote&nbsp;58.
{{refend}}
 
==External links==
* [http://www.math.ucdavis.edu/~kouba/CalcOneDIRECTORY/productruledirectory/ProductRule.html Product Rule Practice Problems [Kouba, University of California: Davis<nowiki>]</nowiki>]
 
{{use dmy dates|date=January 2012}}
 
[[Category:Differentiation rules]]
[[Category:Articles containing proofs]]
 
{{Link GA|de}}

Revision as of 04:23, 3 March 2014

Hello! My name is Ramonita. I smile that I could join to the entire globe. I live in Brazil, in the south region. I dream to check out the different nations, to obtain familiarized with intriguing people.

my web page: Microsoft 070-410 exam