automatic differentiation

In mathematics and computer algebra, automatic differentiation is a method to evaluate the derivative of a function, at a given point. Two classical ways of doing this are:

symbolically differentiate the function and evaluate it in the point; or
use finite differences.

The drawback with symbolic differentiation is low speed. Many functions grow too complex as higher derivatives are calculated. Two important drawbacks with finite differences are round-off errors in the discretization process, and cancellation. Thus both classical methods have problems with calculating higher derivatives, where the complexity and errors increase. Automatic differentiation solves all of the mentioned problems and is easily implemented on computers.

First-order automatic differentiation

A new arithmetic is introduced using ordered pairs as objects. The new arithmetic uses ordinary arithmetic on the first element of the pair and a first order differentiation arithmetic applying the chain rule on the second element. The basic arithmetic operations and some standard functions are defined by

\left(u,u'\right)+\left(v,v'\right)=\left(u+v, u'+v' \right)

\left(u,u'\right)-\left(v,v'\right)=\left(u-v, u'-v' \right)

\left(u,u'\right)*\left(v,v'\right)=\left( u v, u'v+uv' \right)

\left(u,u'\right)/\left(v,v'\right)=\left( \frac{u}{v}, \frac{u'v-uv'}{v^2} \right)\quad \left(v\ne 0\right)

\sin\left(u,u'\right)=\left(\sin\left(u\right),u'\cos\left(u\right)\right)

\cos\left(u,u'\right)=\left(\cos\left(u\right),-u'\sin\left(u\right)\right)

e^{\left(u,u'\right)}=\left(e^u,u'e^u\right)

\log\left(u,u'\right)=\left(\log\left(u\right),u'/u\right)\quad \left(u>0\right)

\left(u,u'\right)^k=\left(u^k,ku^{k-1}u'\right)\quad \left(u\ne 0\right)

\left|\left(u,u'\right)\right|=\left(\left|u\right|,u'\mbox{sign}\left(u\right)\right)\quad \left(u\ne 0\right).

The derivative of a function

f\left( x \right)

at the point

x_0

is now calculated by substituting

x_0

by the ordered pair

\left(x_0,1\right)

in the expression for

f\left( x \right)

. A constant c is substituted by

\left(c,0\right)

. Calculating

f'\left( x_0 \right)

using the above arithmetic gives

\left( f   \left( x_0 \right), f' \left( x_0 \right) \right)

as the answer.

Higher order automatic differentiation

The above arithmetic can be generalized, in the natural way, to second order and higher derivatives. However, the arithmetic rules quickly grow very complicated. Instead, truncated Taylor series arithmetic is used. This is possible because the Taylor summands in a Taylor series of a function are products of known coefficients and derivatives of the function.

Software

There are plenty of libraries for automatic differentiation.

External link

An "entry site to everything you want to know about automatic differentiation"

<< Previous

Word Browser

Next >>

last chance saloon
charles chetwynd talbot, 20th earl of shrewsbury
gwynfor evans
bobby unser
louis untermeyer
coat of arms of the german democratic republic
oscar unzaga
alexander bell patterson
thomas upington
richard upjohn
emory upton

boyz
agathe uwilingiyimana
cassam uteem
liuqin
flight of the migrator
dmitri fyodorovich ustinov
list of scientific howlers in literature
john palmer usher
yangqin
william usery jr.
petrus van der aa

zheng
manuel urrutia lle
ali suat hayri urguplu
hangedup
kicker in tow
mogosoaia palace
charles arthur gauthier
jack cohen (tesco)
bishop's cleeve
mayda
thomas urquhart

gong (title)
gong (disambiguation)
active bat
acteonidae
california state highway 138
darius vassell
marquess of hastings
ken campbell (evangelist)
shanghai metro
cashflow matching
subitizing and counting