artificial neuron

The artificial neuron (also called "node") is the basic unit of an artificial neural network, simulating a biological neuron. It receives one or more inputs, sums these, and produces an output after passing the sum through a (usually) non-linear function known as an activation or transfer function. The canonical form of this function is a sigmoid, but may also be another non-linear function, a piecewise linear function, or a step function. Generally, transfer functions are monotonically increasing.

Basic structure

For a given artificial neuron, let there be n inputs with signals x₁ through x_n and weights w₁ through w_n. The summed input u is given by

u = \sum_{j=1}^n w_j x_j

This value is then passed through a transfer function and either propogates to the next layer (through a weighted synapse) or finally exits the system as part or all of the output.

History

The original artificial neuron is the Threshold Logic Unit first proposed by Warren McCulloch and Walter Pitts in 1943. As a transfer function, it employs a threshold or step function taking on the values '1' or `0' only.

Perceptron

Types of transfer functions

The transfer function of a neuron is chosen to have a number of properties which either enhance or simplify the network containing the neuron. Crucially, for instance, any Artificial_neural_network#Multi-layer_perceptron using a linear transfer function has an equivalent single-layer network; a non-linear function is therefore necessary to gain the advantages of a multi-layer network.

Step function

The output y of this transfer function is binary, depending on whether the input meets a specified threshold, θ. The "signal" is sent, i.e. the output is set to one, if the activation meets the threshold.

y = \left\{ \begin{matrix} 1 & \mbox{if }u \ge \theta \\ 0 & \mbox{if }u < \theta \end{matrix} \right.

See: Step function

Sigmoid

A fairly simple non-linear function, the sigmoid also has an easily calculated derivative, which is used when calculating the weight updates in the network. It thus makes the network more easily manipulable mathematically, and was attractive to early computer scientists who needed to minimise the computational load of their simulations. See: Sigmoid function

Bibliography

* McCulloch, W. and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 7:115 - 133.

<< Previous

Word Browser

Next >>

king's lynn and west norfolk
shrewsbury and atcham
nuneaton and bedworth
small circle
great central railway (preserved)
sedgefield
scrofula
morbillivirus
brixton tube station
vermicelli
mardonius

tekov
trans appalachia
turiec
console manufacturer
baryon number
when the wind blows
payne stewart
minnesota experimental city
uniform norm
list of towns in slovakia
kura river

fizzbuzz
system requirements
artemisia
lost in the stars
german grand prix
convergence
yeah yeah yeahs ep
scissors (game)
khobar towers bombing
fever to tell
frank dobson

movement for democratic change
spim
linus yale, jr.
pin tumbler lock
pa wang chih luan
royal mail
antibacterial soap
postmillennialism
sylvanshine
kington, herefordshire
lindenwood university