Artificial Neural Network- Part 3

- In 1969 a method for learning in multi-layer network,
Backpropagation, was invented by Bryson and Ho.


- The Backpropagation algorithm is a sensible approach
for dividing the contribution of each weight.

- Works basically the same as perceptron
There are two differences for the updating rule :


1) The activation of the hidden unit is used instead of activation of the input value.

2) The rule contains a term for the gradient of the activation
function.

1. Initialize network with random weights
2. For all training cases (called examples):
a. Present training inputs to network and calculate output
b. For all layers (starting with output layer, back to input layer):
i. Compare network output with correct output
(error function)
ii. Adapt weights in current layer

Method for learning weights in feed-forward (FF) nets

Can’t use Perceptron Learning Rule
no teacher values are possible for hidden units

Use gradient descent to minimize the error
propagate deltas to adjust for errors
backward from outputs
to hidden layers
to inputs