Recurrent Neural Networks in PyTorch
Feed forward networks cannot learn from the past, but Recurrent Neural Networks (RNNs) can learn by accepting data in a sequence. Examples of applications for RNNs include the text autocomplete feature on your phone and performing language translations. Recurrent Neurons (RNs) act as the building blocks of RNNs. The difference between RNs and feed forward neurons is that RNs accept input x, at time t, as well as a hidden state or output from time t-1 from another RN. The output of a RN is a vector, unlike for a feed forward neuron. RNNs are trained (i.e. their weights are calculated) using backpropagation via Gradient Descent Optimization in time. Output from a single RNN layer at time instance,t is an input to the next layer. Each layer in the RNN represents an instance in time. ...