Resoruces for learning about RNNs, LSTMs, and Torch

Here’s a very partial list of resources that might come in handy.

Torch and Lua

1. The Lua manual.
2. Documentation for Torch.
3. Paper introducing Torch.
4. Documentation for nn.
5. nngraph library.
6. Penlight, a python-inspired library for Lua.
7. cutorch documentation for using GPUs with Torch.

Recurrent neural nets, including LSTMs

1. Wojciech Zaremba’s LSTM implementation, on which I based my own.
2. Andrej Karpathy’s blog post, The unreasonable effectiveness of recurrent nerual networks
3. Yoshua Bengio’s Practical recommendations for gradient-based training of deep architectures.
4. A deep learning with Torch slidedeck by Jimmy Ba at the University of Toronto.
5. Alex Graves formulation of LSTMs, which I follow.