r/MachineLearning • Researcher • Jul 07 '15

Grid Long Short-Term Memory

http://arxiv.org/abs/1507.01526
19 Upvotes

4 comments sorted by

2

u/NovaRom Jul 07 '15

Section 2.1 contains good and short explanation of a LSTM layer. They use data augmentation for MNIST tests.

3

u/BeatLeJuce Researcher Jul 07 '15

Yeah, I was very excited about the MNIST result before I saw the data augmentation. But then again I guess you have to for results like that. People should just let MNIST die out anyway....

The explanation of basic LSTMs was okay, but I felt that the grid could've been explained better. E.g. all their figures contain both green and red arrows, but they never mention the difference between the two. Nevertheless, this looks like a very interesting and promising architecture.

Sidenote: I wonder why they didn't stick with the "usual" notation most everyone else uses when describing LSTMs (they used "u" instead of "i" for their input gate subscripts).

3

u/marcuniq Jul 07 '15

Grid LSTM seem more general, but convolutional LSTM might be interesting to use for spatiotemporal sequences http://arxiv.org/abs/1506.04214

1

u/siblbombs Jul 07 '15

I appreciate that they included GFRNN for comparison during the character level task, some elements of n-lstm feel similar so it's nice to see a direct comparison listed instead of trying to figure it out.