No. 39
2026-08-27
the AI pulse — brewed, not raw
 VIDEO
Turkish subtitles
  1. Imagine you're at murder mystery dinner.
  2. Right at the start, the Lord of the Manor abruptly keels over and your task is to figure out, who done it?
  3. Could be the maid, could be the butler.
  4. But you've got a problem, your short term memory isn't working so We well. You can't remember any of the clues past the last 10 minutes. Well, that sort of situation, your prediction is going to be nothing better than just a random guess.
  5. Well, imagine you have the opposite problem where you can remember every word of every conversation that you've ever had.
  6. If somebody asked you to outline your partner's wedding vows, well, you might have some trouble doing that.
  7. There's just so many words you'd need to process be much better than if you could just remember, well, the memorable stuff.
  8. And that's where something called Long Short Term Memory comes into play.
  9. Also abbreviated as LSTM, it allows a neural network to remember the stuff that it needs to keep hold of context, but also to forget the stuff that well is no
  10. longer applicable.
  11. So take, for example, this sequence of letters.
  12. We need to predict what the next letter in the sequence is going to be.
  13. Well, just by looking at the letters individually, it's not obvious what the next sequence is, like we have to M's and they both have a different letter following them.
  14. So how do we predict the sequence?
  15. Well, if we have gone back through the time series to look at all of the letters in the sequence, we can establish contacts and we can clearly see, oh yes, it's my name is.
  16. And if we, instead of looking at letters, looked at words, we can establish that the whole sentence here says, my name is, oh yes, Martin.
  17. Now, a recurrent neural network is really where LSTM lives.
  18. So effectively, and LSTM is a type of recurrent neural network.
  19. Recurrent neural net.
  20. And recurrent neural networks work in the sense that they have a node, so there's a node here and this node receives some
  21. input.
  22. So we've got some input coming in.
  23. That input is then processed in some way, so there's some kind of computation and that results in and output.
  24. That's pretty standard stuff.
  25. But what makes an RNN node a little bit different is the fact that it is recurrent.
  26. And that means that it loops around. So the output of a given step is provided alongside the input in the next step.
  27. So step one has some input.
  28. It's processed and that results in some output.
  29. Then step two has some new input, but it also receives the output of the prior step as well. That is what makes an RNN a little bit different, and it allows it to remember previous steps in a sequence.
  30. So when we're looking at a sentence like my name, I, we don't have to go back too far through those steps to figure out what the context is.
  31. But RNN does suffer from what's known as the long term dependency problem, which is to say that over time, as more and more information piles up, then RNNs become less effective at learning
  32. new things.
  33. So while we didn't have to go too far back for my name, I, if we were going back through an hour's worth of clues that our murder mystery dinner, well, that's a lot more information that needs to be processed.
  34. So the LSTM provides a solution to this long term dependency problem, and that is to add something called an internal state to the RNN
  35. node.
  36. Now, when an RNN input comes in, it is receiving at least state information as well.
  37. So a step receives the output from the previous step, the input of the new step, and also some state information from the LSTM state.
  38. Now what is this state?
  39. Well, it's actually a cell.
  40. Let's take a look at what's in there.
  41. So this is an LSTM cell and it consists of three parts.
  42. Each part is a gate.
  43. There is a forget gate.
  44. There's an input gate.
  45. And there's an output gate.
  46. Now the forget gate says, what sort of state information that's stored in this internal state here can be forgotten, it's no longer contextually relevant, the input gate
  47. says. What new information should we add or update into this working storage state information.
  48. And the output gate says of all the information that's stored in that state which part of it should be output in this particular instance?
  49. And these gates can be assigned numbers between zero and one, where zero means that the gate is effectively closed and nothing gets through, and one means the gate is wide open and everything gets
  50. through.
  51. So we can say, forget everything or just forget a little bit.
  52. We can say add everything to the input state or add just a little bit and we can say output everything or just output a little bit of output, nothing at all.
  53. So now when we're processing in our RNN cell we have this additional state information that can provide us with some additional context.
  54. So if we take an example of another sentence like "Martin is buying apples."
  55. There's some information that we might want to store in this state.
  56. Martin is most likely to derive to the gender of males, so we might want to stall that because that might be useful.
  57. Apples is a plural, so maybe we're going to store that is a plural for later on.
  58. Now, as this sentence continues to develop, it now starts to talk about Jennifer.
  59. Jennifer is, at this point, we can make some changes to our state data.
  60. So we've changed subjects from Martin to Jennifer, so we don't care about the gender of Martin anymore, so we can forget that part.
  61. And we can say the most likely gender for Jennifer is female and store that instead.
  62. And really, that is how we can apply this LSTM to any sort of series where we have a sequence prediction that's required and some long term dependency data to go alongside of it.
  63. Now, some, some typical use cases for using LSTM machine translation is a good one.
  64. And another one, chat bots, a Q&A chat bots where we might need to retrieve some information that was in a previous step in that chat bot and recall it
  65. later on.
  66. All good examples of where we have a time sequence of things and some long term dependencies.
  67. And had we also applied stem to a murder mystery dinner, we probably could have won first prize by having it forecast to us that who done it was the butler.
  68. It's always the butler.
  69. If you have any questions, please drop us a line below, and if you want to see more videos like this in the future, please consider liking and subscribing.
Subtitles generated on my computer; they follow the video — click a line to jump there.

What is LSTM (Long Short Term Memory)?

LSTM is an architecture that improves sequence predictions by enabling a neural network to remember long-term context and forget unnecessary information.

This video is published on youtube.com; the player here is YouTube’s. The Turkish summary and subtitles are prepared for AiPulse.
Watch on YouTube   AI Critique →
Back to the issue