Understanding Attention Mechanisms – Part 6...

In the previous article, we obtained the initial output, but we didn’t receive the EOS token yet. To get that, we need to unroll the embedding layer and the LSTMs in the decoder, and then feed the translated word “vamos” into the decoder’s unrolled embedding layer. After that, we follow the same process as before. But this time, we use the encoded values for “vamos”. The second output from the decoder is EOS, which means we are done decoding. When we add attention to an encoder-decoder model, th