Understanding Transformers – Part 16: Preparing for Output Prediction with Residual Connections

In the previous article , we handled values in encoder-decoder attention, now we will simplify the diagram a bit add another set of residual connections. This allows the encoder–decoder attention to focus on the relationships between the output words and the input words, without needing to preserve the self-attention and positional encoding from earlier. Lastly, we need a way to take these two values that represent the token in the decoder and select one of the four output tokens: ir, vamos, y,