Understanding Transformers Part 9: Stacking Self-Attention Layers

In the previous article , we explored how the weights are shared in self-attention. Now we will see why we have these self-attention values instead of the initial positional encoding values. Using Self-Attention Values We now use the self-attention values instead of the original positional encoded values. This is because the self-attention values for each word include information from all the other words in the sentence. This helps give each word context . It also helps establish how each word i