Understanding Transformers Part 15: Scaling and Combining Values in Encoder–Decoder Attention

In the previous article , we gained an understanding how much each input word contributes, in this article we will start to compute the value vectors for each input word and combine them accordingly. We scale those values using the Softmax percentages , and add the scaled values together to obtain the encoder–decoder attention values . The sets of weights used to calculate the queries, keys, and values for encoder–decoder attention are different from the sets of weights used in self-attention .