Understanding Transformers Part 3: How...

In the previous article, we learned how positional encoding is generated using sine and cosine waves. Now we will apply those values to each word in the sentence. Applying Positional Encoding to All Words To get the positional values for the second word, we take the y-axis values from each curve at the x-axis position corresponding to the second word. To get the positional values for the third word, we follow the same process. Positional Values for Each Word By doing this for every word, we get