Understanding Transformers Part 17: Generating the Output Word

In the previous article , we set up the residual connections to get the final output values from the decoder. In this article, we begin by passing these two output values through a fully connected layer. This layer has: One input for each value representing the current token (in this case, 2 inputs) One output for each word in the output vocabulary Since our vocabulary has 4 tokens, this gives us 4 output values. Selecting the Output Word Next, we pass these 4 output values through a softmax fun