Understanding Transformers Part 11: How Decoding Begins

In the previous article we wrapped up the encoder part, In this article, we will start building the second part of the transformer: the decoder . Just like the encoder, the decoder also begins with word embeddings . However, this time the embeddings are created for the output vocabulary , which consists of Spanish words such as: ir vamos y token Starting the Decoding Process To begin decoding, we use the token as the input. This is a common way to initialize the decoding process for an encoded s