Hi and welcome back to this Senate because so we have seen how we used to address the sequence to sequence
task by using Arnon's and how we improve them by adding this attention mechanism.
That was a very powerful tool, but it seems that we might be able to go even further in that direction.
And the reason why we needed to go further and to even more improve it is because the thing about dominance
is that it's a sequential process.
So it doesn't have a global behavior concerning the input sequence.
And so we lose information along the encoding process.
The fact that the large hidden state of the encoder, which is the final output of the encoder, has
not seen the beginning of the sentence for a long time, could be quite disturbing.
So for very long sentences, very long sequences, we lose a lot of information and that's a huge weakness
for our audience.
So what the Google research teams have been thinking about is, OK, so the attention mechanism added
global behavior to the coding phase, but the audience still has this weakness about not being global
enough.
So what if actually all we need in order to perform well in this sequence to see process is attention.
And so all we need is attention is actually the name of the paper that Google released when they introduced
the transformer.
And that just illustrates the fact that the transformer only uses the attention mechanism in order to
encode and decode sequences and completely get rid of the Arnon's.
So let's have a look at the architecture figure that they provide in this paper.
So there is a lot of stuff going on here.
We will go, of course, into the details of each step of each layer later.
But let's start by trying to have a global understanding of what's happening here so we can clearly
see as before that we have two main blocks.
The first one on the left will be the encoder and the second one on the right will be the decoder.
There are some ideas that stay the same, like, for instance, the fact that we have input for the
encoder.
And after that we get outputs from this encoder and those outputs will be part of the inputs of the
koja.
And from that we will get our output sequence.
So this doesn't change.
But one of the main things that are different is that our inputs for the encoder will be the whole sentence
at once.
So we don't feed the encoder one word at a time.
We just give it the whole sequence and it will process the whole sequence at once.
And this is important to keep the global aspect of the processing that we wanted to have.
Second thing that we want to notice is the inputs for the decoder.
You can see that's here to say outputs.
It's actually because we use the outputs of our decoder as new inputs as we did before.
But once again, we do it with the whole sentence.
So let's say, for instance, that all transformer has been made to build a chatbot and the input sequence
will be how are you doing?
For instance?
So first time we apply our transformer, we will have as inputs for the encoder, the whole sentence,
how are you doing?
And we will give here as inputs just a single token that we will call the starts talking.
And it doesn't convey any meaning.
It's just to tell the transformer that this is the beginning of a sentence and giving that after the
first application of the transformer will get as outputs just one element that will be AI.
So the first word of our answer, and we will add this word, I do the output sequence that we had before
and this will become start talking AI then will repeat the process, keeping the same inputs here.
So how are you doing?
And he we will use as inputs for the decoder start to can I and we will get as outputs.
I am for instance and repeating this we will add new words to the output of the decoder to finally get
something like I am fine thank you.
This is where this shift to the right comes from.
It's just that by adding the start talking at the beginning of the input sequence for the decoder,
you shift the whole sentence.
Right.
And with that, we know that the last word from the output sequence here will be the predicted next
words for our sequence.
But that's an implementation detail and we'll also see that later.
So that was the first thing to have in mind, the fact that we use whole sequences at once as inputs.
And that's key.
The sequence we use as inputs for our decoder is the one that we already had at the previous iteration,
plus the new words that we will predict at the end of this sequence.
Right.
He said something that is really important when we have a close.
Look to the architecture of those orange cells right there where you can see attention everywhere,
and that's the most important thing about this transformer.
That's actually what we'll do, the whole job.
And that is exactly based on the attention mechanism that we saw earlier applied to our.
So just to summarize how it works.
The idea is when you have two sequences, it's just we compose the first sentence according to how each
element of this first sentence is related to the elements of the second sentence.
But you might be wondering then why here are the inputs.
We use three times the same sentence, and that's actually what they call self attention, which is
the key of the encoder in the transformer.
So what we did before with the Arnon's is that we used a standard neural network.
We had information about the beginning of the sentence.
We are the new words.
We make some computation and we get new information about the sentence.
But what we do here is that we have a whole sentence and we will see how each word of its sentence is
related to the other words of the same sentence.
And then we will recompose this sentence according to that.
So.
So here is, for instance, an illustration of the self attention mechanism, so if we take the first
sentence, the animal didn't cross the streets because it was too tight.
When we apply the self attention mechanism, it just computes how each words of the sentence is related
to the others.
So here it understands that the it right here refers to the animal.
It's the animal that was too tired.
But if you take another example, like the animal didn't cross the streets because it was too white,
we understand that it refers to the streets because it was the street that was too white, of course,
that the animal and the self attention mechanism understand that.
So.
If we fit this sentence to a self attention, Layo, like we have in the quarter, we will have as inputs
just this sequence of words, each word being, of course, doing the implementation of Vector.
And as the outputs of this attentional layer we will have here instead of the word it, we would have
a combination of those words according to how strong the relation with the words it is.
So we will have a lot of the vector animal a little bit off of streets of the maybe it's this point
and we will have nothing from the other words.
And for this sentence instead of it as outputs of our self attentional layer, we will have a lot of
the vectors tweets and a little bit of those lighter blue ones and none of those ones.
So it's just that instead of having just a sequence of words, we have a new representation of the information
that can convey a sentence, because as inputs, we have the lowest possible way to have information
about a sentence, which is just one word after the other.
And the goal of applying the self-protection mechanism is to combine everything so that each element
doesn't only represent the words, but it represents relations between the different words of a sentence.
So that was kind of an illustration of how self attention works and the self attention mechanism is
how we will encode sentences in the transformer.
So that is used, of course, in the encoder.
And that's the main thing that we use.
We also use fully connected network, which just to learn more, we apply this whole thing several times
and we get our final encoded version of the input sequence and about the G, we start doing the same
thing.
So with a sequence as input here we apply a self attentional layer just to have more information about
the relations between the words.
And after that, we apply an attention mechanism between the output of the encoder and the sequence
that we just had.
So that means mixing information from the encoder according to how each element is related to this sequence
from the decoder.
But we'll see that into more details later when we will go deeper into the attention mechanism.
One last thing I want to show you a small illustration that could help understanding how it works is
this latest version I found that's kind of show an animation of how everything works.
So this is the encoding phase with the input sequence.
And you can see that what happens is that we mix all the words together, all the relations to get new
representation of the input sequence.
And then, as I said, we start the decoder.
We just want token and we apply several times the.
DeCota.
With also the information from the encoder in order to have a new word at each step like this.
Let's have a second look.
So let's get our.
Whole input sequence right there and makes each word.
According to how they are related to the others and get new information and we repeat that several times
and get a new representation of the input sequence and yeah, now we just apply the decoder.
We are the new one each time.
And using the information from the previous words and the whole encoding sentence, we can little by
little, step by step.
Predicts a new word.
And finally, just get our final answer, so she the answers, the sentences are not complete.
Of course, it's just an illustration, but I find it pretty cool and it's and it's easier to understand
what happens this way.
So that was it for the general understanding of the transformer.
Of course, we will go into more details for each element of this model.
But to summarize, what we have to understand here is that there is still an encoder and decoder and
we compute whole sequences at once instead of one word after the other.
And the way we do that is by re composing sequences, making sounds of different items from each sequence
according to how they are related to each other.
So that's just a way to get global information about the whole sentence.
Just an interesting thing actually, that in my opinion would be how the brain processes sentences.
When you have a sentence in mind, we don't process the meaning of each sentence by going word after
the other.
We actually have the whole sentence in mind and we capture the meaning of it at once.
And if you think about it, that's actually easy to see.
When you think about reading.
I don't know if you like me, are a slow reader, but if you read slowly, you may experience that you
have to go back many times when you read a single sentence.
And that's not only because you maybe are not concentrating enough.
That's also because if you read too slow word afterwards, it's not how your brain is optimized to understand
a sentence.
When you arrive at the end of the sentence, you have already forgotten the beginning and you are losing
the whole sense of a sentence.
Actually, a sentence is made to be understood globally as one big book.
Actually, it's kind of a burden that we have to use words in that we have when we read or when we write
to get them one at a time.
Because if you think about that, you might realize that when you hear someone speaking, you wait for
the end of a sentence and then you get the whole idea of the sentence at once when you process the meaning
of it.
And that is actually what first readers do.
Those people that can read a page in a few seconds and have trained a lot to improve their reading speech.
What they do instead of reading word by word, is that they try to use the global vision to get the
biggest group of words at once that they can and to process the whole meaning of it at once.
And you can do that because your brain is made to understand language like that.
And so my understanding of this transformer is that it goes in the same direction.
Instead of processing word by word, we try to capture the meaning of the whole sentence at once by
combining each item with the others.
And I find it so interesting that we naturally get to those kind of models and that they are often the
ones that we used before.
So that's the general understanding of the transformer.
Now we can go into more details and see exactly how the mechanism works and how they use it and all
the other details that they used to make it even more performance.
Hope you're excited to go into those details and see you soon.