Hi and welcome back to this birth course, so let's go into the intrusion parts of births and we will
start with a very little introduction video and this video will be about basically what is Berts?
So we will use the definition of birds.
So what the be all and t stand for to basically get a first general understanding of what it is and
also to give us a guide through the different steps to understand the theory behind birds.
So Bird stands for bidirectional and coded representations from Transformer's.
So that might seem a little bit esoteric of course, but we will go into the details.
So I like to split it into three parts.
The first one would be the encoder representation.
So what does it mean?
It means that we use encoder.
I will talk about that later.
You will see what it means to get a representation system of the language.
So that means that Bird is a language modeling system that is trained with unlabeled data.
So birth is not made to tackle a specific task to answer a specific problem.
What it does is that it gives the best, the most efficient and the most flexible representation for
words and sequences possible.
And then it's our job to get the most out of it, for our problem, for our task.
That is what we mean by language modeling system.
We give a sentence or two sentences.
We will see that later.
And out of it, we just get a list of vector or a single vector that we can really simply use for our
needs.
And that is why we say that birth has been trained.
Actually, this whole language modeling system is already available.
It has been trained before with a huge corpus, the huge processing power, and all of time also.
And this is now available for everyone.
So Google built the transformer, the trainings and then the weights.
The whole architecture is available for everyone to use it and to apply it however we want.
And this applied however we want.
This is what we call fine tuning.
So we take that that is already trained and we add a small layer that is made for our specific task
and then we train the whole thing.
But very likely and this is really fast train and we don't need a lot of data because the biggest part
and the most important part has already been done with the core of the module, which is bits.
So that is for the encoder representations and that will be the first part of our intuition journey,
which is to understand in more details of these three training, then let's go into this from transforming.
This justifies the architecture of.
But this just means that in order to build birds, they used the transformer that actually Google developed
some years before.
And the transformer was a really powerful NLB algorithm that was designed to tackle many sequences to
sequence tasks so you could easily build a translator or a chatbot or whatever with it.
And it was very powerful.
And they thought that maybe by taking just a small part of it and using it in a even smarter way, they
could get good results.
And that's actually what they did with bots.
So that will be the next part of our intuition journey, which is the architecture of birds and finally
the bidirectional parts.
What it means is that most of the time when we do end up, we want to predict the next words in our
sentence, whether it be doing training or doing making a prediction or answering a question or whatever.
What we do is mostly predict the next words.
And so usually to predict would we only have access to the left part of the sentence?
Or sometimes we have access only to the right part, or sometimes some model for training let or would
have access to the left parts and the right parts separately and then concatenate both.
So it's pseudo bidirectional but hey, but achieves to have a fully bidirectional model, which means
that to predict Attwood's, it had access to the whole context, the whole sentence.
And that's much more powerful.
And that is actually what defined the training process outputs and more precisely the pre training process.
And that would be our next step for the intuition part.
So now we basically know what we are dealing with.
We know what will go through to understand.
But so let's go into it right now with the general idea behind the.