Hi and welcome back to this course.
So we have done all the data processing phase and now we are ready to really start building our model.
So this will be the CNN that I'll talk about at the beginning of this spots.
And so the ID, if you remember well, it's just to have three different convolutional filters of size
two, three and four.
And then we will just take the max, concatenate everything and use it, then try to get our classification
done.
So let's start right now by creating the class that will correspond to our model and we'll call it deep
convolutional neural network, and it inherits from the turf that carries that model class.
So we start by doing what we always do when we create a class like that is that we need to define the
Init method, which is a part of any class like this.
And so this will always take care of yourself.
And what we will need for that is the vocabulary size that we had from our Ambedkar so that we can apply
it.
And then Bedelia for our CNN.
So vocab size, we will need the embedded dimensions of the dimension we want to have for our vectors
with our embedding layer that we will build here and train here.
So and building dimension equals, let's say one twenty eight, for instance, by default and the number
of filters.
So that's the number of convolutional filters for each size.
So we will have if we say for instance, that the default value is 50, we will get 50 feature detectors
of size to 50 feature detectors of size three and 50 feature detectors of size four.
So that's what this is about.
Then we need to use and units.
That's number of hidden units that we will use in our then Slayer's at the end, because the end of
the last feed forward new network, part of our CNN will be made of two dense layers.
And between those two layers, there is a hidden number of units.
This is this number, let's say by default, five, 12, then the number of classes for our classification.
That's a two by default.
This is here the number of classes that we actually have, the dropout rate.
So I will when we call dropouts, I will quickly explain what it's for and the training equals force.
We need to know whether we are on the training phase on that in order to know if we apply the drop out
or not.
And let's give a first name to our module with CNN.
So that's it for the parameters for the Init method.
And so we start any method that inherits from a class with super, which allows us to call a method
from the class we are inheriting from.
So we call super and it needs the name of our class and self as always.
And from super we call in.
It's that that means that we use the initialization methods from the 2.0, the model.
So what we just have to give it is actually the name that has been passed as parameter for our own model.
So.
Now we can start creating the layers that we will use in our model, so the first one, as for many
other model, is an embedding layer.
So as we explain in the infusion parts, we will get here as imputes simple tokens.
So each word has been transformed into a single number.
But what we want is actually to have vectors.
It's much more powerful and we will build those vectors right now with this embedding layer and the
parameters, the width of the embedding layer, he will be trained with the whole model.
So this is an embedding layer that is made specially for our model and for our datasets.
And we create it with self and vegging equals layers.
That's embedding.
So we just create an amazing layer and we need to give it the Capsis.
And the amazing dimensions so dimension we want for our victims, for our embedded victims, then let's
create our conventional layers.
So the first set of layers will be the ones of size to so the ones that will focus on two consecutive
words.
And we will call that, let's say, bigram, for instance.
And we say that it's a layer that's conventional, one deep.
So that's not a two convolutional, because as I explain in the intuition part, the width of our feature
detectors are exactly the embedding dimension of our vector.
So that means that we only shift it's that was vertically in the image.
So so we shift our feature detectors in only one dimension.
That's what we use convolutional one delayers.
And we need, of course, to give it the number of filters that we want of this size.
And that's no filters.
We need to give it the size of the Kernell size.
So that's two for the Bagram's then putting.
Will is valid, so what's this biting thing means is that when you have strides of size, more than
one, sometimes the last iterations of our feature detectors could get out of the max range of our metrics
of our sentence.
And that indicates how we want to handle this.
But we don't have to care about that here because our strides are of size one.
So whatever we choose, this will have the same result.
So, hey, we just use the standard value, putting you values and everything will work fine, just
as we saw in the intuition.
But we will just stride our feature detector by one from the beginning to the end.
That's perfect.
And finally, last thing we have to do is to use an activation function.
So after all the conventional computation that we talked about, we can if we want to add a function
to each element of our feature map, so each element of the output of the conventional phase, and usually
we use value.
So value is just a function that gets rid of negative parts of the matrix.
Each negative element will be set to zero and we keep the positive elements.
It's just really standard.
So we'll use this one equals really.
So that's it for the Bigram now, of course, we need to create the same thing for the convolutional
filters of size three and four, and in order to do that, we will copy that because we are lazy, because
we are computer scientists.
So all we have to do is to change the kernel size and of course, the name.
So instead of Bigram, we'll say Trigram and he four and we are lazy, but we still like pretty things.
So let's take care of the indentation right here.
Perfect.
Now we have to create the layer, which is basically a function that will take the max for each of those
outputs, as we said before, and this is done by creating cells.
That pool equals layers, that global max pulling one perfect.
And now let's go to the Feet Forward's neural network part.
And as I said, we will need to then Slayer's with a hidden number of units between the two densities
and let's create the first one dence one equals layer's that thens units.
So the number of output units of the first then of course is exactly 15 units.
The activation once again, who will use the standard really.
So that's it for the first layer.
Now we will apply a little bit of job out.
So what is the dropouts?
This is a tool that we use during training where we actually just shut down a certain number of neurons,
a symptom number of the competition power of our players, and that's mostly used to prevent our model
from operating intuitively.
What it does is that by shutting down some neurons, it forces all the others to handle the problem
by themselves.
So this helps keeping a sense of generosity and flexibility in our model.
Of course, each time the power is applied, it shuts down different neurons.
But that's the principle behind the drop out.
So let's create a drop out layer with layers that drop out and we need to give it the drop out rates.
So rates equals dropout rates.
That's just the part of the neurons that we want to shut down.
We can see here that by default, I decided to shut down a tenth of the neurons.
That's pretty standard as a value for that.
And finally, let's create our less dense layer that will have other outputs, a number of units corresponding
to the number of classes.
And actually we will do something fancy if we have two classes.
Instead of having two output units, we will have only one and will apply a sigmoid to it.
So that means that we will get a number between zero and one.
And if the number is higher than zero point five, it means that it's a one.
And if it's lower than zero point five, that means that it's a zero.
Otherwise, if you have more than two classes, then we will just have a number of output units.
That is exactly the number of classes.
And this time we will use a softmax activation so that we get probabilities for each class to be the
answer.
So let's write it with if no classes.
Equals two to then what we do is that we define last's dance as being a dance with.
One.
Output neuron and activation equals sigmoid that sits and if well, it just say else, then we define
self that lasts dense as being layers, that's dense with the number of units being no classes.
So classes and the activation function this time is different.
It will be this the that it's so perfect.
We have defined actually all the layers that we will use for our model.
And now what we have to do is to create the call function, the call method, which is what is used
each time we use our model to make a prediction or to train it.
So let's define cool, which takes as inputs always self and we get inputs and we get training.
So that's a boolean that says whether we are in a training phase or not.
And that's just to see if we want to apply the job out or not.
Because, of course, the job that is applied to doing the training so that we don't have a fit, but
we don't want to apply it when we are making predictions, of course, because we want to use all the
new ones.
We want to use all the power of our eye when we are making a prediction.
So if training will be false, then we want apply the dropouts.
That's how it works.
So first thing to do with the inputs is to call.
The embedding is to call the embedding.
So we will say that X equals Sayef.
That's embedding.
And we'll apply it to inputs, then let's create the first set of outputs from the first set of convolutional
layers with X1 equals self, that bigram be applied to X and then wants to apply the absolute maximum.
So we will call X one equals self that pool and we apply it to X one.
So that way for each of our 50 features, detectors of size two, we get one number, which is the maximum
activation for this particular feature.
So X one is actually a tenth of dimension batch size of course, and 50.
And let's do the same thing with X2, with self that trigram and we apply it to X and we apply of course
serve that pool.
To to and actually, I forgot that we were lazy, so let's not forget it this time we have pay for the
programs we applied to three and we call here the photographs.
So now that we have the whole embedding and conventional face that are done and all we have to do is
to concoct all the results and to apply our dense layers to that.
So merged will be equal to TV.com cuts.
And we want to concatenated X1.
Two and three, and we want to do that along the axis minus one, because, as I said, if you have
a look at let's take this one, the shape of it here, we get no, not this one.
With Ackermans, we get shapes like batch size and no filters.
That's the shape of our Tensas, right.
That's the same thing for X2, of course, and four x one.
And so by doing this, we concatenate all those vectors right here along the last dimension.
And what we get is something of shape, batch size, three times no filters.
And so now we are ready to apply our first layer.
So we'll say that much equals self, that dense one and the two merged.
This is the moment where we want to apply our job.
Also merged will be equal to self.
That's dropouts applied to merged.
And we say whether we are in a training phase or not.
And finally, of course, the last thing we have to do will actually call its output will be self that
lasts.
Dance applied to our much perfect.
Now, of course, what we have to do is simply to return our outputs and our model is ready to be used.
This is done.
We just have to run this and to check if there is a typo.
There is one here.
And hopefully perfect's this works fine, so we have built and defined our model, our conventional
neural network model.
Now we are ready to train it to create an instance of this class and objects and to use all data sets
to do the training phase.
So that's actually the topic of the next lecture.
So let's go into it.
And Tucson.