Hi and welcome back to this birth cause so now in this section, we will create our own training loop
to train our model so that we have more control and everything is more flexible.
So the way we will do that is, of course, we will go through each epoch with four epochs in range
in the epochs.
That's it.
And so we will first print that.
This is the start of Epoch.
And the.
Curly brackets.
That format's.
Took.
Plus one, of course, because this starts as at zero and we wanted to start at the first, at least
this is what we want to print.
So that's it.
And then we will just create a little bit of Staats equals time that time.
So this will just give the time where we start the airpark and we will be able at the end to print the
time that it took to go through the whole Abebooks.
So.
Let's reset the train laws so that we can keep track of the laws to one whole epoch, that said, now
we are ready to go through all the batches.
So full batch and input's targets.
In enumerates.
Train data sets light, so what we did here is with enumerator, we talked about that before, in the
course we can get access to the number of our converge and here we get what is in our data set.
So in our data set, we have each element being a tuple made of the inputs and the targets.
And also, as we see actually the inputs is a dictionary containing the three different tokens that
we use and also the ID corresponding to the question in squads.
Yeah.
So we have we call those supposed to have separately the inputs and the targets from the data sets.
And we will start by creating an environment of type to create and tape that we will call tape, and
that will allow us to know exactly what happens in the weights when we call birds and when we compute
the lost function.
And so all those informations about how everything is computed will help us compute the gradient and
apply the optimizer.
So let's do it right now with with T.F. that gradient tape as tape.
So what we will do in these environments is, of course, call our bird squads.
Model outputs will be equal to that squad, apply to inputs and we will compute the loss also.
So loss equals squared loss function and we will give it the targets and the model outputs.
So that sits inside these environments.
Now we can use tape to compute the graduates so gradients will be equal to tape that graduates so we
can compute the graduate of the computations that we witnessed just before.
And we will do that using the loss and knowing all the training parameters.
So the trainable variables from that quote.
So Quad's that trainable variables perfect.
Now that have our gradient, we can actually use it to optimize our model with optimizer.
That apply gradients and what we need to give it is a list of gradients and viable pairs, so the way
to do that is to zip gradients and butt squat.
That's trainable fireballs.
OK, so now we gave to the optimizer the list of the best gradients and weights that we want to train
so it can optimize its and modify the weights of birds and of our squad layer so that we decrease the
loss.
Now let's just add a new element to our chinless, which keeps track of the laws doing the whole training
process by adding it's the loss.
And finally, let's just say that if Buche is can be divided by 50.
Then we will, in this case, print that we are in the book.
No curly brackets and the budget, no, we will print it, too, and we'll also print the most important
thing, which is the loss.
So we will print it with Cullen.
That's for.
So that just means that when we want to print a float, no, we only wants to keep four decimals because
the loss could be a no, not a huge number, of course, but a number that uses a lot of digits to be
written because you can have a lot of decimal values.
So writing that, we are sure to write only four decimal values.
So that's it for what we want to print.
We specify formats and we print epoch plus one.
Batch and train lost that results.
So how well I perform so far in the airport or more precisely how bad it performs as we want the laws
to be the lowest possible.
OK, let's do another thing, if this time budge can be divided by five hundreds, and what we do here
is that we will actually save our model, so we will use our check point manager doing KBG Save Path
equals Kapiti manager that save.
And we'll print the fact that we saved our model, so for instance, saving checkpoint's for Époque.
At and here we will put the path so format's.
Époque.
Plus one and keep Kapiti save, but the reason I do that is that training can take a lot of time for
this problem and sometimes the connection to your Google Club server is lost for an unknown reason.
And if this happens at the end of a network and you lost hours and hours of training because of that's
that's quite sad.
So I like to save my model from time to time.
And I decided that 500 lashes was pretty good for that.
So finally, once we finish network, we can just print time taken for one epoch.
And this will be in seconds.
Let's just put the backslash at the end and we use format time lapse time, so that's the current time
and we of course, we move the time when we started the epoch.
So we get exactly the time it took to train one EBBOTT So that's it for our custom training loop.
I run it, but of course we won't wait for any results because it takes hours and hours, even though
I reduced quite drastically the size of our training sets.
So once this is done, we can go to the evaluation phase.
So Susan.