Hi and welcome back to this broadcast.
So we are finally at the last step of this application, which is to evaluate the performance on this
squad's task.
So with our previous lecture, we were able to actually create and write many different files.
But the one that we are interested in, of course, is prediction that Jaeson.
So this should be in your Google Drive folder that contains all the data we were working with so far
and squads.
And so you have to just.
Well, right, click and download this file once you have it.
Just put it in the look out for the well.
You also have the theft data sets that Jason file that you downloaded at the beginning.
And the other file that we need to have that is really important is this script evaluates version one
point, one that pivo that is provided by squads.
And actually when I tried to download it from the squad website, it was not in the right format, it
was the whole HTML page with just the scripts inside it.
So I had to do some modification.
This is the right file and I will make it available for you in the resources of the course, so you
should pick it there.
It's much easier.
And what we have to do now is really simple.
We have to execute these scripts by giving it as arguments the path to this file and the path to our
predictions.
So that's very simple.
We have to do is to call Python or Python three.
It depends of your default Python version on your computer.
Then we want to execute evaluates version one point one to pivo.
And as I am already in the folder that contains all the files, I can just call the dev dodges and file
this way.
And same thing for the predictions that Jaeson.
So we just have to run that and we will very quickly get the results.
And so that is this F one number.
That is actually the accuracy score that is used in squads.
So you can see that it's pretty good.
So that means that I worked, because when you consider this problem using random answers or having
an eye that doesn't really work and has not trained basically means a score close to zero.
So that's good.
That's not as much as what Berts expose in the paper.
I think if I remember well with that base, they achieved a little bit more than 85 percent.
But that's completely normal because if you remember, I only used a tenth of all the data because it
was more convenient to create an example, to train it and to debug it.
So that's actually pretty good to go that far.
Seventy seven percent using only a tenth of the training data.
So basically our model works fine.
And if we were to use all the datasets and to do two or even three Abebooks on all that, that will
take long, of course, but we will achieve pretty much the same results as they did in the paper.
So that's it.
That means that our model of fine tuning, but for an actual, quite complex task works really well.
He of course, we used a benchmark as data sets and to evaluate it.
So we used the outputs in an almost esoteric way in order to get our results and to get a score to evaluate
our model.
But that means that now we perfectly know how to create a model using birds using its outputs and cleverly
applying a Denslow afterwards to get usable final outputs that fit our task.
So I hope you had fun using birds and fine tuning, but for this particular case and that you will find
interesting and powerful way to use it for your personal problems.
So enjoy using birds and see you soon.