The important things about locating, interacting, and basically working with the elements inside a
HTML application or website.
This is generally called Dom, and we have done all of that.
Now in this episode, we'll be learning how we can basically use JavaScript inside this thing, because
fundamentally, JavaScript can run with a browser.
Browser has its own JavaScript, you know, local server.
And normally to use JavaScript you would either need a browser or you would need a normal server.
Uh, for example, like node server, npm, node package manager, right?
Or any other things.
Now we will be generally using the application of a browser's pre-existing, uh, you know, web browser
console.
Uh, inside a web browser, for example.
So if we are looking here at this web browser, then if I go inspect and there will be an option called
console where I can execute my JavaScript commands.
For example, if I want to print something, then I would say console dot log, hello world right?
You can see it's executed.
If I want to assign a variable, I can say var a equals ten, a is ten, right.
So stuff like that.
So we will be using that particular these commands inside our driver.
So let's say I got um I don't know maybe let's go to Yahoo itself.
So https colon slash slash www.yahoo.com.
So let's go to yahoo.com.
And let's time dot sleep for some time maybe 100 seconds.
And let's try with dot close.
So let's close that driver.
And let's try to run this application and see what kind of JavaScript uh command we can run.
So let's see.
Just open the Yahoo.
All right.
So let's now see what we can do.
So maybe what we can do is uh, we can, you know, select if I want to select certain things with JavaScript,
then what I would do is I would get the ID first of this.
So the ID is this one.
So I can get elements.
So I can say document dot get element by ID.
And I can pass in the ID and I will basically get that element.
And I can say dot value equals some value.
So test then you can see that thing is done.
So let's execute this command right here.
So if I want to execute a uh JavaScript uh code snippet I can simply say execute driver dot execute
underscore script.
And inside this I can pass in that particular command.
And using this we can basically run that command.
So let's see how this runs.
So let's stop this and let's see.
So it opened the Yahoo.
And then you can see the JavaScript command was run.
Now let's try to also um print uh what is known as let's go to finance.
And let's say I want to get the Nasdaq current value.
So if I want to do this I would want to firstly go to the Finance.yahoo.com.
So let's just paste it right there.
And let's say I would want to console.log this thing, this particular value.
So what I can do is I can just go here, get the XPath.
So uh sorry.
Yes.
Expat is not possible in the we have to evaluate which is a bit of complex process.
So instead of that let's say I would just go to a top most actives okay let's go to the most actives
and let's see this particular here.
So data field class don active value I think this these these things would be a bit harder to track.
So let's just try with Google maybe.
Let's say covet.
Yeah.
So let's say inside the Google.
Uh, we are trying to get this, uh, print this thing.
So if I inspect so ID is this.
So let's just copy this ID and let's.
Go to google.com, w-w-w dot google.com and we are gonna just use that ID here.
And I'm just going, going to console dot log this value.
So I can just say that and I can say console dot log.
So if I do this then in the console I should be able to see that particular text.
Let's try.
All right.
So let's see the console.
And it just printed undefined because I think that value was not given.
Uh, there.
That's why I think it gave us that the undefined value.
Let's check.
Yeah, exactly.
Um, so I think let's try here with this, whatever we can try to get maybe the Google inspect.
We're using value because yeah, it does not have a value, but it has a text.
So let's try this because right here if you see the that element is there.
But we try to print the value because that uh element didn't have any attribute called value.
It has a text.
So let's try to get the text once dot text.
Um.
I think this should be because the text is inside the A.
So what we can do is so that element dot a dot h ref.
Dot.
A.
Should be possible to get because, um.
Um, okay.
So it just, uh, stopped.
I think it should be inner text.
Let's try with inner text in, uh, text.
I guess it is like that in HTML or text content.
Let's try with, uh, inner HTML first.
And if it doesn't work, we'll try with text content.
Okay.
So let's try to inspect console.
So you can see there is that inner HTML.
So whatever was inside that div we were just console dot logged into it.
If I wanted like the text content then I could try with text content also.
So let's try that.
So it should have consoled it.
Yeah.
So it just gave me only the text, right?
Not the HTML.
It didn't give me this link, but only the, you know, the inner text content.
So yeah, that's how you can run a JavaScript commands inside your selenium driver.
And I think that sums up our selenium course, part of this whole web scraping tutorials.
Now, I think the next thing we'll be doing is going to be on beautiful soup.
This selenium is again we talked about it already.
This is about automating the web elements.
So clicking you know activities.
And on the other hand if I say beautiful soup it's more about scraping the data getting the data.
It cannot do you know functional activities like dynamic like clicking and stuff like that.
Beautiful soup.
But it can get the data here.
In this case, it is quite easy to click and interact with the elements, but it's a little bit hard
to get the values out data out.
So whenever you you need to interact with the, you know, web element, you can use selenium whenever
you want to interact with and get the data.
Then you can use Beautifulsoup.
All right.
So let's meet on the beautiful soup then.