In this video you can actually see my face apologies by the way if you don't like it don't worry trust me you're not the only one and my face isn't going to appear for.
The entire video just for this short introductory section the reason we're doing this by the way is because we now have multiple wise owls making videos.
So we thought it would be nice for you guys to be able to tell at a glance which wise owl you're getting when you first start watching we're treating it as an experiment so feel free to let us.
Know in the comments below whether it's working or not we'd love to hear what you think anyway on to the question that's been asked it isn't actually the same.
As the title of the video in this case the original question came on a video about how to find web elements in selenium and what jay wanted to know was why is.
When you first load the page not all of the content gets loaded at the same time it waits for you to scroll down the page until later content gets loaded.
It's a feature you'll be familiar with from sites like facebook for example where you scroll downwards and more content loads the further you scroll i'd show you the example with my.
Facebook newsfeed but the embarrassing lack of a social life would make that fairly pointless so instead i'll just show you the site that jay is interested in scraping.
So this is the particular link that he gave me um so you can tell here from the scroll bar actually if i just start scrolling down slowly by clicking on the down arrow.
You'll see that the height of the scroll bar once i reach a certain point changes indicating that there we go the page has just loaded a whole bunch of extra content and it's the table that's.
Sitting down towards the bottom of that
Page that jay was interested in scraping so we need some way when we load that page to scroll down until we get that table.Loaded so we'll look at a few different techniques we could use in this video to demonstrate some of the basic techniques for scrolling a web page we're not actually going to start with.
The sofascore.com website we're going to start with a much simpler site called the internet and the infinite scroll test page on there if i right click into one of these.
Blocks of text and choose to inspect it we'll see in the elements page we get a list of div elements with the class jscroll dash added and there are two of those div elements at the moment.
The further down i scroll through this page however the more of those developments get added and you can see those appearing in the elements list so i've already written some basic code.
To get this to work to fire up a new instance of chrome and navigate to this page the code i've written relies on the videos in the getting data from websites.
Playlist so if you're not already familiar with the basics then this is a good place to start the workbook itself is basically empty there's a single module in there with a.
Simple subroutine and all it's going to do is load a new instance of chrome navigate to the infinite scroll page wait for 200 milliseconds.
And then capture a reference to all of the elements with that class and write out the number of them into the immediate window so if we just run this subroutine once.
And
Have a quick check back in the imager window when the page is first loaded we get two of those div elements one very simple way to scroll down the.Page would be to send the page down keyboard shortcut to the chrome browser so if i head back to the open instance of chrome and press the page down key you can see it scrolls down the page and.
Loads more of these div elements so to do that in vba let's head back to the visual basic editor we'll need a new variable to hold a reference to an instance of the keys.
Class so let's say dim ks as selenium dot keys we'll then need to create a new instance of the keys class which we can do in a couple of different ways.
I think i'll do this just above the point where we set our chrome driver let's say set ks equals new selenium dot keys what i'm then going to do is after i've.
Debug.printed the number of developments i've found i'm going to say cd.send keys and then say ks dot page down just once should be sufficient.
To load a new div element i'm then going to make the driver wait again so i'm going to say cd.weight and i'll go for 200 milliseconds again.
And then i'm just going to do the same things i've done here i'm going to capture a reference to all of the div elements with that class and then all of the elements with that.
Class and then count how many there are so let's copy and paste those two lines and then let's say divs when page scrolled one time okay so having done that we can clear.
The contents of the immediate window and then run the subroutine and this time we should see the page loads and then it scrolls and it shows us that we've.
Loaded the page with two elements scrolled and loaded as an extra element so we have a total of three so the page down method works well enough but it is a little inconvenient.
Having to declare this extra variable to hold an instance of the keys class so as an alternative we could execute some javascript to achieve the same end results.
There are several methods in javascript you can use to scroll a page if i just head over to the mozilla website i've brought up a few of these help pages which describe the.
Window.scrollbuy method the window.scroll method and the scroll to method there are several other scrolling methods available as well but they're.
Non-standard and not supported by every browser so i'm not going to cover those i'm going to go back to the scroll buy method this one allows you to scroll.
The documents in the window by a specific amount so it's like a relative scroll it doesn't scroll to a particular position but by a number of pixels basically.
So i'm going to just copy and paste a bit of code here to save writing it out myself basically i'm feeling a bit lazy so i'm going to copy this window.scrollby.
Zero comma window.inner height so the zero is the horizontal scroll and the window.inner height is the vertical scroll that could just be.
A hard-coded number but this will scroll by basically one page height using that property so i'm going to copy that to the clipboard head back to the visual basic editor and.
Then i'm just going to comment out the send keys method and then i'm going to say cd.executescript and then after a space in some double.
Quotes paste in what i've just copied from the mozilla site so window.scrollby zero comma dot inner height it's quite important to remember that.
This code is case sensitive which is why it's kind of useful to be able to copy it from the mozilla site so the capital b there is important the.
Capital h there is important so be careful about that um i'm just going to run this subroutine now just to prove that it works first of all or.
Check that it works i should say there we go so we've seen that it's loaded and it has scrolled down it did it a lot faster than the send keys method but you can see that we.
End up with the same end results with the javascript method there's also less less need to wait if i just go back and comment out the cd.weight.
Line i'm going to run this subroutine again i'm just going to comment sorry clear the contents of the immediate window so let's run this again with the javascript.
Method to scroll by you'll see that the page loads and almost instantly scrolls down by a page and if i have a look back at the end results here you can see that i get.
Two div elements when the page is first loaded and three immediately after it's scrolled but without having to wait if i just comment out the javascript.
Method and then bring back the send keys method you'll see this time if i clear the contents of the immediate window when i run the subroutine the scrolling.