Scraping data from a website

[rdekanter]

Hi,

I'm trying to write a script to pull down the company score from Glassdoor. I came across an example while trying to research this and have modified it as per the code below.

The trouble is, it only appears to work once then on subsequent runs it is no longer able to set the hDiv object. Is this down to something I am doing wrong, or is it just that the webserver is refusing the conection a second time around? Is there any way to avoid this?

All I want to do is load a page, search for the score and then extract it.

thanks for any help.

Sub GetGlassDoorData()
     ' Declare variables (needs reference to the Microsoft HTML object library)
     Dim html As HTMLDocument
     Dim hDiv As HTMLDivElement
     ' Get html from the site
     Set html = New HTMLDocument
     With CreateObject("MSXML2.XMLHTTP")
         .Open "GET", "https://www.glassdoor.co.uk/Overview/Working-at-Google-EI_IE9079.11,17.htm", False 'example page for Google
         .setRequestHeader "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT"
         .send
         html.body.innerHTML = .responseText
     End With
     ' Get company recommendation
    Set hDiv = html.getElementById("EmpStats_Recommend")
    Debug.Print hDiv.innerText
    ' Tidy up
    Set hDiv = Nothing
    Set html = Nothing 
End Sub