Fetching "Inspect Elements" from HTML

Hello

So i dont know if this can be done or not, but i have not been able to find any posts about it. What i want to do is fetch information that changes in diffrent elements on a HTML page. To simply download the page source using URLread gives me nothing since the values are not stored there. It might just be my lack of knowledge about HTML that makes me ask this question, but so be it.

Thanks in advance

[Cedric]

Could you provide the URL of a few of these pages, and explain what you are trying to extract?

[Anton]

What i am trying to extract is odds from a betting site. I was asked by a friend to plot over time how the odds changes before the game starts to find a "sweet-spot" to place the bet.

Here is the URL i've been using: https://svenskaspel.se/?pageid=/sport/oddsetlangen

Its a Swedish betting site, if you wonder why its filled with jibberish.

Answer 1

[Guillaume]

If the data you're interested in gets displayed through javascript then there's not much you can do short of implementing your own web browser (at least the javascript interpreting bit) in matlab or interacting with an existing browser. See this related discussion.

Possibly, the website you're accessing may have an API to fetch the data you want, most likely using xml. In that case, you could use webread.

[Anton]

Thank you.

I've tried the first method you gave but could not get it to work. I get the webpage to open, and i get a handle with information. None of this information is of any relevance anyway - so i suppose that im doing something wrong or it can't be done this way.

I might just use image recognition to read it instead, since thats something i have more of experience with.

[Guillaume]

The page you've linked does not render properly for me at all. It may be because it has some flash content and I've not got flash installed.

If the content you're interested in is indeed served by flash, then it's even more complicated to extract. You'd need to implement a flash renderer (or find an implementation you can use).

I don't speak any swedish so can't really look at the website, but the first thing I'd do is search to see if there's an API to query it.

Failing that, then yes, image recognition may be the fastest route.

[Cedric]

Image recognition will not work too well either, because you have to open pages manually, take a screen shot, etc, so this cannot be done on a regular and fast basis.

If I were you, I would try to see if I could access the content after the JS (or whatever else) is execute, from e.g.

web( 'https://svenskaspel.se/?pageid=/sport/oddsetlangen' )

or from Firefox. If you open the page in Firefox, right click on one of the numbers and select "Inspect element (Q)", you will see that the inspector tool displays the value. There may be a way to get this information from some plugin rather than the raw HTML source which contains the call.

[Anton]

I've sorted out to open the page, screen shot it and crop out the diffrent odds, so i guess is should be doable with image recognition.

Apparently the built-in browser in matlab cant inspect specific elements which is a shame really - or at least i cant figure out how to do it.