Webread Fails to Get All Data

I am using webread to extract tabular data from a website; however, only part of the data is returned. The problem occurs because the website delays in loading the table, returning the mesage "loading additional data" at the bottom of the table until it fills. The webread function thinks that the website has finished loading and terminates the bytestream.

Is there a workaround to this problem, or do I have to develop my own webread function that will wait for all the data to load?

[Tarunbir Gambhir]

It would be really helpful, if you could provide the URL to the website with that table you are referring.

[Adam]

For example:

https://finance.yahoo.com/quote/AAPL/history?p=AAPL

If you load the site on your browser and scroll down quickly before the table fully loads you will read "loading more data..."

Answer 1

[Tarunbir Gambhir]

webread() is typically used to access a RESTful API but it is also able to download the HTML source of a website. However, content that is dynamically loaded and not present in the HTML source will not be downloaded. A possible workaround is to use the website's API (if one is available) to collect the data and import that into MATLAB.

This is a current limitation of the webread() function and I have brought this issue to the concerned people and it might be considered in any future release.