Issues reading updated HTML page content using MATLAB

I am writing the web page HTML into a text file as line by line filename = 'webpagecontents.txt'; urlwrite(URL,filename); C = textread(filename, '%s','delimiter', '\n');

The text I am searching for is then extracted from the 16th line as msgbox(C(16))

Now the user interacts with the web page and the status message changes on the actual web page However, I am unable to retrieve it in the text file. It still shows the same old content.

How do I get the updated status message? I right clicked the web page and viewed its source file after the status updation. No change is observed there also. Any help is appreciated

Answer 1

[Guillaume]

Two things:

1. When matlab requests the page from the server (using urlwrite or any other method), it's never going to see the changes generated by the user, the same way you wouldn't expect another user to see the changes you make locally. The only way for you to pick up the changes would be either to talk to the web browser the user is using (assuming it is possible) or to actually render the page in matlab and have the user interact with the page from matlab (assuming it is possible with the limited gui tools of matlab).

2. If, in the web browser, you can't see the changes to the source of the page when you interact with it this means that the page is manipulated using javascript. This only change the content of the page in memory and on screen. Again, just reading the source won't help you and you either need to ask the web browser or implement your own browser.

Basically, your assumption is wrong. The page source itself is not updated, it's only the in-memory representation that is changed.

[Madhu Kodappully]

Thanks for your answer.

I would like to go further on the statements "The only way for you to pick up the changes would be either to talk to the web browser the user is using (assuming it is possible) or to actually render the page in matlab and have the user interact with the page from matlab (assuming it is possible with the limited gui tools of matlab)"

Actually, I am opening the web page in MATLAB GUI using the following code hand = actxcontrol('Shell.Explorer.2',[5, 150, 5000, 575], gcf); Navigate2(hand, url);

What do you see on this? Is there a possibility now?

[Guillaume]

Yes, you would have to use whichever method of the browser object allows you to get to the in-memory document, possibly the Document property. From there, you can use DOM parsing to get to the element you want.

[Madhu Kodappully]

h = actxcontrol('Shell.Explorer.2',[5, 150, 5000, 575], gcf);

h object stays in the MATLAB GUI and I am navigating a URL there. I would like to read the web page updates after a user interacts with the web page.

I feel I can get this by writing eventhandle function for the Shell.Explorer.2.

There are many event handle functions for h which is available by typing h.events

Any thoughts?

[Guillaume]

I don't believe that you can get the page modification from the WebBrowser control events themselves, as none of them relate to in-memory page change. The only relevant event would be DocumentComplete which is fired when the page is fully loaded, but I don't believe it would detect subsequent changes.

What may work is to listen to some events of the Document property of the control.

Alternatively, you could just query at regular intervals the html element that you want to watch, something like:

h.Document.getElementbyId('some ID')
%or
h.Document.getElementbyName('some Name e.g. body')

I've never used the webbrowser control and the MSDN doc is hard to navigate, so I don't think I can help you further.

[Madhu Kodappully]

You are correct! It was in the document property. As I am using MATLAB, I changed the syntax accordingly.

I have used the following syntax to solve it:

variable=handle.Document.get.documentElement.outerHTML;

It now works really great! W2G