ThingHTTP pulling MarketWatch data stops working during trading hours

Did a project that uses an ESP8266 and WS2812B, using ThingHTTP to pull MarketWatch data to display live stock symbols, trending prices, and trending percentages. Overall, everything works. I then discovered that during trading hours (09:30-16:00), all of the data displays on both the LED matrix and serial monitor as jumbled characters.

Am I missing some neccessary and obvious line of code that allows me to pull live data curing market hours?

Any direction is appreciated. Rather than including the ino code here, I'll just point to my public Github repo.

https://github.com/derekhuether/stockticker

Thank you in advance!

[Christopher Stapels]

Are you sure the ThingHTTP querry is working during biz hours? They might change the page during when the market is open so the xpath may change. Ive seen the xpath in your example change frequently, though not quite daily.

I definitetly appreciate you not pasting the code here - its a few lines.

I reccomend you write MATLAB code that reads the page 'webread(url)' , then parse, and then writes the value to a ThingSpeak channel. Then you can see what is going on over time. You can trigger the code to run regularly with Timecontrol.

[Derek Huether]

That was it!

MarketWatch does indeed change the page during market hours.

During market hours, it is listing it in a div "intraday__data".

Thanks!

Answer 1

[Christopher Stapels]

In case anyone stubles on this, Ill promote Derek's findings to an answer. During business hours, the xpath for the data changes, so you need dynamic code to get the value at all times. See the comments for more details.