It's hard to define what a scene is. If the camera smoothly turns around 180 degrees is that a new scene?
See my attached example where I take the mean of all frames. You could examine the means and try to find abrupt changes in the means which could indicate a change in scene. Though it could also be the same scene where someone just put their hand over the lens for a short time.