Neural Network (NARX) performance interpretation - how small is small?
조회 수: 2 (최근 30일)
이전 댓글 표시
I have built a neural network to forecast currency pair prices (trend) a few steps ahead. How do I interpret the performance plot results? For instance, the error histogram, the NMSE (what is considered small for currency pairs? How do I assess if these metrics are optimal?
댓글 수: 0
답변 (1개)
Greg Heath
2017년 8월 19일
If you naively model a function with a straight line, the MSE is just the variance of the function and varies with the scale of the function.
In previous MATLAB posts I have used the notations
MSE00 = vart1 = mean(var(target'),1)
and use it as a standard for measuring MSE.
Using this NAIVE model as a comparative standard, the normalized MSE is defined by
NMSE = MSE/mean(var(target',1))
which is related to the statistical RSQUARE via
RSQUARE = 1 - NMSE
For reasonable models both NMSE and RSQUARE lie within the closed interval [ 0,1].
For most regression and classification models, I use a goal
NMSEgoal = 0.01 % Regression & Classification
However, in order to obtain that goal for CLOSED LOOP time series, I use the OPEN LOOP goals
NMSEgoal = 0.005 or 0.001
I don't know of anyone who complains when Rsquare >= 0.99 or equivalently, NMSE <= 0.01.
Hope this helps.
Thank you for formally accepting my answer
Greg
댓글 수: 0
참고 항목
카테고리
Help Center 및 File Exchange에서 Deep Learning Toolbox에 대해 자세히 알아보기
제품
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!