How can I effectively pre- process data?

I have an input-output dataset to be used in ANFIS modelling. I need to pre-process the data! I've already removed the outliers outside +/-3 standard deviation of the group mean! I've also employed the Moving Average Smoothing technique. The issue here is that the RMSE is still undesirably large! I need a more effective way to pre-process the data!

댓글 수: 4

Walter Roberson
Walter Roberson 2013년 9월 1일
Is there good reason to believe that there is a meaningful amount of "signal" in the data? An actual pattern that is recognizable to be trained on?
Yes! There is a significant amount of "Noise". It is noticeable from the plot. The system I'm trying to describe exhibit a high degree of non-linearity coupled with some errors incurred during measuring and recording! I discovered smoothing the data reduces the RMSE!
Walter Roberson
Walter Roberson 2013년 9월 1일
Perhaps you could give us an example plot or two? I would be interested in original data, then outlier-removed, then smoothed; and, of course, a line showing approximately what your ideal output would be.
I'm sorry for the delayed response! The two inputs are to be modeled separately with each of the outputs (Sugeno Fuzzy). Here are the download links for the data plots! Figure 1: shows the original data http://www.sendspace.com/file/vsghku Figure 2: shows the data after the outliers have been removed http://www.sendspace.com/file/suiwet Figure 3: plot after applying the moving average smoothing technique http://www.sendspace.com/file/1svdkd Figure 4: after scaling the smoothed dataset http://www.sendspace.com/file/1gs4yi

답변 (0개)

이 질문은 마감되었습니다.

제품

질문:

2013년 9월 1일

마감:

2025년 4월 11일

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by