Ensemble Neural Network in Matlab
이전 댓글 표시
Dear All, I am creating ensemble neural network comprises 3 component neural networks (NNs) with different number of hidden neurons. There are 3 input neurons and 2 output neurons with 1 hidden layer where hidden neurons are varied set as 3, 2, and 1 for each component NN. The 3 input neurons correspond with on-site displacement measurement at 3 different locations while the 2 output neurons correspond with Young’s modulus (E) and horizontal to vertical stress ratio (k). Total training data is 39 and testing data is 10.
These are the testing results:
E = [77.78224 42.33231 59.33344 53.02192 49.52091 85.40332 56.09939 53.33929 56.81571 16.95089];
target E = [80 70 60 30 50 80 40 70 30 20];
error E = [-2.21776 -27.6677 -0.66656 23.02192 -0.47909 5.403321 16.09939 -16.6607 26.81571 -3.04911];
k = [2.104783 2.32816 2.943708 3.00457 2.723133 2.291231 3.009603 3.024749 3.021532 3.002399];
target k = [1 1.5 2 2 2.5 2.5 3 3 3.5 4];
error k = [1.104783 0.82816 0.943708 1.00457 0.223133 -0.20877 0.009603 0.024749 -0.47847 -0.9976];
error E = E – target E
The results are not good especially for k value.
I worried that the correlation between input and output is not that good; hence I checked using simple model. I used 3 inputs (x1, x2, x3) and 2 outputs (y, z) and they have these relationships: y = x1^3-x2^2+x3; z = 1/x1-1/x2^2+1/(x3^2)
Total training data is 10 and testing data is 5. But again the testing result is not good especially for z.
y = [2831.213 5304.791 6902.318 12844.01 24449.48];
target y = [2088 4758 6650 11896 24048];
error y = [743.2125 546.7907 252.318 948.0119 401.4825];
z = [-0.18233 -0.18192 -0.18165 -0.16924 -0.16051];
target z = [-0.00704 -0.013 -0.01013 -0.01226 -0.01565];
error z = [-0.17528 -0.16893 -0.17153 -0.15698 -0.14486]
Please advise how to improve the ensemble neural networks.
Below is the code of my neural network for your reference:
clc % clear the command window clear; rand('seed', 1); randn('seed', 1);
ensemblesize =3; maxepoch=30;
for runno=1:1:10
fprintf('ENN dda 3 different number of hidden nodes epoch = %g hidden node no = var', maxepoch);
fprintf('\n');
% training data set
E=[20 30 40 50 60 70 20 30 40 50 60 80 20 40 50 70 80 20 30 40 60 70 20 30 50 60 80 20 40 50 60 70 80 30 40 50 60 70 80]; k=[1 1 1 1 1 1 1.5 1.5 1.5 1.5 1.5 1.5 2 2 2 2 2 2.5 2.5 2.5 2.5 2.5 3 3 3 3 3 3.5 3.5 3.5 3.5 3.5 3.5 4 4 4 4 4 4];
ux1=[0.82 0.76 0.75 0.69 0.74 0.83 1.31 1.35 1.1 1.14 1.39 1.09 1.84 1.62 1.66 1.6 1.56 2 1.95 1.96 1.98 1.89 2.79 2.68 2.41 2.51 2.55 3.63 2.92 2.91 2.93 3 2.86 3.38 3.41 4.16 3.31 3.32 3.27]; ux2=[-0.83 -0.83 -0.79 -0.77 -0.77 -0.89 -1.3 -1.6 -1.28 -1.3 -1.6 -1.26 -1.93 -1.84 -1.91 -1.86 -1.78 -2.28 -2.29 -2.28 -2.27 -2.29 -2.88 -2.97 -2.78 -2.81 -2.92 -3.51 -3.44 -3.38 -3.43 -3.46 -3.36 -3.92 -3.97 -3.83 -3.97 -3.9 -3.89]; ux3=[-0.53 -0.58 -0.53 -0.53 -0.52 -0.62 -0.9 -1.11 -0.86 -0.88 -1.09 -0.85 -1.39 -1.26 -1.32 -1.27 -1.2 -1.56 -1.6 -1.61 -1.55 -1.59 -2.06 -2.1 -1.89 -1.94 -2.06 -2.5 -2.44 -2.35 -2.38 -2.38 -2.36 -2.75 -2.75 -2.65 -2.74 -2.72 -2.72]; ux4=[-0.36 -0.39 -0.37 -0.36 -0.36 -0.42 -0.61 -0.81 -0.6 -0.61 -0.78 -0.6 -0.96 -0.89 -0.94 -0.9 -0.86 -1.12 -1.14 -1.14 -1.11 -1.12 -1.46 -1.5 -1.37 -1.42 -1.5 -1.76 -1.72 -1.69 -1.7 -1.74 -1.69 -1.97 -1.99 -1.9 -1.94 -1.94 -2]; ux5=[-0.25 -0.27 -0.25 -0.25 -0.25 -0.3 -0.42 -0.57 -0.42 -0.43 -0.56 -0.41 -0.69 -0.61 -0.66 -0.63 -0.6 -0.79 -0.81 -0.79 -0.78 -0.79 -1.02 -1.06 -0.96 -1 -1.05 -1.25 -1.21 -1.19 -1.19 -1.24 -1.19 -1.39 -1.42 -1.35 -1.36 -1.37 -1.41]; ux6=[-0.17 -0.19 -0.17 -0.17 -0.17 -0.2 -0.28 -0.39 -0.29 -0.29 -0.38 -0.28 -0.48 -0.41 -0.46 -0.44 -0.41 -0.55 -0.57 -0.56 -0.54 -0.54 -0.7 -0.73 -0.66 -0.69 -0.73 -0.87 -0.85 -0.82 -0.83 -0.85 -0.82 -0.97 -1 -0.92 -0.95 -0.96 -0.96]; ux7=[-0.1 -0.11 -0.11 -0.11 -0.11 -0.13 -0.18 -0.25 -0.18 -0.17 -0.23 -0.16 -0.3 -0.25 -0.28 -0.26 -0.24 -0.33 -0.35 -0.34 -0.33 -0.33 -0.43 -0.46 -0.41 -0.43 -0.45 -0.54 -0.53 -0.5 -0.51 -0.53 -0.49 -0.61 -0.62 -0.55 -0.6 -0.59 -0.58]; ux8=[-0.04 -0.05 -0.05 -0.05 -0.05 -0.05 -0.07 -0.1 -0.07 -0.07 -0.1 -0.07 -0.12 -0.11 -0.12 -0.12 -0.1 -0.14 -0.14 -0.15 -0.14 -0.14 -0.17 -0.19 -0.17 -0.18 -0.19 -0.23 -0.23 -0.21 -0.22 -0.22 -0.21 -0.26 -0.26 -0.23 -0.26 -0.25 -0.25]; ux9=[-0.01 0 -0.01 0 -0.01 -0.01 -0.01 -0.02 -0.02 -0.02 -0.02 -0.02 -0.03 -0.02 -0.02 -0.02 -0.02 -0.03 -0.03 -0.03 -0.03 -0.03 -0.04 -0.04 -0.04 -0.04 -0.04 -0.05 -0.05 -0.04 -0.04 -0.04 -0.04 -0.05 -0.05 -0.05 -0.05 -0.05 -0.05];
uy10=[-2.83 -0.93 -0.55 -0.58 -0.47 -1.24 -2.56 -2.55 -0.54 -0.49 -2.75 0.93 -3.5 -0.36 -1.92 -1.23 0.54 -1.33 -0.74 -0.62 -0.46 -0.43 -2.64 -2.91 -0.5 -1.45 0.45 -2.47 -1.65 -1.48 -1.48 -1.85 -1.5 -1.57 -1.36 -1.57 -1.41 -1.47 -1.43]; uy11=[-2.68 -0.77 -0.37 -0.38 -0.31 -0.25 -2.4 -0.45 -0.35 -0.3 -0.14 1.07 -1.56 0.41 -0.57 -0.07 0.67 -1.09 -0.48 -0.36 -0.23 -0.2 -2.35 -0.21 -0.23 -0.02 1.95 0.13 0.2 0.21 0.13 0.36 0.25 0.17 0.2 -0.23 0.21 0.25 0.2]; uy12=[-2.65 -0.65 -0.27 -0.28 -0.23 -0.18 -2.34 -0.38 -0.27 -0.21 -0.08 1.09 -1.51 0.36 -0.46 -0.03 0.64 -1 -0.38 -0.21 -0.14 -0.09 -2.13 -0.17 -0.11 0.03 1.85 0.25 0.24 0.21 0.18 0.33 0.24 0.22 0.24 -0.1 0.25 0.28 0.23]; uy13=[-2.63 -0.57 -0.17 -0.21 -0.16 -0.13 -2.3 -0.29 -0.18 -0.15 -0.03 1.09 -1.53 0.31 -0.36 -0.04 0.6 -0.93 -0.26 -0.15 -0.06 -0.03 -1.91 -0.19 -0.04 0.08 1.46 0.3 0.23 0.21 0.19 0.29 0.24 0.22 0.24 -0.02 0.24 0.26 0.23]; uy14=[-2.64 -0.47 -0.05 -0.2 -0.09 -0.07 -2.28 -0.19 -0.11 -0.04 -0.03 1.05 -1.53 0.22 -0.22 0.03 0.46 -0.86 -0.18 -0.09 -0.02 0.01 -1.13 -0.2 0.01 0.07 1.01 0.28 0.17 0.17 0.15 0.22 0.18 0.18 0.18 0.02 0.18 0.2 0.17]; uy15=[-2.67 -0.38 -0.04 -0.07 -0.13 -0.05 -2.08 -0.09 -0.04 -0.08 0.02 0.82 -1.37 0.09 -0.09 0.07 0.21 -0.7 -0.08 -0.05 0.02 -0.01 -0.81 0 -0.03 0.05 0.41 0.26 0.08 0.08 0.07 0.11 0.08 0.08 0.1 0.02 0.1 0.1 0.1]; uy16=[-2.62 -0.34 0.01 -0.03 0.01 -0.05 -2.03 -0.02 -0.01 0.02 -0.05 0.71 -1.4 0.04 -0.05 0 0.09 -0.65 -0.01 -0.03 0.01 0 -0.64 -0.03 -0.01 0 0.03 0.23 0 0 0 0.01 0.01 0 0.01 -0.03 0 0.01 0];
% test data set
teE=[80 70 60 30 50 80 40 70 30 20]; tek=[1 1.5 2 2 2.5 2.5 3 3 3.5 4]; teux1=[0.77 1.16 1.61 1.54 2.25 2.02 2.48 2.53 2.92 3.56]; teux2=[-0.79 -1.31 -1.93 -1.78 -2.56 -2.29 -2.87 -2.89 -3.38 -3.91]; teux3=[-0.52 -0.88 -1.31 -1.22 -1.79 -1.61 -2.01 -2 -2.33 -2.79]; teux4=[-0.37 -0.62 -0.93 -0.89 -1.29 -1.12 -1.47 -1.43 -1.69 -2.02]; teux5=[-0.25 -0.44 -0.64 -0.62 -0.91 -0.77 -1.05 -1.01 -1.19 -1.41]; teux6=[-0.17 -0.3 -0.44 -0.43 -0.64 -0.53 -0.74 -0.68 -0.83 -0.97]; teux7=[-0.11 -0.18 -0.27 -0.27 -0.4 -0.33 -0.45 -0.42 -0.52 -0.59]; teux8=[-0.04 -0.07 -0.12 -0.11 -0.18 -0.14 -0.18 -0.18 -0.22 -0.27]; teux9=[-0.01 -0.02 -0.03 -0.02 -0.03 -0.02 -0.04 -0.04 -0.05 -0.05];
teuy10=[1.12 -0.97 -0.83 -1.49 -2.5 0.86 -1.69 -1.91 -1.93 -2.08]; teuy11=[1.27 -0.19 0.25 -1.2 0.08 1.71 -0.05 0.15 0.15 0.28]; teuy12=[1.33 -0.16 0.29 -1.02 0.1 1.66 0.02 0.17 0.19 0.29]; teuy13=[1.34 -0.13 0.23 -0.81 0.09 1.53 0.03 0.15 0.19 0.25]; teuy14=[1.31 -0.1 0.18 -0.61 0.08 1.25 0.03 0.1 0.16 0.17]; teuy15=[1.22 -0.02 0.04 -0.31 0.04 0.71 0.07 0.06 0.09 0.08]; teuy16=[1.11 0.01 0.01 -0.1 0 0.43 0.01 0 0 -0.03];
% generate the component neural networks
for i = 1:ensemblesize
% generate the component training sets
input = [ux1;ux2;uy10];
comptarget = [E;k];
testinput = [teux1;teux2;teuy10];
testtarget = [teE;tek];
if i==1 % hidden nodes=3
hiddenno1=3;
fprintf('ENN dda component NN 1 epoch = %g hidden node no=%g ', maxepoch, hiddenno1);
fprintf('\n');
net = newff(input,comptarget,hiddenno1,{'tansig' 'purelin'});
net.divideFcn='';
net.trainParam.epochs = maxepoch;
net.trainParam.goal = 0.0;
net.trainParam.mc=0.7;
net.trainParam.lr=0.05;
net.trainParam.show=NaN;
net = train(net,input,comptarget);
f1 = sim(net,input);
output1 = sim(net,testinput); % now 'output' stores the real-va
elseif i==2 % hidden nodes=2
hiddenno2=2;
fprintf('ENN dda component NN 2 epoch = %g hidden node no=%g ', maxepoch, hiddenno2);
fprintf('\n');
net = newff(input,comptarget,hiddenno2,{'tansig' 'purelin'});
net.divideFcn='';
net.trainParam.epochs = maxepoch;
net.trainParam.goal = 0.0;
net.trainParam.mc=0.7;
net.trainParam.lr=0.05;
net.trainParam.show=NaN;
net = train(net,input,comptarget);
f2 = sim(net,input);
output2 = sim(net,testinput);
elseif i==3 % hidden nodes=1
hiddenno3=1;
fprintf('ENN dda component NN 3 epoch = %g hidden node no=%g ', maxepoch, hiddenno3);
fprintf('\n');
net = newff(input,comptarget,hiddenno3,{'tansig' 'purelin'});
net.divideFcn='';
net.trainParam.epochs = maxepoch;
net.trainParam.goal = 0.0;
net.trainParam.mc=0.7;
net.trainParam.lr=0.05;
net.trainParam.show=NaN;
net = train(net,input,comptarget);
f3 = sim(net,input);
output3 = sim(net,testinput);
else
fprintf('wrong input--ensemble size ');
fprintf('\n');
end
end
enoutput_training= (f1+f2+f3)/ensemblesize; enoutput_tes= (output1+output2+output3)/ensemblesize;
fprintf('ENN dda 3 different number of hidden neurons epoch = %g hidden node no = var', maxepoch);
fprintf('\n');
mse_training = mse(enoutput_training - comptarget) ;
mse_tes = mse(enoutput_tes - testtarget) ;
fprintf('dda ENN simple average mse_training = %12.5g \n', mse_training );
fprintf('dda ENN simple average mse_testing = %12.5g \n\n', mse_tes );
% % fprintf(' dda E value = \n'); % % fprintf(' target = '); % % fprintf(1,' %12.4g', teE1(1,:)); % % fprintf(' \n '); % % % % fprintf(' simple enn = '); % % fprintf(1,' %12.4g',enoutput_tes(1,:)); % % fprintf(' \n '); % % % % fprintf(' better enn = '); % % fprintf(1,' %12.4g',enoutput_better(1,:)); % % fprintf(' \n\n ');
end % end of function
댓글 수: 1
Greg Heath
2013년 1월 23일
I don't understand why you are posting data for ux1 to ux9 and uy10 to uy16
when your input is only 3-dimensional.
채택된 답변
추가 답변 (0개)
카테고리
도움말 센터 및 File Exchange에서 Deep Learning Toolbox에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!