how do i optimize weights of neural network using GA in MATLAB(need codes to achieve that), again how do i call this function that optimzes the neural network weights in GA GUI so i can work with it from the interface

i need to train a neural network(a feedforward network and also a narx network) using genetic algorithm.and i need to do this by using GA to search for weights that will make the network learn very well.

[Alisina Gharjestani]

Hi,have you developed a code for optimizing a narx network by using genetic algorithm?

[Greg Heath]

What have you found when you searched the NEWSGROUP and ANSWERS using

 genetic narx

and

 genetic narxnet

?

Answer 1

[Greg Heath]

When

 [ I N ] = size(input)
 [ O N ] = size(target)
  MSE00  = mean(var(target',1))% Reference MSE

 The number of available training equations is

Ntrneq = N*O

 For an I-H-O node topology the number of unknown weights is

Nw = (I+1)*H+(H+1)*O = (I+O+1)*H+1

 Obviously, 

 1. The "3" in the GA input is replaced by I+O+1

 2. The normalized mse calculation is

     NMSE = mean((target(:)-output(:)).^2)/MSE00

Of course there will be more complications if the performance measure has to be decomposed into class and trn/val/tst subsets.

Hope this helps.

Greg

[Donatus]

Thanks for your answer, but somethings not clear.H is no of hidden neuron which is ok,but what are the values of I and O(or how do i determine I and O) and what is N

[Greg Heath]

The first two equations yield the sizes of the input and target matrices. I and O are the dimensions of the input and target vectors, respectively. N is the total number of examples.

Answer 2

[Gurvinder Singh]

this code is valid for only one input. if i provide more than one input the code gives an error;

Index exceeds matrix dimensions.

Error in separatewb (line 34) iw{i,j} = reshape(wb(hints.iwInd{i,j}),...

Error in setwb (line 23) [b,IW,LW] = separatewb(net,wb,hints);

Error in mse_test (line 9) net = setwb(net, x);

Error in Untitled2>@(x)mse_test(x,net,inputs,targets)

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11) fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in makeState (line 47) firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));

Error in gaunc (line 40) state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in ga (line 371) [x,fval,exitFlag,output,population,scores] = gaunc(FitnessFcn,nvars, ...

Error in Untitled2 (line 24) [x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

Caused by: Failure in initial user-supplied fitness function evaluation. GA cannot continue.

how to run it for more than one input???

Answer 3

[Greg Heath]

I have posted a new MIMO GA code in the NEWSGROUP.

http://www.mathworks.com/matlabcentral/newsreader/view_thread/344888#944621

Hope this helps.

Thank you for formally accepting my answer

Greg

[thomas lass]

please can I have the modified matlab codes for the above problem.The code above have errors and cannot run on matlab 2012.

[Greg Heath]

List the errors and MATLAB comments.

Greg

[Harshith Bhat]

These are the errors Error in @(x)mse_test(x,net,inputs,targets)

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11) fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in makeState (line 47) firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));

Error in gaunc (line 40) state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in ga (line 356) [x,fval,exitFlag,output,population,scores] = gaunc(FitnessFcn,nvars, ...

Error in experiment (line 30) [x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

Caused by: Failure in initial user-supplied fitness function evaluation. GA cannot continue.

>>

[Terry Chen]

Dear Greg,

where can get this link : http://www.mathworks.com/matlabcentral/newsreader/view_thread/344888#944621

It seems disappear.

Terry

Answer 4

[Greg Heath]

Unfortunately, I lost my own NN GA code to a computer crash many years ago. I have not attempted to replace it. However, I have searched the NEWSGROUP and ANSWERS for examples combining the NN and OPTIMIZATION Toolboxes.

There is a post by the MATLAB SUPPORT TEAM that outlines the procedure for a SISO problem. However, I have not been successful in using it.

There is a more recent post by a user that successfully yields an answer called the Pareto Front. I haven't learned enough to go beyond that.

Try searching the NEWSGROUP and ANSWERS using

 mathworks support team neural genetic

and

 mathworks support team neural GA

Then

 neural genetic pareto

and

 neural GA pareto

and finally, just

 neural genetic

and

 neural GA

If you are successful, please bless us with your knowledge.

Hope this helps.

Thank you if you think this answer is worth a formal acceptance

Greg

[Donatus]

yea, thanks for your answer, i found some codes trying to explain it, but some things are not clear.My question: is the code for a MIMO or SISO network, again i want to know if this optimization is a kind of hybrid GA or is purely GA, because i tried it on the simple_fit dataset and it worked:here is the code from MATHWORK

The GA function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.

For a neural network, the weights and biases are a Mx1 vector. These may be optimized using GA.

A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as GA requires a function handle that only returns a scalar value.

The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

The following code example describes a script that sets up a basic Neural Network problem and the definition of a function handle to be passed to GA. It uses the above function to calculate the Mean Squared Error.

% INITIALIZE THE NEURAL NETWORK PROBLEM %

% inputs for the neural net

inputs = (1:10);

% targets for the neural net

targets = cos(inputs.^2);

% number of neurons

n = 2;

% create a neural network

net = feedforwardnet(n);

% configure the neural network for this dataset

net = configure(net, inputs, targets);

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

[Greg Heath]

 inputs =1:10; 
 targets = cos(inputs.^2)

is a RIDICULOUS example:

 plot(inputs,targets)

 targets = cos(inputs).^2

is no better.

The problem is in the inputs spacing. 16 points per period should do the trick.

Hope this helps.

Greg