how to use two separate files for ANN?

Hi, I would like to use two separate files for ANN.one file for training and the other file for test of data.Here is for the one input.

inputs = ii';
targets = f';
inputGap = ig;
targetGap = fg;
    hiddenLayerSize = j;
         net = fitnet(hiddenLayerSize);
   net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
        net.outputs{2}.processFcns = {'removeconstantrows','mapminmax'};
        net.divideFcn = 'dividerand';  
        net.divideMode = 'sample';  
        net.divideParam.trainRatio = 70/100;
        net.divideParam.valRatio = 15/100;
        net.divideParam.testRatio = 15/100;
        net.trainFcn = 'trainlm';  
        net.performFcn = 'mse';         
        % Train the Network
        [net,tr] = train(net,inputs,targets);
        outputs = net(inputs);
        errors = gsubtract(targets,outputs);
        performance = perform(net,targets,outputs);
         valTargets = targets  .* tr.valMask{1};
        testTargets = target  .* tr.testMask;
        trainPerformance = perform(net,trainTagets,outputs);
        testPerformance = perform(net,testTargets,outputs);
        out1.net = net;
        out1.inputs = inputs;
        out1.targets = targets;

[Greg Heath]

Insufficient information as to what you want to do.

Meanwhile

 GEH1. What is the purpose of inputGap?
GEH2. Sizes of inputs and targets?
GEH3. Why 4 <= H <= 19? 
GEH4: Why not initialize/store RNG(i,j) = rng for reproducibility?
GEH5. Why waste space with 9 fitnet default assignments?
GEH6. Can include outputs and errors in the output of train
GEH7: Why no 30x16 (i,j) summary of all that work?
GEH8: Why 0.7 for TestReg threshold? Corresponds to 
           a Rsquare of only sqrt(0.7) = 0.8367

Answer 1

[Greg Heath]

> Hi Greg,Thank you so much for your attention. 1- I wanted to fill some data that I missed during measurement by ANN and using some parameters which may impact on my target data. Inputgap are those data(8 parameters) that may be helpful to fill the gap.

Please do not refer to input variables and output variables as parameters. Parameters are entities that stay constant during the process:

outputs = f(inputs,parameters).

> 2- size of my input is 1525 data and size of gap is 666(target).

Still not clear: Apparently I need some physical insight:

 What are your inputs, parameters and targets?

size(targets) = [ 1  859]    
size(inputs)  = [ 8 1525]  
==> size(inputs) = [ 8 859] for net training?
    size(inputs) = [ 8 666] for estimating missing data?

> 3- I wanted to try several H. I think(I am not sure if it is correct)If I use from 4 to 2n+3 for H (which n is my 8 parameters) would be enough to get the high mse and R-squared .

Not sure where you got that estimate. If the target plot is relatively smooth you will need at least one hidden node for each local extremum.

 [ I   N ] = size(inputs)
 [ O N ]= size(targets)
 Ntrn = round(0.7*N)   % approx fitnet default
 Ntrneq = Ntrn*O         % No. training equations

% No. of unknown weights to estimate for H hidden nodes

 Nw = (I+1)*H+(H+1)*O 
 Ntrneq >= Nw when H <= Hub where
 Hub = floor( (Ntrneq-O)/(I+O+1)) % Hidden Node upper bound

>4. since I have 60 artificial gap scenarios to prove that ANN can be a good method to simulate my missing data I need to test R2 and TsetReg. Actually 0.7 was the highest amount that I think might get from ANN.

Correction: if output vs target slope is R = 0.7, then R2 is only 0.49.

If you have to set a goal, try R2 >= 0.995 i.e., mse(error) <= 0.005*var(targets,1) via

 net.trainParam.goal =  0.005*var(targets,1)  % 1-dim target

If the net can't do it your computer will not explode.

First try Ntrials = 10. If need be, you can always design more.

4- Honestly I thought it can give me the answer sooner!! But unfortunetly after running 8 months I could not get a good result for "mse" comparing to linear interpolation!! but r squared was a little bit better in ANN method. my data was for six years (2191 data which has 666 gaps)and because I could get a good result with running all of the data I have decided to use 4 years data for training and 2 years for testing.probably in this way I could get a good result.but unfortunately I don't know how to write the script for introducing one file (4 years)as a training and another file(2 years)as a testing to ANN. Thanks for your help and time.

It is as simple as

 savednet = save(net);
 ...
 loadednet = load(savednet);
 newoutput = loadednet(newinput)

Hope this helps

Thank you for fomally accepting my answer

Greg

[Rita]

Thank you so much Greg and Walter.

Answer 2

[Walter Roberson]

Instead of using 'dividerand' use dividind; see http://www.mathworks.com/help/nnet/ug/divide-data-for-optimal-neural-network-training.html for the divideParam field names to use.

The "inputs" would be the concatenation of the two files's data, and the "targets" would be the concatenation of the respective targets, and the ind for the training would be 1 : numberoftraining with the ind for test being ((1:numberoftest) + numberoftraining)

[Greg Heath]

Replace numberoftraining with numberofdesign where

 numberofdesign = numberoftraining + numberofvalidation

This can also be implemented using divideblock

Hope this helps.

Greg