What is the best training data set to train a Neural Network from the given two options?

Previously, I had a Neural Network with a small number of input-nodes (around 6) and I used to train it with a large dataset (around 1000 input/target pairs). The variation of data for each input-node was really high.

Because of that, I changed the NN structure and increased the number of input-nodes of the network so that the variation of data to input-nodes become low. It caused to decrease the input/target pairs and now it is around 50 (Maximum).

But still, I feel that my previous structure is more stable than the new one, since I could train it with a large data set and higher variations of data help to tackle any value in the testing phase.

Here again, I have summarized the two training data sets.

1). Variation of data is high but had a large amount of input/target pairs to train the network.

2). Variation of data is low but had a small amount of input/target pairs to train the network.

Really appreciate if someone can give an idea about this issue and help me to select the best option :)

Thanks and Regards, Dara

Answer 1

[Greg Heath]

Since it takes at least 2 sigmoids to create a local extremum for a smooth data curve, a lower bound for the number of hidden nodes is

 H <= 2 * NLE

where NLE is the number of local extrema.

Hope this helps.

Thank you for formally accepting my answer

Greg