neural network for classification feature extraction

I have read articles about feature extraction using neural networks, my understanding is that neural networks naturally extract high-order features based on the weights on the edges of the neural networks. Is it possible to extract these features from a patternnet or other matlab neural network implementations, and then use these features for other classifiers?

In particular, I am working with about 450 training examples, 13 classes and about 280 features, and I expect some combination of features (F1-F2)*(F3-90)/(F4-10) etc, to be very predictive of my class labels, but all of the feature extraction methods I have found only work for images and not general classification problems.

Any advice is appreciated

Answer 1

[Greg Heath]

The best nonlinear features are usually found from some knowledge of the physical or mathematical nature of the problem.

So, what are you trying to classify?

The next best techniques to try are linear and clustering.

 [ I N ] = size(input)       % [ 280 450 ]
 [ O N ] = size(target)      % [ 13  450 ] 
  Ntrn   = N-2*round(0.15*N  % 314
 Ntrneq  = Ntrn*O            % 4082

The largest problem appears to be the number of inputs. I would consider the following approaches.

 1. Subdivide the data into subsets of smaller input dimensions that won't choke your computer (trial and error definitely needed).
2. Standardize inputs via zscore or mapstd. Remove or modify outliers.
3. Use PLSregress to reduce input dimensions by determining effective linear combinations.
4. Consider discarding the subdominant inputs of the dominant linear combinations.
5. I'm not sure if stepwise and stepwise fit will work on classification problems with {0,1} unit vector targets. However, it is worth a try.
6. Once you have a manageable number of inputs, the use patternnet.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Lee]

Thanks Greg! This was very helpful. I got busy for a while with work and family but tonight I'm gonna try PLSgress tonight and possibly stepwife fit afterwards.

I'm trying to perform a biological classification and I have 13 classes. Currently I have found Neural Networks, SVMs and Random Forests to work well as classification models, but the all seem to make the same mistakes (theyre around 70-80% accurate but most of the mistakes are shared with all models) even with different feature selection sets (I'm already performing mRMR/Gini index search down to 30-80 features) so that's what drove me to want to perform feature extraction.

Thanks again and I'll let you know how your suggestions work!